ADial-UpNetworkofUNIXTMSystems D.A.Nowitz M.E.Lesk ABSTRACT AnetworkofovereightyUNIX◯rcomputersystemshasbeenestablished usingthetelephonesystemasitsprimarycommunicationmedium.Thenetwork wasdesignedtomeetthegrowingdemandsforsoftwaredistributionand exchange.Someadvantagesofourdesignare: -Thestartupcostislow.Asystemneedsonlyadial-upport,butsystems withautomaticcallingunitshavemuchmoreflexibility. -Nooperatingsystemchangesarerequiredtoinstallorusethesystem. -Thecommunicationisbasicallyoverdial-uplines,however,hardwiredcom-municationlinescanbeusedtoincreasespeed. -Thecommandforsending/receivingfilesissimpletouse. Keywords:networks,communications,softwaredistribution,softwaremaintenance August18,1978 ADial-UpNetworkofUNIXTMSystems D.A.Nowitz M.E.Lesk 1.Purpose ThewidespreaduseoftheUNIXsystemritchiethompsonbstj1978withinBellLaboratories hasproducedproblemsofsoftwaredistributionandmaintenance.Aconventionalmechanismwas setuptodistributetheoperatingsystemandassociatedprogramsfromacentralsitetothevarioususers.Howeverthismechanismalonedoesnotmeetallsoftwaredistributionneeds.Remote sitesgeneratemuchsoftwareandmusttransmitittoothersites.SomeUNIXsystemsarethemselvescentralsitesforredistributionofaparticularspecializedutility,suchastheSwitchingControlCenterSystem.Othersiteshaveparticular,oftenlong-distanceneedsforsoftwareexchange; switchingresearch,forexample,iscarriedoninNewJersey,Illinois,Ohio,andColorado.Inaddition,generalpurposeutilityprogramsarewrittenatallUNIXsystemsites.TheUNIXsystemis modifiedandenhancedbymanypeopleinmanyplacesanditwouldbeveryconstrictingto delivernewsoftwareinaone-waystreamwithoutanyalternativefortheusersitestorespond withchangesoftheirown. Straightforwardsoftwaredistributionisonlypartoftheproblem.Alargeprojectmay exceedthecapacityofasinglecomputerandseveralmachinesmaybeusedbytheonegroupof people.Itthenbecomesnecessaryforthemtopassmessages,dataandotherinformationbackan forthbetweencomputers. Severalgroupswithsimilarproblems,bothinsideandoutsideofBellLaboratories,haveconstructednetworksbuiltofhardwiredconnectionsonly.dolottamashey1978bstjnetworkunix systemchessonOurnetwork,however,usesbothdial-upandhardwiredconnectionssothatser-vicecanbeprovidedtoasmanysitesaspossible. 2.DesignGoals Althoughsomeofourmachinesareconnecteddirectly,otherscanonlycommunicateover low-speeddial-uplines.Sincethedial-uplinesareoftenunavailableandfiletransfersmaytake considerabletime,wespoolallworkandtransmitinthebackground.Wealsohadtoadapttoa communityofsystemswhichareindependentlyoperatedandresistanttosuggestionsthatthey shouldallbuyparticularhardwareorinstallparticularoperatingsystemmodifications.Therefore,wemakeminimaldemandsonthelocalsitesinthenetwork.Ourimplementationrequires nooperatingsystemchanges;infact,thetransferprogramslooklikeanyotheruserenteringthe systemthroughthenormaldial-uploginports,andobeyingalllocalprotectionrules. Wedistinguish‘‘active’’and‘‘passive’’systemsonthenetwork.Activesystemshavean automaticcallingunitorahardwiredlinetoanothersystem,andcaninitiateaconnection.Passivesystemsdonothavethehardwaretoinitiateaconnection.However,anactivesystemcanbe assignedthejobofcallingpassivesystemsandexecutingworkfoundthere;thismakesapassive systemthefunctionalequivalentofanactivesystem,exceptforanadditionaldelaywhileitwaits tobepolled.Also,peoplefrequentlylogintoactivesystemsandrequestcopyingfromonepassive systemtoanother.Thisrequirestwotelephonecalls,butevenso,itisfasterthanmailingtapes. Whereconvenient,weusehardwiredcommunicationlines.Thesepermitmuchfastertransmissionandmultiplexingofthecommunicationslink.Dial-upconnectionsaremadeateither300 or1200baud;hardwiredconnectionsareasynchronousupto9600baudandmightrunevenfaster onspecial-purposecommunicationshardware.fraserspider1974ieeefraserchannelnetwork -2- Nowitz datamation1975Thus,systemstypicallyjoinournetworkfirstaspassivesystemsandwhenthey findtheservicemoreimportant,theyacquireautomaticcallingunitsandbecomeactivesystems; eventually,theymayinstallhigh-speedlinkstoparticularmachineswithwhichtheyhandlea greatdealoftraffic.Atnopoint,however,mustuserschangetheirprogramsorprocedures. Thebasicoperationofthenetworkisverysimple.Eachparticipatingsystemhasaspool directory,inwhichworktobedone(filestobemoved,orcommandstobeexecutedremotely)is stored.Astandardprogram,uucico,performsalltransfers.Thisprogramstartsbyidentifyinga particularcommunicationchanneltoaremotesystemwithwhichitwillholdaconversation. Uucicothenselectsadeviceandestablishestheconnection,logsontotheremotemachineand startstheuucicoprogramontheremotemachine.Oncetwooftheseprogramsareconnected, theyfirstagreeonalineprotocol,andthenstartexchangingwork.Eachprograminturn,beginningwiththecalling(activesystem)program,transmitseverythingitneeds,andthenasksthe otherwhatitwantsdone.Eventuallyneitherhasanymorework,andbothexit. Inthisway,allservicesareavailablefromallsites;passivesites,however,mustwaituntil called.Avarietyofprotocolsmaybeused;thisconformstothereal,non-standardworld.As longasthecallerandcalledprogramshaveaprotocolincommon,theycancommunicate.Furthermore,eachcallerknowsthehourswheneachdestinationsystemshouldbecalled.Ifadestinationisunavailable,thedataintendedforitremaininthespooldirectoryuntilthedestination machinecanbereached. TheimplementationofthisBellLaboratoriesnetworkbetweenindependentsites,allof whichstoreproprietaryprogramsanddata,illustrativesthepervasiveneedforsecurityand administrativecontrolsoverfileaccess.Eachsite,inconfiguringitsprogramsandsystemfiles, limitsandmonitorstransmission.Inordertoaccessafileauserneedsaccesspermissionforthe machinethatcontainsthefileandaccesspermissionforthefileitself.Thisisachievedbyfirst requiringtheusertousehispasswordtologintohislocalmachineandthenhislocalmachinelogs intotheremotemachinewhosefilesaretobeaccessed.Inaddition,recordsarekeptidentifying allfilesthataremovedintoandoutofthelocalsystem,andhowtherequestorofsuchaccesses identifiedhimself.Somesitesmayarrangetopermitusersonlytocallupandrequestworktobe done;thecallingusersarethencalledbackbeforetheworkisactuallydone.Itisthenpossibleto verifythattherequestislegitimatefromthestandpointofthetargetsystem,aswellastheoriginatingsystem.Furthermore,becauseofthecall-back,nositecanmasqueradeasanotherevenif itknowsallthenecessarypasswords. Eachmachinecanoptionallymaintainasequencecountforconversationswithother machinesandrequireaverificationofthecountatthestartofeachconversation.Thus,evenif callbackisnotinuse,asuccessfulmasqueraderequiresthecallingpartytopresentthecorrect sequencenumber.Awould-beimpersonatormustnotjuststealthecorrectphonenumber,user name,andpassword,butalsothesequencecount,andmustcallinsufficientlypromptlytoprecedethenextlegitimaterequestfromeitherside.Evenasuccessfulmasqueradewillbedetected onthenextcorrectconversation. 3.Processing Theuserhastwocommandswhichsetupcommunications,uucptosetupfilecopying,and uuxtosetupcommandexecutionwheresomeoftherequiredresources(systemand/orfiles)are notonthelocalmachine.Eachofthesecommandswillputworkanddatafilesintothespool directoryforexecutionbyuucpdaemons.Figure1showsthemajorblocksofthefiletransferprocess. FileCopy Theuucicoprogramisusedtoperformallcommunicationsbetweenthetwosystems.Itperformsthefollowingfunctions: -Scanthespooldirectoryforwork.

-3- Nowitz -Placeacalltoaremotesystem. -Negotiatealineprotocoltobeused. -Startprogramuucicoontheremotesystem. -Executeallrequestsfrombothsystems. -Logworkrequestsandworkcompletions. Uucicomaybestartedinseveralways; a)byasystemdaemon, b)byoneoftheuucporuuxprograms, c)byaremotesystem. ScanForWork Thefilenamesinthespooldirectoryareconstructedtoallowthedaemonprograms(uucico, uuxqt)todeterminethefilestheyshouldlookat,theremotemachinestheyshouldcallandthe orderinwhichthefilesforaparticularremotemachineshouldbeprocessed. CallRemoteSystem Thecallismadeusinginformationfromseveralfileswhichresideintheuucpprogramdirectory.Atthestartofthecallprocess,alockissetonthesystembeingcalledsothatanothercall willnotbeattemptedatthesametime. Thesystemnameisfoundina‘‘systems’’file.Theinformationcontainedforeachsystemis: [1]systemname, [2]timestocallthesystem(days-of-weekandtimes-of-day), [3]deviceordevicetypetobeusedforcall, [4]linespeed, [5]phonenumber, [6]logininformation(multiplefields). Thetimefieldischeckedagainstthepresenttimetoseeifthecallshouldbemade.The phonenumbermaycontainabbreviations(e.g.‘‘nyc’’,‘‘boston’’)whichgettranslatedintodial sequencesusinga‘‘dial-codes’’file.Thispermitsthesame‘‘phonenumber’’tobestoredatevery site,despitelocalvariationsintelephoneservicesanddialingconventions. A‘‘devices’’fileisscannedusingfields[3]and[4]fromthe‘‘systems’’filetofindanavailable devicefortheconnection.Theprogramwilltryalldeviceswhichsatisfy[3]and[4]untilaconnectionismade,ornomoredevicescanbetried.Ifanon-multiplexabledeviceissuccessfully opened,alockfileiscreatedsothatanothercopyofuucicowillnottrytouseit.Iftheconnectioniscomplete,thelogininformationisusedtologintotheremotesystem.Thenacommandis senttotheremotesystemtostarttheuucicoprogram.Theconversationbetweenthetwouucico programsbeginswithahandshakestartedbythecalled,SLAVE,system.TheSLAVEsendsa messagetolettheMASTERknowitisreadytoreceivethesystemidentificationandconversation sequencenumber.TheresponsefromtheMASTERisverifiedbytheSLAVEandifacceptable, protocolselectionbegins. LineProtocolSelection Theremotesystemsendsamessage Pproto-list whereproto-listisastringofcharacters,eachrepresentingalineprotocol.Thecallingprogram checkstheproto-listforalettercorrespondingtoanavailablelineprotocolandreturnsause-protocolmessage.Theuse-protocolmessageis -4- Nowitz Ucode wherecodeiseitheraonecharacterprotocolletteroraNwhichmeansthereisnocommonprotocol. GregChessondesignedandimplementedthestandardlineprotocolusedbytheuucptransmissionprogram.Otherprotocolsmaybeaddedbyindividualinstallations. WorkProcessing Duringprocessing,oneprogramistheMASTERandtheotherisSLAVE.Initially,thecallingprogramistheMASTER.Theserolesmayswitchoneormoretimesduringtheconversation. Therearefourmessagesusedduringtheworkprocessing,eachspecifiedbythefirstcharacterofthemessage.Theyare center;cl.Ssendafile,Rreceiveafile,Ccopycomplete,Hhangup. TheMASTERwillsendRorSmessagesuntilallworkfromthespooldirectoryiscomplete,at whichpointanHmessagewillbesent.TheSLAVEwillreplywithSY,SN,RY,RN,HY,HN, correspondingtoyesornoforeachrequest. Thesendandreceiverepliesarebasedonpermissiontoaccesstherequestedfile/directory. Aftereachfileiscopiedintothespooldirectoryofthereceivingsystem,acopy-completemessage issentbythereceiverofthefile.ThemessageCYwillbesentiftheUNIXcpcommand,usedto copyfromthespooldirectory,issuccessful.Otherwise,aCNmessageissent.Therequestsand resultsareloggedonbothsystems,and,ifrequested,mailissenttotheuserreportingcompletion (ortheusercanrequeststatusinformationfromthelogprogramatanytime). ThehangupresponseisdeterminedbytheSLAVEprogrambyaworkscanofthespool directory.IfworkfortheremotesystemexistsintheSLAVE’sspooldirectory,aHNmessageis sentandtheprogramsswitchroles.Ifnoworkexists,anHYresponseissent. AsampleconversationisshowninFigure2. ConversationTermination WhenaHYmessageisreceivedbytheMASTERitisechoedbacktotheSLAVEandthe protocolsareturnedoff.Eachprogramsendsafinal"OO"messagetotheother. 4.PresentUses Oneapplicationofthissoftwareisremotemail.Normally,aUNIXsystemuserwrites‘‘mail dan’’tosendmailtouser‘‘dan’’.Bywriting‘‘mailusg!dan’’themailissenttouser‘‘dan’’onsystem‘‘usg’’. Theprimaryusesofournetworktodatehavebeeninsoftwaremaintenance.Relativelyfew ofthebytespassedbetweensystemsareintendedforpeopletoread.Instead,newprograms(or newversionsofprograms)aresenttousers,andpotentialbugsarereturnedtoauthors.Aaron Cohenhasimplementeda‘‘stockroom’’whichallowsremoteuserstocallinandrequestsoftware. Hekeepsa‘‘stocklist’’ofavailableprograms,andnewbugfixesandutilitiesareaddedregularly. Inthisway,userscanalwaysobtainthelatestversionofanythingwithoutbotheringtheauthors oftheprograms.Althoughthestocklistismaintainedonaparticularsystem,theitemsinthe stockroommaybewarehousedinmanyplaces;typicallyeachprogramisdistributedfromthe homesiteofitsauthor.Wherenecessary,uucpdoesremote-to-remotecopies. Wealsoroutinelyretrievetestcasesfromothersystemstodeterminewhethererrorson remotesystemsarecausedbylocalmisconfigurationsoroldversionsofsoftware,orwhetherthey arebugsthatmustbefixedatthehomesite.Thishelpsidentifyerrorsrapidly.Foronesetof testprogramsmaintainedbyus,over70%ofthebugsreportedfromremotesiteswereduetoold software,andwerefixedmerelybydistributingthecurrentversion. Anotherapplicationofthenetworkforsoftwaremaintenanceistocomparefilesontwodifferentmachines.AveryusefulutilityononemachinehasbeenDougMcIlroy’s‘‘diff’’program -5- Nowitz whichcomparestwotextfilesandindicatesthedifferences,linebyline,betweenthem.huntmcilroyfileOnlylineswhicharenotidenticalareprinted.Similarly,theprogram‘‘uudiff’’compares files(ordirectories)ontwomachines.Oneofthesedirectoriesmaybeonapassivesystem.The ‘‘uudiff’’programissetuptoworksimilarlytotheinter-systemmail,butitisslightlymorecom-plicated. Toavoidmovinglargenumbersofusuallyidenticalfiles,uudiffcomputesfilechecksumson eachside,andonlymovesfilesthataredifferentfordetailedcomparison.Forlargefiles,thisprocesscanbeiterated;checksumscanbecomputedforeachline,andonlythoselinesthataredifferentactuallymoved. The‘‘uux’’commandhasbeenusefulforprovidingremoteoutput.Therearesomemachines whichdonothavehard-copydevices,butwhichareconnectedover9600baudcommunication linestomachineswithprinters.Theuuxcommandallowstheformattingoftheprintoutonthe localmachineandprintingontheremotemachineusingstandardUNIXcommandprograms. 5.Performance Throughput,ofcourse,isprimarilydependentontransmissionspeed.Thetablebelow showstherealthroughputofcharactersoncommunicationlinksofdifferentspeeds.Thesenumbersrepresentactualdatatransferred;theydonotincludebytesusedbythelineprotocolfordata validationsuchaschecksumsandmessages.Atthehigherspeeds,contentionfortheprocessors onbothendspreventsthenetworkfromdrivingthelinefullspeed.Therangeofspeedsrepresentsthedifferencebetweenlightandheavyloadsonthetwosystems.Ifdesired,operatingsystemmodificationscanbeinstalledthatpermitfulluseofevenveryfastlinks. center;ccnn.NominalspeedCharacters/sec.300baud271200baud100-1109600 baud200-850 Inadditiontothetransfertime,thereissomeoverheadformakingtheconnectionandloggingin rangingfrom15secondsto1minute.Evenat300baud,however,atypical5,000bytesourceprogramcanbetransferredinfourminutesinsteadofthe2daysthatmightberequiredtomaila tape. Trafficbetweensystemsisvariable.Betweentwocloselyrelatedsystems,weobserved20 filesmovedand5remotecommandsexecutedinatypicalday.Amorenormaltrafficoutofasinglesystemwouldbearoundadozenfilesperday. Thetotalnumberofsitesatpresentinthemainnetworkis82,whichincludesmostofthe BellLaboratoriesfull-sizemachineswhichruntheUNIXoperatingsystem.Geographically,the machinesrangefromAndover,MassachusettstoDenver,Colorado. Uucphasalsobeenusedtosetupanothernetworkwhichconnectsagroupofsystemsin operationalsiteswiththehomesite.ThetwonetworkstouchatoneBellLabscomputer. 6.FurtherGoals Eventually,wewouldliketodevelopafullsystemofremotesoftwaremaintenance.Conventionalmaintenance(asupportgroupwhichmailstapes)hasmanywell-knowndisadvantages. brooksmythicalmanmonth1975Therearedistributionerrorsanddelays,resultinginoldsoftwarerunningatremotesitesandoldbugscontinuallyreappearing.Thesedifficultiesareaggravatedwhenthereare100differentsmallsystems,insteadofafewlargeones. Theavailabilityoffiletransferonanetworkofcompatibleoperatingsystemsmakesitpossiblejusttosendprogramsdirectlytotheenduserwhowantsthem.Thisavoidsthebottleneckof negotiationandpackaginginthecentralsupportgroup.The‘‘stockroom’’servesthisfunctionfor newutilitiesandfixestooldutilities.However,itisstilllikelythatdistributionswillnotbesent andinstalledasoftenasneeded.Usersarejustifiablysuspiciousofthe‘‘latestversion’’thathas justarrived;alltoooftenitfeaturesthe‘‘latestbug.’’Whatisneededistoaddressbothproblems simultaneously:

-6- Nowitz 1.Senddistributionswheneverprogramschange. 2.Havesufficientqualitycontrolsothatuserswillinstallthem. Todothis,werecommendsystematicregressiontestingbothonthedistributingandreceiving systems.Acceptancetestingonthereceivingsystemscanbeautomatedandpermitsthelocal systemtoensurethatitsessentialworkcancontinuedespitetheconstantinstallationofchanges sentfromelsewhere.Theworkofwritingthetestsequencesshouldberecoveredinlowercounselinganddistributioncosts. Someslow-speednetworkservicesarealsobeingimplemented.Wenowhaveinter-system ‘‘mail’’and‘‘diff,’’plusthemanyimpliedcommandsrepresentedby‘‘uux.’’However,westillneed inter-system‘‘write’’(real-timeinter-usercommunication)and‘‘who’’(listofpeopleloggedinon differentsystems).Aslow-speednetworkofthissortmaybeveryusefulforspeedingupcounsel-ingandeducation,evenifnotfastenoughforthedistributeddatabaseapplicationsthatattract manyuserstonetworks.Effectiveuseofremoteexecutionoverslow-speedlines,however,must awaitthegeneralinstallationofmultiplexablechannelssothatlongfiletransfersdonotlockout shortinquiries. 7.Lessons Thefollowingisasummaryofthelessonswelearnedinbuildingtheseprograms. 1.Bystartingyournetworkinawaythatrequiresnohardwareormajoroperatingsystem changes,youcangetgoingquickly. 2.Supportwillfollowuse.Sincethenetworkexistedandwasbeingused,systemmaintainers wereeasilypersuadedtohelpkeepitoperating,includingpurchasingadditionalhardwareto speedtraffic. 3.Makethenetworkcommandslooklikelocalcommands.Ourusershavearesistanceto learninganythingnew:alltheinter-systemcommandslookverysimilartostandardUNIX systemcommandssothatlittletrainingcostisinvolved. 4.Aninitialerrorwasnotcoordinatingenoughwithexistingcommunicationsprojects:thus, thefirstversionofthisnetworkwasrestrictedtodial-up,sinceitdidnotsupportthevarious hardwarelinksbetweensystems.Thishasbeenfixedinthecurrentsystem. Acknowledgements WethankG.L.Chessonforhisdesignandimplementationofthepacketdriverandprotocol,andA.S.Cohen,J.Lions,andP.F.Longfortheirsuggestionsandassistance.$LIST$