Harsh J

You may also interpret this article as ‘pushing files to multiple hosts’ or ‘scp-ing to multiple hosts’, or even ‘syncing files over multiple hosts’.

rdist is a wonderful tool for either of the above, and largely unpopular by certain standards. RDist can be found here (also installable as a package in many Linux distributions). Beginning to use rdist is a little overwhelming at first but easy once you get the syntaxes right.

As an example, imagine you have your user home directory ($HOME) as /home/lost across all hosts. Now, on your master/main/primary host (say master.lost.com) you have created a folder named ‘season_1‘. You want to propagate this folder and all its contents to your other hosts (imagine 5 of them, slave1 through slave5.lost.com).

The directory structure of ‘season_1‘ looks like:

season_1/
--jack.txt
--locke.txt
--deadguys/
  --artz.txt

If you’d like to copy the entire directory over to all the slave hosts, create a file ($HOME/distfile in my case) with the following contents:

SLAVES = ( slave1.lost.com slave2.lost.com slave3.lost.com slave4.lost.com slave5.lost.com )

# Lets push our season_1 for all slave hosts to see!
season1: (
    /home/lost/season_1
) -> ( ${SLAVES} )

That’s all you’d need if you want to transfer the entire tree to all the hosts. To begin the distribution, simply do the following and follow the output which gets printed:

rdist -f $HOME/distfile

That’s it, as simple as that. Now lets get a little complex.

You suddenly decide that you don’t want to let the slave hosts to see the deadguys folder. In this case you can do:

SLAVES = ( slave1.lost.com slave2.lost.com slave3.lost.com slave4.lost.com slave5.lost.com )

# Lets push our season_1 for all slave hosts to see!
season1: (
    /home/lost/season_1
) -> ( ${SLAVES} )
except /home/lost/season_1/deadguys;

As you’re about to finally begin the push, you get a request from a friend who’s connected to the network as well (friend.lost.com), asking you to push all your season_1 updates ALONE to him. And you also decide that slave4 should not get this push update. With these new rules you’d do:

SLAVES = ( slave1.lost.com slave2.lost.com slave3.lost.com slave4.lost.com slave5.lost.com )

# Lets push our season_1 for all slave hosts to see!
season1: (
    /home/lost/season_1
) -> ( ${SLAVES} ) - ( slave4.lost.com ) + ( friend.lost.com )
except /home/lost/season_1/deadguys;

Likewise, when you have a new folder season_2 needed to be pushed in the future, you edit your distfile as (notice the minus and plus added to destination):

SLAVES = ( slave1.lost.com slave2.lost.com slave3.lost.com slave4.lost.com slave5.lost.com )

# Lets push our season_1 for all slave hosts to see!
season1: (
    /home/lost/season_1
) -> ( ${SLAVES} ) - ( slave4.lost.com ) + ( friend.lost.com )
except /home/lost/season_1/deadguys;

season2: (
    /home/lost/season_2
) -> ( ${SLAVES} )

I hope you got a hang of the rdist destfile syntax by now. You can read its manual for more advanced options that allow notifications (mails), timestamp based pushing (::), and many other options you may need for bending around the corner and taking the shortcuts.

Note: You can also push incremental updates by issuing rdist again — only the changed or new files will be pushed, neat huh? Don’t bug me about the security and setup pass-phrase-less ssh keys on all hosts for hassle-free shoot-your-feet setups.