The Blog

random stuff.

Quick Tip: Automate RSync (or other interactive input) in Shell Script

So you want to automate rsync, or something else, that requires interactive input from a user, but you have no way to do it. I have a pretty simple solution that I came across, but had some issues getting to work correctly because of lack of documentation online. Because of that, I thought I’d share my solution in case any others are in a similar situation.

In my particular case, I was writing a script that pulled down a local copy of a website for archival purposes and sent the contents (un-tarred) to a remote repository. The contents needed to be un-tarred because a user would need to access them remotely with nothing more than SCP access to the remote end. The user would then have to sift through the documents archived to determine how the site was on a particular day. If the contents were tarred, I could’ve just FTPed the documents over, but since FTP cannot recursively put files, I had to use rsync.

With the built-in rsync command, you have the option of using a password file, but that option is only valid when rsync is run as a daemon. Outside of that, your only option is to generate crypto keys for the local and remote sites and send the data on without using a password at all (an example of that is over here at TechnologyPlus). That’s a great option except in my case where the remote end was hosted by rsync and was simply storage and provided no console access. What do you do in this case?

Well, here’s the steps to my solution:

  1. Install expect if it’s not already included with your OS. I think it’s included by default in most distros, but it wasn’t in Ubuntu 12.04 for me.
    • For RedHat/CentOS: yum install expect
    • For Debian/Ubuntu: apt-get install expect
  2. Use it in your shell script like so:
    • expect -c "spawn rsync -r LOCALFOLDER USER@REMOTEHOST:REMOTEFOLDER
      expect \"*?assword*\"
      send -- \"PASSWORDTOSEND\r\"
      send -- \"\r\"
      expect eof"

    The above code will spawn an rsync command and recursively copy things from the LOCALFOLDER to the REMOTEFOLDER after logging in as the USER.

  3. Enjoy the benefits of having automated something so you don’t have to waste your time doing it manually anymore 🙂

Keep in mind, this solution can be used with anything that requires interactive input from a user like a username or password or option specifications. The most popular use is spawning an SSH session and running commands on a remote machine. The solution is pretty simple, but, like I said, I was unable to find clear documentation of integrating expect into a shell script online so I figured I’d save someone the trouble. I hope this helps someone! If you have any questions, comments, concerns, etc. let me know in the comments.

Join in on the conversation!

I will not share your email address with anyone.

*