tar | ssh: files disappear from the tar file on the backup server

[erinsaurus87]

Hey all,
So I'm using the following script to tar a copy of a sql database and send it using ssh over to my backup server, but although the tar file shows up on the backup server, the tar doesnt have the files inside it that it should have. If I tar the file and dont redirect the tar over to my backup server then the files show up in the tar on the local server.

#backup script for SOC projects filesystem/code and mysql database, to be executed weekly.
#################################
a=$(date +%T-%d_%m_%Y)
b=$(date +%m_%d_%Y)
#################################
echo "mkdir /home/erind/socprojects_backups/".$b
mkdir /home/erind/socprojects_backups/$b/
################################
#echo "copying /var/www/html/SocProjects to /home/erind/socprojects_backups/$b/SocProjects".$a
#cp -f -p -R /var/www/html/SocProjects /home/erind/socprojects_backups/$b/SocProjects.$a
###############################
chown -Rf erind /home/erind/socprojects_backups/$b
###############################
echo "dump socProjects sql database"
mysqldump socProjects > /home/erind/socprojects_backups/$b/socProjects.$a.sql
##############################
cd /home/erind/socprojects_backups/
echo "tar backup directory".$b
tar -cfpz /home/erind/socprojects_backups/$b | ssh erind@remoteserver "cat > socProjects.$b.tar"

Any idea why the files arent transferring over?

[leetuckert]

Re: tar | ssh: files disappear from the tar file on the backup server

I'm not sure why the pipe doesn't work as expected as I have never done anything like what you are doing; however, I would suggest you simply tar up the directory and scp the file over to the machine on the next line. I guarantee this will work properly.

[kubicle]

Re: tar | ssh: files disappear from the tar file on the backup server

Piping doesn't quite work like that.

Pipe takes the output of the previous command not the file it creates.
I'm also not quite certain that "cat > socProjects.$b.tar" does what you expect it to do (to me it looks like creating an empty file).

I agree with leetuckert, create the tar and then copy it over to the server (keep things simple).

[jlittle]

Re: tar | ssh: files disappear from the tar file on the backup server

I'm using the following script to tar ...

tar -cfpz /home/erind/socprojects_backups/$b | ...

You are missing a "-". That command tells tar to write to a file called "pz" (the leading - on the tar command seems to trigger this, I expect due to some historic compatibility).
What you're trying for is

tar czf - /home/erind/socprojects_backups/$b | ...

but for tasks like this, especially in a script, using GNU tar's long options is recommended, and avoids nasty gotchas (if you put the f at the end of the commands, and leave out the - as you did, and specify several files to archive, you overwrite the first file). (Also, my reading of the man page is that the -p option does not apply to creating archives.)

So prefer

tar --create --gzip --file=- /home/erind/socprojects_backups/$b | ...

Regards, John

[kubicle]

Re: tar | ssh: files disappear from the tar file on the backup server

@jlittle
Good tips, but I don't think creating the tar file is the key problem:

If I tar the file and dont redirect the tar over to my backup server then the files show up in the tar on the local server.

[doctordruidphd]

Re: tar | ssh: files disappear from the tar file on the backup server

tar -cfpz /home/erind/socprojects_backups/$b | ssh ----- "cat > socProjects.$b.tar"

agree, create the tar file then copy it. You might want to look into using sftp rather than an ssh pipeline.

Edit: sorry about the ip, just didn't register with me what it actually was.

[erinsaurus87]

Re: tar | ssh: files disappear from the tar file on the backup server

hi all thanks for your help, I think I will do the scp because I want to have the tar placed in a directory other than the ssh user's home directory and I dont know how to do that with the command I have now. The reason I liked the combined tar to ssh transfer was because I didnt have to add another step of deleting the tar on the local server after a copy is sent to the remote server.

[kubicle]

Re: tar | ssh: files disappear from the tar file on the backup server

I took another look at it...and I take back what I said previously, you should be able to get it to work with piping. Like jlittle already suggested, try with something like:

tar -cf - /home/erind/socprojects_backups/$b | ssh remote_machine "cat > /path/to/socProjects.$b.tar"
(You can use -czf (or -cjf) if you wish to compress the tarball.)

btw, it might not be a good idea to write your ssh username and ip address (if those are the real ones you use) on public forums, especially if the server is publicly accessible.

I still think it usually pays off to keep things as simple as possible, but you should be able to get it to work, I managed to get it working on my end.

[erinsaurus87]

Re: tar | ssh: files disappear from the tar file on the backup server

oh whoops, yeah i usually pull that info when i post on forums, missed it this time

[kubicle]

Re: tar | ssh: files disappear from the tar file on the backup server

oh whoops, yeah i usually pull that info when i post on forums, missed it this time

Been there, done that

Seems doctordruidphd already quoted the info, hopefully he (or a mod) can mask that as well.