How to scp mutiple files (using wild chars ? or *) in a single scp command

[ezhilarasan_1]

Hi Unix gurus,

I need a help from you guys,
I have backup (backtrack backup) files with names as given below

ssng0011dwk:/tmp$ls -1

datafile_1-0.25-09-2005.23:56:39-18218
datafile_2-0.25-09-2005.23:57:35-18362
datafile_3-0.25-09-2005.23:58:02-18480
datafile_4-0.25-09-2005.23:58:15-18506
datafile_5-0.25-09-2005.23:58:20-18527


I am trying to scp these files from another server as

schi6049ndb:/home/ezhilara$

scp ezhilara@ssng0011dwk:/tmp/datafile_3-0.25-09-2005.23:58:02-18480 .

It is working fine. but it asks password for each scp command
( whereas rcp is not asking for password, but now disabled rcp in our environment )

Suppose we have about 50 backup files, then I have to make 50 scp command for each file
and I have to monitor so closely to check whether current scp is over and is it waiting for password for next scp
It is practically very difficult.

So my requests are
------------------

1)
I want to hardcode my password in a file, and scp should take password from this file instead of waiting to enter password

I tried as below, but it is NOT working

scp ezhilara@ssng0011dwk:/tmp/datafile_3-0.25-09-2005.23:58:02-18480 . < /tmp/pwdfile

2)
I want to use wild characters ? or * to scp multiple files in a single command instead one scp command for each file

I tried as below, but it is NOT working

scp ezhilara@ssng0011dwk:/tmp/datafile_* .

scp ezhilara@ssng0011dwk:/tmp/datafile_[?]-0.25-09-2005.* .

Please reply me

Thanks and regards
Ezhil

[Muthukumar_5]

2) Unfortunatly, scp will not support wild cards as like rcp. However, you can copy a directory with it's files and sub directories as,


scp ezhilara@ssng0011dwk:/tmp/ .

1) I hope you have not setup ssh commands key sharing properly. You can use temporary file to keep password. SSH is more secured one on handling passwd.

You can use expect scripting to handle passwd like this.

hth.

[Muthukumar_5]

To resolve this problem, you can prefer ftp simply to do this as,

ftp -i -n <<-EOF
user
cd /tmp
lcd /tmp/
mget datafile_*
bye
EOF

You can put it in a script like,

USER="root"
PASSWD="PASSwd123"

ftp -i -n <<-EOF
user $USER $PASSWD
cd /tmp
lcd /tmp/
mget datafile_*
bye
EOF

hth.

[Muthukumar_5]

If you want security then prefer sftp also by making key generation and sharing. sftp will suppot wild card.

If security is not a matter then setup rcp (with available security) and do wild-card file copying.

hth.

[dirk dierickx · Accepted solution]

to avoid the password issue, you have to setup your public key.

a work around for scp and wildcards is the following:

for i in `ssh ls `
do
scp :$i
done

[Kasper Hedensted]

Hi Ezhil,

Regarding the wildcards, try and put "" around the path, like this:

scp ezhilara@ssng0011dwk:"/tmp/datafile_*" .

Regards,
Kasper

[Jeff Lightner_1]

1) sftp (uses same ssh setup as scp) can be used like ftp but with better security so no reason to go to ftp. You can do mput and mget with sftp like you can with ftp so can do the multiple files.

2) If this is something you're doing frequently you might want to explore the open source package, rsync. It is designed for transferring multiple files and more - it will keep directory contents in sync between two servers and there are even flag that will let you have it delete files from the target that are no longer on the source.

[Bob_Vance]

What systems are running on?
What versions of SSH/scp?
What were the errors that you received"?

Wild cards with 'scp' work just fine for me and always have:

hpux <--> hpux
linux <--> hpux


linux ## rm /tmp/x* ; mkdir /tmp/tmp
hpux ## rm /tmp/x* ; mkdir /tmp/tmp

linux ## touch /tmp/xyz
hpux ## touch /tmp/xmm

hpux ## scp -p linux:/tmp/x? /tmp/tmp
scp: /tmp/x?: No such file or directory

hpux ## scp -p linux:/tmp/x?? /tmp/tmp
xyz 00% 0 0.0KB/s 00:00

hpux ## scp -p linux:/tmp/x* /tmp/tmp
xyz 100% 0 0.0KB/s 00:00


linux ## scp -p hpux:/tmp/x? /tmp/tmp
scp: /tmp/x?: No such file or directory

linux ## scp -p hpux:/tmp/x?? /tmp/tmp
xmm 100% 0 0.0KB/s 00:00

linux ## scp -p hpux:/tmp/x* /tmp/tmp
xmm 100% 0 0.0KB/s 00:00


And same results for hpux<-->hpux.


bv

[ezhilarasan_1]

Hi,

Dirk Dierickx's suggestion help me for the issue of scp is not supporting wildcard characters.

Now I have logined as my userid and running scp command, it is not asking for password.

(Previous I logined as oracle userid without password, then tried scp using myid, it asked for password for each scp command).

Thanks for all of you.
And my special thanks to Dirk Dierickx.

Regards
Ezhil