scp command to exclude particular directory

[HP-UX_Ali]

Hi All,

Please adivce, i have to copy all directories & files under directory to remote server but wanna exclude 1 directory.

kindly confirm the scp command option.

else i am thinking of making 1 server as nfs & mount the fs & then start copy with cp.
but cp too doesnt have exclude option .

rsync is having but --exclude doest recognize the PATTERN directory name.

can any confirm the correct command & syntax.
for exclude sigle directory....

[Raj D.]

ALI,
scp do not have any include exclude option and that is why rsync can be use.

> rsync is having but --exclude doest recognize the PATTERN directory name.

What patern you are trying. To exclude any directory you can use the --exclude option with rsync.

You have to the exclude option with double quote that will match the regex. Also you must put the / in the directory you want to exclude:


Ex:

# rsync -avz --exclude=sapr1/ --exclude=saparch1/ source_host:/data/ target_host:/data/


Hth,
Raj.

[Raj D.]

Also check this out:
http://lists.samba.org/archive/rsync/2002-June/002970.html

[John Guster]

Normally one does in this situation is to make a tar of that directory, scp it over, untar it, then you can remove that one directory one doesn't want. Hope help.

[Dennis Handly]

>John: make a tar of that directory

Or use pax(1) or gnu tar and exclude that directory when you create the tarfile (or when you extract.)

[Steve Post]

What I do....

I have boxA
I want to copy over boxA:/opt/stuff to boxB:/mydir/stuff
I want to exclude subdirectory /opt/stuff/racerX
I want to exclude any file with the word BLUE in them.

on boxA
cd /opt
find ./stuff -print > /tmp/stufflist.txt
cd /tmp
edit text file ./stufflist.txt
remove subdirectory ./stuff/racerX
remove any lines with BLUE in them
save the file

Log into box B
cd to mydir
Just sit there.

Go back to box A.
--Now here comes the magic part--

cat ./stufflist.txt | cpio -odumc | /usr/bin/ssh \
-2 \
mylogin@boxB \
"cd /mydir/.
/usr/bin/cpio -idumc"



BUT! be careful. I'd start with just 2 or 3 files and directories.
SMALL ones.
Make sure it works.
Then move to larger things.


Now other guys... This has worked fine for me. But perhaps there is something wrong? Let me know.

Oh...
The find command will not follow links unless you tell it to. And I would NOT tell it to.

[Tingli]

You can use this:

scp -r $(ls | -e /directory_not_to_copy/d) remote:

[Tingli]

Sorry, I missed something.

scp -r $(ls | sed /directory_not_to_copy/d) remote:

[Raj D. · Accepted solution]

Ali, you can also use grep similarly to exclude multiple directory:

# scp -r $(ls | grep -v -e dir_to_exclude1 -e dir_to_exclude2 ) trg_host:/trg_dir/


Hth,
Raj.

[Steve Post]

So I guess this means you can do this

find /stuff > filelist.txt

vi filelist.txt

scp -r $( cat filelist.txt ) trg_host:/trg_dir/

[Dennis Handly]

>Steve: find ./stuff -print > /tmp/stufflist.txt
edit text file ./stufflist.txt
remove subdirectory ./stuff/racerX
remove any lines with BLUE in them

You can also use find's ! -name/-path and -name and -prune to eliminate them.

>cat ./stufflist.txt | cpio -odumc | ...
>scp -r $( cat filelist.txt ) ...

I'd leave out the evil cats:
cpio -odumc < stufflist.txt | ...
scp -r $( < filelist.txt ) ...

>be careful. I'd start with just 2 or 3 files and directories. Make sure it works.

Right.

[HP-UX_Ali]

Thanks to all of them.........

Raj.D

can i use below syntax..

# scp -pr * $(ls | grep -v -e dir_to_exclude1 -e dir_to_exclude2 ) trg_host:/trg_dir/

scp all of them except above 2 directories.

Please confirm...

regards

[Steven Schweda]

> can i use below syntax..

Probably. Will it do what you want? Seems
unlikely.

> scp -pr * $(ls | [...]

Why "*" _and_ all the ls+grep stuff?

> Please confirm...

What happens when you _try_ it? (What is my
opinion worth? Ask the computer.)

> >be careful. I'd start with just 2 or 3
> files and directories. Make sure it works.
>
> Right.

I'm with them.

[Raj D.]

Ali,

why the * before: as Steven said , thats correct, and test before actual run.

# scp -pr * $(ls | grep -v -e dir_to_exclude1 -e dir_to_exclude2 ) trg_host:/trg_dir #incorrect!#


This is not right When you specify * before the 'filtered listing' , it will generate the 'complete' list from your current directory.
Remove the * and you can use it.




Lets see the syntax:

SYNTAX: scp [options] [user@]host1:]file1 [user@]host2:]file2

or to make it simple it is just similar to copy command:
# scp [options]


Here: $(ls | grep -v -e dir_to_exclude1 -e dir_to_exclude2 ) will work as source file listing:


You can check by executing :
# echo $(ls | grep -v -e dir_to_exclude1 -e dir_to_exclude2)

- replace the exclude1 and 2 with your actual directory name. You will see the list. And all those scp will copy. Remeber this will not copy the hidden files under current directory. You can do a small scp test from a test directory and then run the actual copy you are intending to copy.



Hth,
Raj.

[Raj D.]

Ali,
Another way you can achieve the same thing:
And if you have large number of file/dir to exclude:

1. Generate the scp command list.
2. Edit with vi & Remove the lines not needed.
3. save the file, execute the file .






Here is is example:
-------------------
1. Generate the command list:

# ls -l | grep -v total |awk '{print "scp -pr " $9 " adms-desktop:/home/2/ ;printf " NR ":RC='\$\?' "}' > command_list


# ls -l | grep -v total |awk '{print "scp -pr " $9 " trg_host:/trg_dir/ ;printf " NR ":RC='\$\?' "}'
scp -pr 1 trg_host:/trg_dir/ ;printf 1:RC=$?
scp -pr abc trg_host:/trg_dir/ ;printf 2:RC=$?
scp -pr abcd trg_host:/trg_dir/ ;printf 3:RC=$?
scp -pr abcd1 trg_host:/trg_dir/ ;printf 4:RC=$?
scp -pr abcd23 trg_host:/trg_dir/ ;printf 5:RC=$?
scp -pr abcdEE trg_host:/trg_dir/ ;printf 6:RC=$?
....




2. Edit the list: & remove un-wanted:
# vi command_list

- /search_file_dir --> and delete the line
- save.




3. Excute:
# chmod +x command_list
# ./command_list


- Limitations: Any file/directory name in the current directory contains white space, it will not copy.
- Hidden files on current directory will not copy.
- The output will be on its way as the copy progress with Line_number:Result Code (RC=0). 0 means successful.


Hth,
Raj.

[Mohammed.Muneer]

Thanks Raj..

Your solution has worked. i have tried & successfull transferred.

Full marks... ;)

Dear Steven
i have asked .. because i was not in office & was no access to boxes.

But thanks for your time & consideration.

regards.

[HP-UX_Ali]

Thanks to Raj & Mohammed.

I too have tried & successful.

thanks to all of them for there time & consideration.

regards
Ali

[HP-UX_Ali]

I have found the solutions & hence closing the thread.