topic Re: to ftp a file with same time stamp. in Operating System - Linux

to ftp a file with same time stamp.

viseshu — Wed, 20 Sep 2006 08:43:50 GMT

i need to get from a server(there are many files of different file types) through ftp, the recent file of each file type.When i try to ftp all the files and try to do this, its giving the present time stamp to the ftped files.So, can any one please help me in this regard
NOTE: need only recent file of particular file type. file type will be the first 3 letters.

Re: to ftp a file with same time stamp.

James R. Ferguson — Wed, 20 Sep 2006 08:54:28 GMT

Hi:

If you are using FTP to transfer files, then as you have seen, the files will be created on the receiving server with a current timestamp. If you want to replicate the timestamp of the file as it exists on the source server, I suggest that you perform an 'ls' of the file on the source server (with FTP); capture the timestamp information into a log file; and use the captured information to 'touch' your transferred file changing its 'mtime' accordingly.

Regards!

...JRF...

Re: to ftp a file with same time stamp.

Mel Burslan — Wed, 20 Sep 2006 08:54:51 GMT

Some newer versions of ftp servers support such preserving the timestamp option but as far as I know not the default ftp server which comes with hpux. You need to write a script on both sending and receiving sides to watch for incoming files and when sending these files, you will also send another file with the content of the timestamp of the original file. In turn, the receiving server's script will look at both files and set the time with a touch command.

Or you will relinquish control to the receiving side. Receptor then will issue modtime command before receiving the file and upon completing of transfer, it will touch the file with the desired timestamp.

Neither of these methods are easy but doable.

Hope this helps.

Re: to ftp a file with same time stamp.

Peter Godron — Wed, 20 Sep 2006 09:04:20 GMT

Hi,
and welcome to your first ITRC question.
Can you please confirm that you want to transfer only the last file of a certain filetype from serverA onto serrverB.
So on ServerA:
AAAfile1
AAAfile2
BBBfile1
BBBfile2
BBBfile3

where the first three letters would indicate the filetype and the files are created in cronological order file1, file2,file3.

You would only want to ftp AAAfile2 and BBBfile3,as they are the last created?

Quickest way would be to rsh to ServerB, do a ls -rt AAA* | tail -1, which gives you AAAfile2. Then rcp that file.
Repeat for ls -rt BBB* | tail -1.

Otherwise you are probably stuck with the ftp "newer" command.

Re: to ftp a file with same time stamp.

viseshu — Wed, 20 Sep 2006 22:05:05 GMT

hi Peter,
thnks for ur suggestion. But the problem being we dont know how many file types are present on that server!!!and i have to embed this ftp script in my shell script. So tail pipe(|)commands are not working and rsh is not working :( ...

and James thanks for ur nice idea.How can i redirect in an ftp session the output of ls command into a log?? and after redirecting, how can i match if there are so many files.?

thnks in advance

Re: to ftp a file with same time stamp.

Dennis Handly — Thu, 21 Sep 2006 01:37:19 GMT

If you want to preserve properties of files, you may want to consider tarring them up and then untar on the other side.

As to the keeping the last one, you may want to use sort -r u -k1.1,1.3 (the key is only the first 3 chars)

Re: to ftp a file with same time stamp.

viseshu — Thu, 21 Sep 2006 02:06:16 GMT

Dennis,
Just have a look at my ftp script embedded in my unix script and plz let me know how and where to use tar/untar commands.

ftp -niv <<-EOF |tee -a $FTPLOG
open $SERVER
user $USER $PASSWORD
cd $FIRST_DIR
mget *.txt
mdelete *.txt
bye
Ican do any scripting related to my server..i cantdo anything at source side.
Thanks in advacne

Re: to ftp a file with same time stamp.

Dennis Handly — Thu, 21 Sep 2006 02:35:53 GMT

(Sorry, I didn't notice you said get.)
So it looks like only the sort trick can be used to get the "last" files. You should copy the files to a temp directory, use the script to get the latest, move to the real directory, then remove the temp dir.

cd tmp-dir
mv $(ls | sort -r u -k1.1,1.3) real-dir/

This like Peter's suggestion but uses vector methods to get all at once.

Re: to ftp a file with same time stamp.

viseshu — Thu, 21 Sep 2006 03:46:41 GMT

Dennis,
we cant use mv,sort commmands in ftp script. And if i use that after i get all the files, that will b of no use as the time stamp will be same for all the files.

Re: to ftp a file with same time stamp.

Dennis Handly — Thu, 21 Sep 2006 04:15:37 GMT

>as the time stamp will be same for all the files

My assumption was as Peter said, the time was encoded into the name of the file. If not, you will have to script on the source side or use ls to capture the time.

>How can I redirect in an ftp session the output of ls command into a log?? and after redirecting, how can I match if there are so many files?

Ah, looking at the man page:
ls [remote-directory] [local-file]
mdir remote-files local-file

(read carefully about the prompting issue)
ls "-tog ." same4
output to local-file: same4? y

Once you have the sorted time in same4, you can then figure out which is last.

Re: to ftp a file with same time stamp.

Peter Godron — Thu, 21 Sep 2006 04:47:01 GMT

Hi,
can you please supply an actual example of some of the filetypes.

You only want to retrieve the latest generated file of each filetype from server A, correct ?

Do you want to retain the creation date/time of the file as on ServerA onto ServerB ?

Re: to ftp a file with same time stamp.

viseshu — Thu, 21 Sep 2006 04:58:11 GMT

Hi Peter/Dennis,
As dennis adviced, i have kept the files in a log file. But my problem now is im notknowing how to take the latesst file from that.
total 16
-rw-r----- 1 122 Sep 7 10:45 BUS01.PN000001.txt
-rw-r----- 1 129 Aug 18 09:52 VEC01.PN170899.txt
-rw-r----- 1 210 Aug 18 05:43 VEC01.PN170698.txt
-rw-r----- 1 154 Aug 18 05:43 VEC01.PN170598.txt
-rw-r----- 1 100 Aug 18 05:43 VEC01.PN170498.txt
-rw-r----- 1 210 Aug 18 05:35 VEC01.PN170699.txt
-rw-r----- 1 154 Aug 18 05:35 VEC01.PN170599.txt
-rw-r----- 1 100 Aug 18 05:35 VEC01.PN170499.txt
-rw-r----- 1 103 Sep 21 10:55 BUS01.PN000002.txt

This is the log and i want latest file of BUS and VEC type.this is just a sample there will be so many such types. Plz help me how to get this from the log??
Peter pls go thru this sample files.

Re: to ftp a file with same time stamp.

Dennis Handly — Thu, 21 Sep 2006 05:19:27 GMT

This list doesn't seem to be in time order. Did you use ls -tog?

If it was in time order, then you could do:
$ sort -u -k7.1,7.3 filelist
-rw-r-- 1 103 Sep 21 10:55 BUS01.PN000002.txt
-rw---- 1 100 Aug 18 05:35 VEC01.PN170499.txt

(The only possible problem with sort -u, it doesn't say which one it takes. This could be solved by using nl to number each line, then sort without -u with the two keys, then use awk to compare sequential lines and pick the biggest if the same partial key.)

Note you can't tell the difference between files modified in the same 60 second interval. But that should be handled by ls -t.

Piping this to awk '{print $7}' would give you the latest filenames.

Re: to ftp a file with same time stamp.

viseshu — Thu, 21 Sep 2006 05:28:41 GMT

Dennis, half understood...can you please eloborate/or pen down the commands on the same file?
total 16
-rw-r----- 1 103 Sep 21 10:55 BUS01.PN000002.txt
-rw-r----- 1 122 Sep 7 10:45 BUS01.PN000001.txt
-rw-r----- 1 129 Aug 18 09:52 VEC01.PN170899.txt
-rw-r----- 1 210 Aug 18 05:43 VEC01.PN170698.txt
-rw-r----- 1 154 Aug 18 05:43 VEC01.PN170598.txt
-rw-r----- 1 100 Aug 18 05:43 VEC01.PN170498.txt
-rw-r----- 1 210 Aug 18 05:35 VEC01.PN170699.txt
-rw-r----- 1 154 Aug 18 05:35 VEC01.PN170599.txt
-rw-r----- 1 100 Aug 18 05:35 VEC01.PN170499.txt

Re: to ftp a file with same time stamp.

Dennis Handly — Thu, 21 Sep 2006 05:59:16 GMT

Attached is a script that sorts on the two keys and then selects the first in the partial key.

Re: to ftp a file with same time stamp.

viseshu — Thu, 21 Sep 2006 06:13:00 GMT

Dennis,
I can hardly see the script in the attachment. There are so many ASCII symbols(jargons) in the txt file.Can you please paste ur script in the box itself???

and will your script give me the recent file of each file type???

Re: to ftp a file with same time stamp.

Peter Godron — Thu, 21 Sep 2006 06:14:13 GMT

Hi,
attached my version of a possible solution.

Once you have the answer you need, can you please also read:
http://forums1.itrc.hp.com/service/forums/helptips.do?#28

Re: to ftp a file with same time stamp.

Peter Godron — Thu, 21 Sep 2006 06:14:52 GMT

Missing the attachment ;-)

Re: to ftp a file with same time stamp.

Dennis Handly — Thu, 21 Sep 2006 06:30:44 GMT

>I can hardly see the script in the attachment. There are so many ASCII symbols(jargons) in the txt file.Can you please paste ur script in the box itself???

It's going to look the same but may have poor indentation.

>will your script give me the recent file of each file type???

Naturally. ;-)

Peter seems to have the ftp communication down, (but not taking advantage of (ls [remote-directory] [local-file])
I have just the sorting/selection here.

#!/usr/bin/ksh

# sorts filelist (-tog ouput) based on initial time order. If duplicates
# in first 3 bytes, pick first.
# -rw-r----- 1 103 Sep 21 10:55 BUS01.PN000002.txt
# Since can't trust sort -u -k, add line numbers.

FILE=filelist

nl -nrz $FILE | sort -k8.1,8.3 -k1n,1n | awk '
BEGIN { # prime for first line
getline
LAST=substr($8, 1, 3) # partial key
print $8 # filename
}
{
KEY=substr($8, 1, 3)
if (KEY != LAST) { # if new type
print $8 # filename
LAST=KEY # save key
}
next
} '

Re: to ftp a file with same time stamp.

viseshu — Thu, 21 Sep 2006 07:44:06 GMT

hi all,
thanks for your help. please let me keep this post open for 3 more days till i get some confirmations DENNIS i specially thank u for ur patience and great help.
Thnku very much PETER...