- Community Home
- >
- Servers and Operating Systems
- >
- Operating Systems
- >
- Operating System - Linux
- >
- sort data into monthly directories
Categories
Company
Local Language
Forums
Discussions
Forums
- Data Protection and Retention
- Entry Storage Systems
- Legacy
- Midrange and Enterprise Storage
- Storage Networking
- HPE Nimble Storage
Discussions
Discussions
Discussions
Forums
Forums
Discussions
Discussion Boards
Discussion Boards
Discussion Boards
Discussion Boards
- BladeSystem Infrastructure and Application Solutions
- Appliance Servers
- Alpha Servers
- BackOffice Products
- Internet Products
- HPE 9000 and HPE e3000 Servers
- Networking
- Netservers
- Secure OS Software for Linux
- Server Management (Insight Manager 7)
- Windows Server 2003
- Operating System - Tru64 Unix
- ProLiant Deployment and Provisioning
- Linux-Based Community / Regional
- Microsoft System Center Integration
Discussion Boards
Discussion Boards
Discussion Boards
Discussion Boards
Discussion Boards
Discussion Boards
Discussion Boards
Discussion Boards
Discussion Boards
Discussion Boards
Discussion Boards
Discussion Boards
Discussion Boards
Discussion Boards
Discussion Boards
Discussion Boards
Discussion Boards
Discussion Boards
Discussion Boards
Discussion Boards
Discussion Boards
Discussion Boards
Community
Resources
Forums
Blogs
- Subscribe to RSS Feed
- Mark Topic as New
- Mark Topic as Read
- Float this Topic for Current User
- Bookmark
- Subscribe
- Printer Friendly Page
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
тАО04-10-2007 09:37 PM
тАО04-10-2007 09:37 PM
I have a file that lists all data in a directory ( about 2 years worth of data )
I need to search the list file and create a directory for each month and year ie
archive.
archive.outdata.0703
I have an idea how to do this however am stuggling getting the sytax.
I will then need to move the files in outdata to the relevant archive directory.
please any help will be much appreciated.
Thanks
Chris.
Solved! Go to Solution.
- Tags:
- date arithmetic
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
тАО04-10-2007 09:52 PM
тАО04-10-2007 09:52 PM
Re: sort data into monthly directories
Pl. clearify following,
1] I have a file that lists all data in a directory ( about 2 years worth of data )
What is the data format of the that 'lists all data in a directory', give sample.
2] I need to search the list file and create a directory for each month and year
list file is the same file that 'lists all data in a directory'
3] I will then need to move the files in outdata to the relevant archive directory.
What is 'outdata' ?
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
тАО04-10-2007 09:55 PM
тАО04-10-2007 09:55 PM
Re: sort data into monthly directories
archive.outdata.0703
Is this the contents of your data file? Or the resultant directory you want to create?
Is "archive.outdata.0703" is a directory with files you want to move?
>I will then need to move the files in outdata to the relevant archive directory.
Where is your list of files?
And you show some lines of your data file??
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
тАО04-10-2007 09:57 PM
тАО04-10-2007 09:57 PM
Re: sort data into monthly directories
1)What is the data format of the that 'lists all data in a directory', give sample.
the list files are produced from ls -l:
-rw-r--r-- 1 950 staff 43 05 Oct 2006 AST0000015
-rw-r--r-- 1 950 staff 43 17 Oct 18:42 AST0000016
2)list file is the same file that 'lists all data in a directory'
yes that is correct - the file that details every file to be moved to the newly created directory
3) What is 'outdata' ?
outdata is an example of the directory name where the files exist.
Thanks
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
тАО04-10-2007 10:11 PM
тАО04-10-2007 10:11 PM
Re: sort data into monthly directories
archive.outdata.0703
Is this the contents of your data file? Or the resultant directory you want to create?
--> this will be the resulying directory - the contents file is called outdata
Is "archive.outdata.0703" is a directory with files you want to move?
--> all files with March date stamp will be moved to archive.outdata.0703 directory once created by the script
Thanks
I am working through an awk book to attempt the correct syntax however not found the answer yet.
Thanks for the help
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
тАО04-10-2007 11:27 PM
тАО04-10-2007 11:27 PM
Re: sort data into monthly directories
go to the dir where ur dir is located
execute following script
----------------------------
set -vx
Cur_Year=`date +'%Y'`
for File_cur_year in `ls -lt|awk 'length($8) == 5 {print $9}'`
do
if [ ! -d ${File_cur_year} ]
then
File_cur_mon=`ll $File_cur_year|awk 'length($8) == 5 {print $6}'`
else
File_cur_mon=`ls -ld $File_cur_year|awk 'length($8) == 5 {print $6}'`
fi
if [ ! -d archive.outdata.${File_cur_year}${File_cur_mon} ]
then
mkdir archive.outdata.${Cur_Year}${File_cur_mon}
cp ${File_cur_year} archive.outdata.${Cur_Year}${File_cur_mon}
else
cp ${File_cur_year} archive.outdata.${Cur_Year}${File_cur_mon}
fi
done
for File_cur_year in `ls -lt|awk 'length($8) == 4 {print $9}'`
do
if [ ! -d ${File_cur_year} ]
then
File_cur_mon=`ll $File_cur_year|awk 'length($8) == 5 {print $6}'`
else
File_cur_mon=`ls -ld $File_cur_year|awk 'length($8) == 5 {print $6}'`
fi
if [ ! -d archive.outdata.${File_cur_year}${File_cur_mon} ]
then
mkdir archive.outdata.${Cur_Year}${File_cur_mon}
cp ${File_cur_year} archive.outdata.${Cur_Year}${File_cur_mon}
else
cp ${File_cur_year} archive.outdata.${Cur_Year}${File_cur_mon}
fi
done
------------------------------------
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
тАО04-10-2007 11:36 PM
тАО04-10-2007 11:36 PM
Re: sort data into monthly directories
Sorry one line to add in second for loop for
Cur_Year variable
----------------------
Cur_Year=`date +'%Y'`
for File_cur_year in `ls -lt|awk 'length($8) == 5 {print $9}'`
do
if [ ! -d ${File_cur_year} ]
then
File_cur_mon=`ll $File_cur_year|awk 'length($8) == 5 {print $6}'`
else
File_cur_mon=`ls -ld $File_cur_year|awk 'length($8) == 5 {print $6}'`
fi
if [ ! -d archive.outdata.${File_cur_year}${File_cur_mon} ]
then
mkdir archive.outdata.${Cur_Year}${File_cur_mon}
cp ${File_cur_year} archive.outdata.${Cur_Year}${File_cur_mon}
else
cp ${File_cur_year} archive.outdata.${Cur_Year}${File_cur_mon}
fi
done
for File_cur_year in `ls -lt|awk 'length($8) == 4 {print $9}'`
Cur_Year=`ll ${File_cur_year}|awk 'length($8) == 4 {print $6`
do
if [ ! -d ${File_cur_year} ]
then
File_cur_mon=`ll $File_cur_year|awk 'length($8) == 5 {print $6}'`
else
File_cur_mon=`ls -ld $File_cur_year|awk 'length($8) == 5 {print $6}'`
fi
if [ ! -d archive.outdata.${File_cur_year}${File_cur_mon} ]
then
mkdir archive.outdata.${Cur_Year}${File_cur_mon}
cp ${File_cur_year} archive.outdata.${Cur_Year}${File_cur_mon}
else
cp ${File_cur_year} archive.outdata.${Cur_Year}${File_cur_mon}
fi
done
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
тАО04-11-2007 12:44 AM
тАО04-11-2007 12:44 AM
Re: sort data into monthly directories
I will make a start with this syntax as I still require the month to be displayed as a numeric ie archive.outdir.0703
this indicates 07 being the year and 03 being the month.
cheers
Chris
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
тАО04-11-2007 01:53 AM
тАО04-11-2007 01:53 AM
SolutionUse perl to 'glob' through the files;
'stat' the files found for the last modification time in seconds
'localtime' to translate to year and month
'sprintf' to nicely format
mkdir if needed
rename
presto.
See working example below
Now adapt to your eact needs.
Hope this helps some,
Hein van den Heuvel (at gmail dot com)
HvdH Performance Consulting
$cat move.pl
use strict;
use warnings;
my $source = shift or die "Please provide source path";
my $target = shift;
$target = "archive" unless $target;
my %seen;
while (<$source/*>) {
my ($m,$y) = (localtime((stat)[9]))[4,5];
my $yymm = sprintf ("%s.%02d%02d", $target, $y%100, ++$m);
my $old = $_;
s/^$source/$yymm/;
# print "$m $y $old $_\n";
if (!exists($seen{$yymm})) {
mkdir $yymm unless -d $yymm;
$seen{$yymm}++;
}
rename $old, $_;
}
#-----------------------------------
$ touch -t 0102030405 a
$ touch -t 0105030405 b
$ touch -t 0505030405 c
$ touch -t 0505030405 d
$ ls -l
total 0
-rw-rw-rw- 1 hein 513 0 Feb 3 2001 a
-rw-rw-rw- 1 hein 513 0 May 3 2001 b
-rw-rw-rw- 1 hein 513 0 May 3 2005 c
-rw-rw-rw- 1 hein 513 0 May 3 2005 d
$ perl move.pl x
2 101 archive.0102 archive.0102/a
5 101 archive.0105 archive.0105/b
5 105 archive.0505 archive.0505/c
5 105 archive.0505 archive.0505/d
$ ls -l arc*
archive.0102:
total 0
-rw-rw-rw- 1 hein 513 0 Feb 3 2001 a
archive.0105:
total 0
-rw-rw-rw- 1 hein 513 0 May 3 2001 b
archive.0505:
total 0
-rw-rw-rw- 1 hein 513 0 May 3 2005 c
-rw-rw-rw- 1 hein 513 0 May 3 2005 d
- Tags:
- Perl
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
тАО04-11-2007 02:06 AM
тАО04-11-2007 02:06 AM