- Community Home
- >
- Servers and Operating Systems
- >
- Operating Systems
- >
- Operating System - HP-UX
- >
- Re: script to check files on landing directory
Categories
Company
Local Language
Forums
Discussions
Forums
- Data Protection and Retention
- Entry Storage Systems
- Legacy
- Midrange and Enterprise Storage
- Storage Networking
- HPE Nimble Storage
Discussions
Discussions
Discussions
Forums
Discussions
Discussion Boards
Discussion Boards
Discussion Boards
Discussion Boards
- BladeSystem Infrastructure and Application Solutions
- Appliance Servers
- Alpha Servers
- BackOffice Products
- Internet Products
- HPE 9000 and HPE e3000 Servers
- Networking
- Netservers
- Secure OS Software for Linux
- Server Management (Insight Manager 7)
- Windows Server 2003
- Operating System - Tru64 Unix
- ProLiant Deployment and Provisioning
- Linux-Based Community / Regional
- Microsoft System Center Integration
Discussion Boards
Discussion Boards
Discussion Boards
Discussion Boards
Discussion Boards
Discussion Boards
Discussion Boards
Discussion Boards
Discussion Boards
Discussion Boards
Discussion Boards
Discussion Boards
Discussion Boards
Discussion Boards
Discussion Boards
Discussion Boards
Discussion Boards
Discussion Boards
Discussion Boards
Community
Resources
Forums
Blogs
- Subscribe to RSS Feed
- Mark Topic as New
- Mark Topic as Read
- Float this Topic for Current User
- Bookmark
- Subscribe
- Printer Friendly Page
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
04-26-2013 02:45 AM
04-26-2013 02:45 AM
script to check files on landing directory
Hi
I am trying to write a script to check for files that are arriving on a particular directory, but sometimes I get no output at all, because this files once they arrive they are picked up & processed therefore going to another directory. So how can I overcame this? This is what I wrote so far:
#!/bin/sh echo " " echo " Files arriving...." ls -lrt /data/ICTPRD/bmd1/rating/data | tail -2 echo " " echo " files processed..." ls -lrt /data/ICTPRD/bmd1/rating/processed | tail -2
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
04-26-2013 03:10 AM
04-26-2013 03:10 AM
Re: script to check files on landing directory
Assuming a separate process is responsible for depositing the files and processing them, your script obviously has an inherent race condition.
What exactly is the requirement?
Assuming that you want to monitor/list the files that have "arrived" since you last checked and that the file names do not change as a result of the processing, you can do something similar to:
$ diff -r /data/ICTPRD/bmd1/rating/data \ /data/ICTPRD/bmd1/rating/processed | \ sed -ne 's#Only in /data/ICTPRD/bmd1/rating/data: \(.*\)#\1#p'
HP-UX Compilers
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
04-26-2013 04:44 AM
04-26-2013 04:44 AM
Re: script to check files on landing directory
Hi
I dont quite understand your 3rd line, please can you explain, if yo dont mind
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
04-26-2013 04:50 AM
04-26-2013 04:50 AM
Re: script to check files on landing directory
The third line uses sed(1) to filter out the filenames from the recursive directory diff(1) that are present only in the "data" directory, as compared to the "processed" directory.
Since the assumption is that the "processing" is copying/moving files from the "data" directory to the "processed" directory, the directory diff would list out the files that have not been processed as yet.
Hope that explains.
HP-UX Compilers
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
04-26-2013 05:10 AM
04-26-2013 05:10 AM
Re: script to check files on landing directory
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
04-26-2013 05:48 AM
04-26-2013 05:48 AM
Re: script to check files on landing directory
>once they arrive they are picked up & processed therefore going to another directory
Why not fix the process that processes them to also give you a list of files?
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
04-26-2013 05:53 AM
04-26-2013 05:53 AM
Re: script to check files on landing directory
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
04-26-2013 05:59 AM
04-26-2013 05:59 AM
Re: script to check files on landing directory
@NDO wrote:
If that series of commands returns nothing means all files are the same on both directories?
That's right.
HP-UX Compilers
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
05-06-2013 01:33 PM
05-06-2013 01:33 PM
Re: script to check files on landing directory
Does your process preserve file time stamps? If not, you could use perl to find all files modified since the last run.
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
05-07-2013 02:07 AM
05-07-2013 02:07 AM
Re: script to check files on landing directory
Can you change the directory where the files arrive initially?
For example, if you could make the files to arrive into /data/ICTPRD/bmd1/rating/incoming instead of /data/ICTPRD/bmd1/rating/data, then your script could check the incoming directory and move them to the data directory once your script has done everything it wants to do. Then the application could pick them up in the data directory exactly as it does now.
With this change, you could be certain that no files will be missed, because the application would not even see the files in the data directory before your script had moved them there. But your script would become a necessary part of the overall process: if your script would fail, the application would not get its files at all.