- Community Home
- >
- Servers and Operating Systems
- >
- Operating Systems
- >
- Operating System - HP-UX
- >
- Trying to capture dd and pv output in a file
Categories
Company
Local Language
Forums
Discussions
Forums
- Data Protection and Retention
- Entry Storage Systems
- Legacy
- Midrange and Enterprise Storage
- Storage Networking
- HPE Nimble Storage
Discussions
Forums
Discussions
Discussions
Discussions
Forums
Discussions
Discussion Boards
Discussion Boards
Discussion Boards
Discussion Boards
- BladeSystem Infrastructure and Application Solutions
- Appliance Servers
- Alpha Servers
- BackOffice Products
- Internet Products
- HPE 9000 and HPE e3000 Servers
- Networking
- Netservers
- Secure OS Software for Linux
- Server Management (Insight Manager 7)
- Windows Server 2003
- Operating System - Tru64 Unix
- ProLiant Deployment and Provisioning
- Linux-Based Community / Regional
- Microsoft System Center Integration
Discussion Boards
Discussion Boards
Discussion Boards
Discussion Boards
Discussion Boards
Discussion Boards
Discussion Boards
Discussion Boards
Discussion Boards
Discussion Boards
Discussion Boards
Discussion Boards
Discussion Boards
Discussion Boards
Discussion Boards
Discussion Boards
Discussion Boards
Discussion Boards
Discussion Boards
Discussion Boards
Community
Resources
Forums
Blogs
- Subscribe to RSS Feed
- Mark Topic as New
- Mark Topic as Read
- Float this Topic for Current User
- Bookmark
- Subscribe
- Printer Friendly Page
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
03-27-2018 09:43 AM
03-27-2018 09:43 AM
Trying to capture dd and pv output in a file
Hello everyone.
I'm trying to capture all output of the following command to a file.
dd if=/dev/zero | pv | dd of=/xtra/dd_bench_1.img bs=512000k count=400000
Tried re-directing standard output but that didn't work.
dd if=/dev/zero | pv | dd of=/xtra/dd_bench_1.img bs=512000k count=400000 >/var/tmp/output.txt 2>&1
I'm sure I'm overlooking something simple.
Please advise.
Thank you.
- Tags:
- redirect
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
03-27-2018 05:58 PM
03-27-2018 05:58 PM
Re: Trying to capture dd and pv output in a file
> Tried re-directing standard output but that didn't work.
"didn't work" is not a particularly useful description of what
happened, or what you didn't like about it.
Given that you have a pipeline of three commands (dd, pv, dd), and
only the last stage has any redirection, I'll guess that you're seeing
output to stderr from the first two stages. You might try using a
sub-shell to collect the output from all the stages. For example:
( dd blah1 | pv | dd blah2 ) > file 2>&1
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
03-28-2018 08:58 AM
03-28-2018 08:58 AM
Re: Trying to capture dd and pv output in a file
Hello Steven.
Thanks for replying.
I apologize.
You're right, my comment "didn't work" is rather vague.
Specifically, when we run this command without redirecting anything it writes the following to stdout (the screen).
# dd if=/dev/zero | pv | dd of=/xtra/dd_bench_1.img bs=512000k count=400000
0444MiB 0:00:19 [23.4MiB/s] [ <=> ]
+400000 records in.
0+400000 records out.
We want ALL of that output, especially the MiB/s data, to be written to a file.
When we try your suggestion, something like this:
# (dd if=/dev/zero | pv | dd of=/xtra/dd_bench_1.img bs=512000k count=400000) > /var/tmp/output.file 2>&1
Only the "records in" and "records out" data is written to the output file /var/tmp/output.file as shown here:
# cat /var/tmp/output.file
0+400000 records in.
0+400000 records out.
dd: 0511-041 The process was killed by signal number 13.
The MiB/s data is discarded.
How do we write that MiB/s data to a file as well as the "records in" and "records out" info.
Please let me know if you have any suggestions.
Thanks again for replying.
-Tom
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
03-28-2018 03:08 PM
03-28-2018 03:08 PM
Re: Trying to capture dd and pv output in a file
> 0444MiB 0:00:19 [23.4MiB/s] [ <=> ]
I hate to sound ignorant, but, what produces that? Also, what's
"pv"? All I expect to see from "dd" are the "records in" and "records
out" messages. Are you using some fancy "dd", or is that stuff coming
from "pv" (whatever that is), or what?
type dd
There are programs whose behavior depends on whether stdout/stderr is
a terminal. Redirect them, and you get different results.
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
03-28-2018 03:40 PM
03-28-2018 03:40 PM
Re: Trying to capture dd and pv output in a file
Hello again.
The dd command is standard.
# type dd
dd is /usr/bin/dd
The pv command shows the progress of data going through a pipeline.
That's the MiB output.
My organization has us benchmarking disk I/O (as well as CPU metrics) both before and after applying Unix OS spectre/meltdown fixes.
This dd command is being used to capture disk I/O benchmarking.
I'm sure there are better ways to do this but I'm just doing as instructed.
I'm trying to script this dd command and that's when I found I couldn't capture the pv output.
From the command line it's just fine, but trying to assign the dd/pv output to a file or variable does not work for me.
Any other suggestions are welcome.
Thanks.
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
03-28-2018 09:04 PM
03-28-2018 09:04 PM
Re: Trying to capture dd and pv output in a file
> The pv command shows the progress of data going through a pipeline.
Ok. I found the source, and built one. Its bouncing "<=>" thing
uses cursor positioning, which wouldn't look good when stored in a file.
Having only glanced at the source, I know nothing, but I'd bet a small
sum that if isatty(STDERR_FILENO) fails, then you won't see any of that
output, which was always intended for (only) an interactive user.
Certainly, "pv 2> file" seems to disable it.
> [...] trying to assign the dd/pv output to a file or variable does not
> work for me.
You can redirect stderr from "dd" without data loss, but, apparently,
the "pv" progress stuff is provided only to a user at a terminal.
That's not unusual; For example, Wget has a similar bouncing "<=>" thing
for an interactive user, but a different-format progress report for
redirected output.
> Any other suggestions are welcome.
Why not just time the command, and if you want a speed, divide the
byte count (which you're specifying) by the measured time? It's not as
if you'd be watching the "pv" output while it's running, which is what
that eye-candy was intended for.
man time
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
03-31-2018 06:13 PM - edited 03-31-2018 06:20 PM
03-31-2018 06:13 PM - edited 03-31-2018 06:20 PM
Re: Trying to capture dd and pv output in a file
As Steven said, pv seems to be detecting how it's run.
Perhaps script(1) will fool it and capture the output?
Note: Using pv and pipelines will kill performance. Pipes are limited to an 8 KB buffer, not your 512 MB writes.
> both before and after applying Unix OS spectre/meltdown fixes.
This is a waste of time on HP-UX, no X86-64 here, unless over NFS.
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
04-01-2018 01:25 PM
04-01-2018 01:25 PM
Re: Trying to capture dd and pv output in a file
Thanks again for the feedback.
I'm going to try the time command and divide the bytes by time of dd execution.
This is actually for an AIX server and not HP-UX.