Command line options to xargs

[Sandman!]

Can somebody explain why xargs gives erroneous results depending on the order of the command line options and arguments. For example to list the files in a directory:

# ls -1 | xargs -i -n1 echo {}
{} file1.txt
{} file2.txt
{} file3.txt
{} file4.txt
{} file5.txt

Output of the above command is correct if the options to xargs are switched i.e.

# ls -1 | xargs -n1 -i echo {}
file1.txt
file2.txt
file3.txt
file4.txt
file5.txt

Could someone clarify why this is so?

[Marco A. · Accepted solution]

Hello,
What I know is that the -i and the echo option must interact together, if you want to "assume" the {}, then the -i option must be before the echo.

Let's see the following examples ..

root@marco139:/tmp/marco> ls -1 |xargs -i -n 1 -i echo {}
file1
file2
file3
file4
file5

root@marco139:/tmp/marco> ls -1 |xargs -i -n 1 -i -n 1 echo {}
{} file1
{} file2
{} file3
{} file4
{} file5

Ok.. another think is that you can put more than one -i in the same line, and the only one that really work is the one that is close to the echo or print.

this other 2 examples shows the relation between the -i and the echo ...

-------------------------------------------
root@marco139:/tmp/marco> ls -1 |xargs -n1 -i echo mar{}co
marfile1co
marfile2co
marfile3co
marfile4co
marfile5co

root@marco139:/tmp/marco> ls -1 |xargs -i -n1 echo mar{}co
mar{}co file1
mar{}co file2
mar{}co file3
mar{}co file4
mar{}co file5

I hope this helps ...

Marc'o

[Matti_Kurkela]

That's an interesting puzzle.

The combination of -i and -n1 is redundant, because -i forces the command to be run once per line of input anyway. On the manual page, the description of the options -I and -i is "Insert mode: command is executed for each line from standard input..."

"xargs -i echo {}" produces the correct kind of output, as expected.

"xargs -n1 echo {}" produces output just like your first example.

Apparently the options -i and -n1 override each other: if -n1 is on the command line after -i, it cancels the "insert mode" but makes xargs run the command with one line of input at a time. If -i is after -n1, -i takes effect and -n1 is ignored.

MK

[Marco A.]

That's correct Matti, I believe this is an application bug ...but that's the way I have used that before because that issue.

Regards,

Marco

[TwoProc]

Well,

In looking at this, it seems that it somehow thinks that "-n1" is THE ARGUMENT to the "-i" command.

Look at :
> ls -1 | xargs -i -n1 echo hello
hello file1.txt
hello file2.txt
...

Meaning that the "{}" is ignored in your test case because the "-i" has something after it that it *thinks* is going to defined as the new definition of the replacement string.

HOWEVER (counter-counter argument), if that's true - look at:
> ls -1 | xargs -i -n1 echo -n1
-n1 file1.txt
-n1 file2.txt
...

this means that the "-i" portion thought it was getting redefined as "-n1", but clearly it didn't like using it, maybe because of the standard globber parsing the "-" part as a switch? Maybe?

NO (counter-counter-counter): look at:
> ls -1 | xargs -i bb -n1 echo bb
xargs: bb not found


What? that's odd, so now I can't even use the the "-i" switch to redefine anything, cept maybe "-n1" which the "redefiner" code likes, but the "command executor" code doesn't. It doesn't seem to like anything as a "replacement" argument that it is willing to work with.

TDH (that didn't help)

John

[Sandman!]

I don't think that they are redundant options otherwise they would give the same output if used independently i.e.

# echo hello world | xargs -i echo {}
hello world

# echo hello world | xargs -n1 echo
hello
world

And if they are conflicting options then the man page for xargs(1) should reflect that. However it helps to know that others have similar experiences. Thank you for your help.

[Matti_Kurkela]

Sandman, you missed one other documented effect of the "insert mode" option:

# echo hello world | xargs -i echo {}
hello world

The man page says (emphasis mine):
-I (and -i): Insert mode: command is executed for each line from standard input, _taking the entire line as a single arg_, inserting it in initial-arguments for each occurrence of replstr.

When xargs is not in insert mode, the processing is different:

# echo hello world | xargs -n1 echo
hello
world

Again from the man page:
Arguments read in from standard input are defined to be contiguous strings of characters _delimited by one or more blanks, tabs, or new-lines_; empty lines are always discarded. Spaces and tabs can be embedded as part of an argument if escaped or quoted.

So, the "-i" option actually has three effects:

- it changes the way xargs parses the standard input (whitespace-delimited vs. strictly newline-delimited)

- it makes xargs construct the command differently (which is the primary effect)

- it optionally allows you to specify the string which should be replaced with a line of standard input.

I think you need lawyer-like reading skills to get all this information out of the man page if you don't know what to expect with xargs... but yes, the information _is_ there, that cannot be denied.

John Joubert's first example:
> ls -1 | xargs -i -n1 echo hello
hello file1.txt
hello file2.txt

The actual output is the same if "-i" is removed, so looks like -i is ignored.

If -i would take -n1 as argument, the output should have been:
hello
hello
because with -i, the output must go exactly where indicated and nowhere else.

John's second example:
> ls -1 | xargs -i -n1 echo -n1
-n1 file1.txt
-n1 file2.txt

Again, the -i is ignored and you get commands like "echo -n1 file1.txt" and "echo -n1 file2.txt".

John's third example is curious:
> ls -1 | xargs -i bb -n1 echo bb
xargs: bb not found

Xargs probably uses getopt() function to parse the options. (man 3 getopt)
HP-UX getopt() does not recognize optional arguments to options, so the options themselves and any mandatory arguments to them are parsed first.

If "insert mode" is in effect, the first remaining non-option string may be the string to be replaced, so it will be handled specially. But in the example, -n1 has overridden -i, so "insert mode" is not in effect and the special handling does not happen.

MK

[Sandman!]

Thank you all for the clarification.

[Sandman!]

.