double vs single brackets in shell script tests

[Scott Lindstrom_2]

I have read everything I can get my hands on about this, but still cannot figure out when I should use single brackets and when I should use double brackets.

Here is the code snippet that raised this issue for me:

#!/usr/bin/sh

if [[ $DEVICE = 'ftp' ]]
then continue
fi
if [[ $USER = 'reboot' ]]
then continue
fi
if [[ $USER = 'root' ]]
then continue
fi
if [[ $USER = 'wtmp' || $USER < 'a' ]]
then break
fi
if [[ $USER = 'cduser' && $DEVICE = 'ftp' ]]
then continue
fi
if [[ $DEVICE = 'console' ]]
then continue
fi

When I trace the code I get:

+ [[ remshd = ftp ]]
+ [[ root = reboot ]]
+ [[ root = root ]]
+ [[ root = wtmp ]]
+ [[ root < a ]]
+ [[ root = cduser ]]
+ [[ remshd = console ]]

I would have expected the third line should have been true. (The trailing spaces are odd).

If I change some of the double brackets to single brackets as follows:

#!/usr/bin/sh

if [[ $DEVICE = 'ftp' ]]
then continue
fi
if [ $USER = 'reboot' ]
then continue
fi
if [ $USER = 'root' ]
then continue
fi
if [[ $USER = 'wtmp' || $USER < 'a' ]]
then break
fi
if [[ $USER = 'cduser' && $DEVICE = 'ftp' ]]
then continue
fi
if [ $DEVICE = 'console' ]
then continue
fi


Then the code runs fine:

+ [[ remshd = ftp ]]
+ [ root = reboot ]
+ [ root = root ]

But I don't understand why.

I did try to change all the double bracket tests to single brackets, but then the shell objected on the compound conditions.

I would very very grateful to anyone could clear up why this happens and perhaps explain in simpler terms than I have
been able to find on how to decide which form to use.

Scott

[Oviwan]

Hey

I guess double brackets is ksh style but you are using sh. check the man pages for each shell.

Regards

[A. Clay Stephenson]

The rules are rather simple and in general you should always use double brackets because they are internal to the shell and thus more efficient rather than invoking the external test command. Moreover, there are more tests available with the internal test. The only downside to the [[..]]'s is that not all shell recognize this construct but any modern shell does.

The main gotcha is that the syntax for the logical operators are different in the two version

if [[ "${A}" = "Hook" && "${B}" = "Smee" ]]
is equivalent to:
if [ "${A}" = "Hook" -a "${B}" = "Smee" ].

Similarly:
if [[ "${A}" = "Hook" || "${B}" = "Smee" ]]
is equivalent to:
if [ "${A}" = "Hook" -o "${B}" = "Smee" ].

It is always a very good idea to enclose the tested variables inside double quotes because should the instantiated variables happen to be null and not enclosed within quotes then a syntax error results. Moreover, without quotes white space is not preserver and that is probably your fundamental problem in your example.

[Scott Lindstrom_2]

Oviwan -

I have read the man page and unfortunately it makes no sense on this subject. It talks about word splitting, etc but does not have anything (at least that I understand) that helps me.

[Scott Lindstrom_2]

Clay -

Let me try quoting the variable names and see what happens. (That's just not something I am used to having to do in other languages I have used).

Scott

[Oviwan]

Hey again

man test is more detailed how clay mentioned.

Regards

[Dennis Handly]

>It talks about word splitting, etc but does not have anything (at least that I understand)

I think this is exactly your problem. See "Blank Interpretation". For [[ ]] the trailing blanks in $USER are not split.

Note also [[ = ]] is NOT a test for string equality but pattern matching.

Assuming that USER="root ", these are all true:
[[ $USER = r* ]]
[[ $USER = root* ]]

>Clay: you should always use double brackets because they are internal to the shell and thus more efficient

Using tusc, I seen no evidence that test(1) is executed for [ ], both are real shell builtins.

In fact, /usr/bin/test just invokes the Posix shell builtin.

>It is always a very good idea to enclose the tested variables inside double quotes

It appears you don't have to do that for [[ ]]. But still a very good idea.

BUT you must NOT use "" or "" quoting if you want pattern matching.

[Scott Lindstrom_2]

Clay -

I changed the third "if" as you suggested and still get the same results:

if [[ "${USER}" = 'root' ]]
then continue
fi


+ [[ remshd = ftp ]]
+ [ root = reboot ]
+ [[ root = root ]]

I also change the single quotes around 'root' to double quotes but this does not seem to have any effect:

if [[ "${USER}" = "root" ]]
then continue
fi


+ [[ remshd = ftp ]]
+ [ root = reboot ]
+ [[ root = root ]]

I am curious, the first if looks like this and seems to work OK:

if [[ $DEVICE = 'ftp' ]]
then continue
fi

+ [[ remshd = ftp ]]

(Note the lack of padding on the right of "remshd").

I can get this to work just by changing to single brackets, but am hoping this will be a good learning opportunity for ma and anyone else reading this thread.

Scott

[Scott Lindstrom_2]

Dennis:

>I think this is exactly your problem. >See "Blank Interpretaion". For [[ ]] the >trailing blanks in $USER are not split.

>Note also [[ = ]] is NOT a test for string >equality but pattern matching.

I guess I am being dense here, but I've never worked in a language where xxx did not equal xxx. Otherwise almost all the if statements I've written in my life would be false.

So what do I do if I want string equality and not pattern matching?

Scott

[Peter Nikitka]

Hi Scott,

then use [ .. ] - you know the side effects much better than in [[ .. ]] so leave it at old style (IMHO).

mfG Peter

[Dennis Handly]

>but I've never worked in a language where "xxx " did not equal "xxx".

Are you dating yourself, only COBOL does this?  (And all other languages are broken. ;-)

C doesn't. Fortran may. Pascal for PACs but not for string.

>Peter: so leave it at old style (IMHO).

You may want to add a comment saying why you did that and the fact you have evil trailing spaces.

Or you can remove them with:
$ USER=${USER-%% }

Hmm that didn't work! This will:
USER=$(echo $USER)

[A. Clay Stephenson]

... and moreover, I always want to be able to distinguish between "Dumb", " Dumb", "Dumb ", " Dumb ", and "Du mb".

... and because it is perfectly legal (if dumb) for filenames to contain white space, what you thought was one filename becomes two or more unless you quote. That is why quoting is so important. Tons of scripts will fail which process filenames unless one is very diligent about quoting --- especially if you happen to have PC clients and their stupid "folders".

Dennis, thanks for testing that []'s are no longer external. I was repeating something I learned from one of the shell guys several years ago at HPWorld.

[James R. Ferguson]

Hi:

> Dennis wrote: Or you can remove them with:
$ USER=${USER-%% }

Hmm that didn't work!

Perhaps you meant:

# cat ./mysh
#!/usr/bin/sh
USER="abc "
echo "was: $USER"|cat -etv
USER=${USER%% *}
echo "is : $USER"|cat -etv

# ./mysh
was: abc $
is : abc$

Regards!

...JRF...

[Dennis Handly]

>JRF: Perhaps you meant: USER=${USER%% *}

No, I tried that and it only "appears" to work.
But that is NOT a shell pattern. That is a RE.

That says look for blank then anything else. That isn't what we want but works since there can't be embedded blanks (the [...] won't work).

So the shell ${X%%} appears to be broken. Or we can't read. ;-)

[Scott Lindstrom_2]

>>but I've never worked in a language where >xxx did not equal xxx.

>Are you dating yourself, only COBOL does this?
>(And all other languages are broken. ;-)

>C doesn't. Fortran may. Pascal for PACs but not for string.

First of all - thanks to everyone for their answers. As stated before I can make this work but am trying to figure why things operate the way they do.

Dennis - I guess I am dating myself here, but I've never used a 3rd or 4th GL that the above test would be false. It's been a while since I did C, so I'm not sure there. Maybe strcmp hid the issue from me.

Clay - I certainly agree that I would not want "^dumb" to equal "dumb" or equal "du^mb" (^ being a space). But in my coding experience if someone keyed "dumb^" on an input screen/form it was expected it would equal the literal "dumb" in the program.

I guess this all shows I have a lot to learn about shell scripting :-)

Scott

[Dennis Handly]

>It's been a while since I did C, so I'm not sure there. Maybe strcmp hid the issue from me.

You don't have to be sure but it's my job. ;-)
7.21.4 Comparison functions, says it takes the difference in unsigned chars for the first two pair of chars that are different.

So strcmp(3) won't hide it and the NUL will be compared with the space.

[Dennis Handly]

(You do know you can assign points to every response. ;-)

[Dennis Handly]

Unfortunately you should have assigned at least 8 points to one response so a bunny appears. And you should close it so everyone knows you got a solution.

[Bill Hassell]

> But in my coding experience if someone keyed "dumb^" on an input screen/form it was expected it would equal the literal "dumb" in the program.

Well, that is not necessarily a desired behavior in every case. A programming language that automatically removes whitespace without an override (to retain the spaces) is a poor design, especially for text processing. Modern POSIX shells (Korn, HP POSIX, BASH, etc) handle this fairly well. When you use quotes around a variable, the result is a single string. Leave out the quotes and leading/trailing spaces are treated as whitespace or separators. Consider:

X=" a b "
echo $X
a b
echo $X | xd
0000000 6120 620a
0000004

(I am using xd to show the exact characters since the forums doesn't show whitespace -- ironic, eh?) 61=a, 20=space, 62=b So X contains leading, trailing and imbedded spaces. $X by itself shows the variable without separators. Change to:

echo "$X" | xd
0000000 2061 2062 200a
0000006

Now you see 20 61 20 62 20 which means space a space b space. So you have to be aware of the potential for leading or trailing spaces in variables depending on how they are assigned. For instance, the "read" shell construct will strip off leading spaces since (like awk) whitespace is a separator and not part of the string.

Another area where comparisons can be tricky are things like echo and print. If you look at these examples:

echo A | wc -c
2
echo A | xd
0000000 410a
0000002

echo A | wc -c
1
echo "A\c" | xd
0000000 4100
0000001

In these examples, using echo can produce two different string lengths because 0a (the linefeed character) is appended by default.

And to make things really versatile, you can change the IFS variable to something other than a space.