System Administration
cancel
Showing results for 
Search instead for 
Did you mean: 

discovered an issue when performing a remote tr (translate) command on hpux 11v3.

Manuel Contreras
Regular Advisor

discovered an issue when performing a remote tr (translate) command on hpux 11v3.

recently discovered an issue when performing a remote tr (translate) command on hpux 11v3.

test results listed below (same test.file exists on both boxes)...

$ remsh goodBOX -n " head -1 test.file | cut -c60-69 | tr '[a-z]' '[A-Z]'>
OABMIS02


$ remsh badBOX -n " head -1 test.file | cut -c60-69 | tr '[a-z]' '[A-Z]'"
��hr02

without the tr:
$ remsh badBOX -n " head -1 165700047_20100728_095914 | cut -c60-69 "
OABMIS02


the above command works correctly when performed locally...


already checked the tr executable...same on both boxes:

> ll /usr/bin/tr
-r-xr-xr-x 1 bin bin 76508 Feb 15 2007 /usr/bin/tr*

> cksum /usr/bin/tr
3992625266 76508 /usr/bin/tr



largest difference between boxes is goodBOX has a 2007 quality pack, and the badBOX is 2008.

input is appreciated...
manny
14 REPLIES
Manuel Contreras
Regular Advisor

Re: discovered an issue when performing a remote tr (translate) command on hpux 11v3.

thought I would provide another example:

$ remsh badBOX "echo abCd | tr '[a-z]' '[A-Z]'"
ABbD

$ remsh goodBOX "echo abCd | tr '[a-z]' '[A-Z]'"
ABCD

Dennis Handly
Acclaimed Contributor

Re: discovered an issue when performing a remote tr (translate) command on hpux 11v3.

>the above command works correctly when performed locally.

In the same directory? Also, try using the absolute path to tr(1).

Since you are using '', it can't be filename completion that is expanding your strings.

It could possibly be your locale settings?
Manuel Contreras
Regular Advisor

Re: discovered an issue when performing a remote tr (translate) command on hpux 11v3.

I like the latest echo example, as it simplifies the test.

locally works fine:
> echo abCd | /usr/bin/tr '[a-z]' '[A-Z]'
ABCD


remotely does not :(
$ remsh badBOX "echo abCd | /usr/bin/tr '[a-z]' '[A-Z]'"
ABbD


does work on goodBOX:
$ remsh goodBOX "echo abCd | /usr/bin/tr '[a-z]' '[A-Z]'"
ABCD


both boxes are 11.31
Dennis Handly
Acclaimed Contributor

Re: discovered an issue when performing a remote tr (translate) command on hpux 11v3.

What happens if you invoke a remote script like:
#!/usr/bin/sh
env
echo "The tr output ====="
echo 'abCd' | /usr/bin/tr '[a-z]' '[A-Z]'
Manuel Contreras
Regular Advisor

Re: discovered an issue when performing a remote tr (translate) command on hpux 11v3.

$ remsh badBOX "/home/is/testuser/test.sh"
_=/usr/bin/env
LANG=en_US.iso88591
PATH=/usr/bin:/usr/ccs/bin:/usr/bin/X11:/usr/contrib/bin:/usr/local/bin:
LOGNAME=testuser
SHELL=/usr/bin/ksh
HOME=/home/is/testuser
PWD=/home/is/testuser
TZ=PST8PDT
The tr output =====
ABbD
$
Matti_Kurkela
Honored Contributor

Re: discovered an issue when performing a remote tr (translate) command on hpux 11v3.

Several possible causes:

1.) tr makes a wrong conversion on plain US-ASCII character "C", which would be pretty weird indeed.

2.) tr works OK, but something in the data transmission chain converts the uppercase C to lowercase b (or to something else that ultimately ends up displayed as lowercase b). Strange but possible, if your terminal/emulator includes key macro features.

3.) something else, more information needed.


To test (and possibly exclude) hypothesis 2) and to get more information, we need a way to express the characters without directly inputting them. The "printf" command and the ASCII code table (see "man 5 ascii") are helpful here.


Connection sanity check in remote -> local direction:

remsh badBOX 'printf "\\141\\142\\103\\144\\n"'

(output should be "abCd"; if it's something else, the output of badBOX is being modified on the way from the badBOX to the local display.)


Connection sanity check in local -> remote direction:
remsh badBOX "echo abCd | od -b"

(output should include "141 142 103 144 012" which is abCd + line termination character expressed in octal ASCII codes; if the codes are different, your input is being modified on the way from local to badBOX.)


Explicit remote tr sanity check:

remsh badBOX 'printf "\\141\\142\\103\\144\\n" | /usr/bin/tr "[a-z]" "[A-Z]" | od -b'

(output should include "101 102 103 104 012" which is ABCD + line termination character expressed in octal ASCII codes)

MK
MK
Manuel Contreras
Regular Advisor

Re: discovered an issue when performing a remote tr (translate) command on hpux 11v3.

To test (and possibly exclude) hypothesis 2) and to get more information, we need a way to express the characters without directly inputting them. The "printf" command and the ASCII code table (see "man 5 ascii") are helpful here.


Connection sanity check in remote -> local direction:

remsh badBOX 'printf "\\141\\142\\103\\144\\n"'

(output should be "abCd"; if it's something else, the output of badBOX is being modified on the way from the badBOX to the local display.)


$ remsh badBOX 'printf "\\141\\142\\103\\144\\n"'
abCd




Connection sanity check in local -> remote direction:
remsh badBOX "echo abCd | od -b"

(output should include "141 142 103 144 012" which is abCd + line termination character expressed in octal ASCII codes; if the codes are different, your input is being modified on the way from local to badBOX.)


$ remsh badBOX "echo abCd | od -b"
0000000 141 142 103 144 012
0000005




Explicit remote tr sanity check:

remsh badBOX 'printf "\\141\\142\\103\\144\\n" | /usr/bin/tr "[a-z]" "[A-Z]" | od -b'

(output should include "101 102 103 104 012" which is ABCD + line termination character expressed in octal ASCII codes)


$ remsh badBOX 'printf "\\141\\142\\103\\144\\n" | /usr/bin/tr "[a-z]" "[A-Z]" | od -b'
0000000 101 102 142 104 012
0000005


$ remsh goodBOX 'printf "\\141\\142\\103\\144\\n" | /usr/bin/tr "[a-z]" "[A-Z]" | od -b'
0000000 101 102 103 104 012
0000005




Matti, thanks for your Extended analysis.
Dennis Handly
Acclaimed Contributor

Re: discovered an issue when performing a remote tr (translate) command on hpux 11v3.

>MK: Connection sanity check in local -> remote direction:
>remsh badBOX "echo abCd | od -b"

A better output is hex, not octal: echo abCd | xd -tx1 -tc"

What architecture/model are the two boxes?

Perhaps the next check is to do all this on the badBOX in your home directory. Invoke the script with:
env -i ./script

#!/usr/bin/ksh

export _=/usr/bin/env
export LANG=en_US.iso88591
export PATH=/usr/bin:/usr/ccs/bin:/usr/bin/X11:/usr/contrib/bin:/usr/local/bin:
export LOGNAME=testuser
export SHELL=/usr/bin/ksh
export HOME=/home/is/testuser
export PWD=/home/is/testuser
export TZ=PST8PDT

echo 'abCd' | /usr/bin/tr '[a-z]' '[A-Z]' | xd -tx1 tc


Other next steps could be putting "abCd" in a file, rather than using echo:
/usr/bin/tr '[a-z]' '[A-Z]' < file | xd -tx1 tc
Raj D.
Honored Contributor

Re: discovered an issue when performing a remote tr (translate) command on hpux 11v3.

Manuel,

What happens if you try with ssh , instead of remsh :

$ ssh -q badBOX "echo abCd | tr '[a-z]' '[A-Z]'"

do you get ABCD , the correct output.
- may be some flaws over remsh services.


Hth,
Raj.
" If u think u can , If u think u cannot , - You are always Right . "
Matti_Kurkela
Honored Contributor

Re: discovered an issue when performing a remote tr (translate) command on hpux 11v3.

The connection sanity checks were OK, so it probably isn't a remsh problem.

$ remsh badBOX 'printf "\\141\\142\\103\\144\\n" | /usr/bin/tr "[a-z]" "[A-Z]" | od -b'
0000000 101 102 142 104 012
0000005

OK, so it definitely looks like a problem with tr. But on second thought, it might be a locale problem too.

One more test:

remsh badBOX 'echo abCd | LC_ALL=C tr "[a-z]" "[A-Z]"'

Setting LC_ALL=C makes all locale-specific processing fall back to POSIX standrd US-ASCII. If this makes the problem go away, then perhaps the locale files for your default locale en_US.iso88591 (located in /usr/lib/nls/loc/) are broken on badBOX.

Perhaps someone with root access has experimented with the "localedef" command and made a little mistake?

You might also copy /usr/bin/tr from goodBOX to /tmp/tr on the badBOX, then run the tests using /tmp/tr instead of /usr/bin/tr. If this fixes the problem, then it conclusively proves that badBOX's /usr/bin/tr is broken.

------

Dennis> A better output is hex, not octal: echo abCd | xd -tx1 -tc"

Maybe, assuming that you are already familiar with hex values of ASCII characters. I would have used it if printf(1) could accept hex escapes.

But as it can't, octal is unavoidable. And as I'm already potentially confusing people, having input in octal and output in hex would make it even harder to figure out what's going on if the reader is not already familiar with all the tools and concepts I used.

MK
MK
Dennis Handly
Acclaimed Contributor

Re: discovered an issue when performing a remote tr (translate) command on hpux 11v3.

>MK: it might be a locale problem too.

That's why I had the ENVs printed.

>You might also copy /usr/bin/tr from goodBOX to /tmp/tr on the badBOX

Manny said the cksum matched. But libc and the locale libs could be different.

>assuming that you are already familiar with hex values of ASCII characters.

That's why there is ascii(5) with octal and hex.

>octal is unavoidable.

You could put the chars in a file and do a hex dump on them to prove they're correct.
Manuel Contreras
Regular Advisor

Re: discovered an issue when performing a remote tr (translate) command on hpux 11v3.

your not going to believe the root cause:

# remsh badBOX -l testuser "echo abCd | tr '[a-z]' '[A-Z]'"
ABbD

# remsh badBOX -l testuser "unset LANG;echo abCd | tr '[a-z]' '[A-Z]'"
ABCD
#

thank you all...your input is always appreciated :)

Manuel Contreras
Regular Advisor

Re: discovered an issue when performing a remote tr (translate) command on hpux 11v3.

the LANG setting was the root cause for my remote tr issues.

# remsh badBOX -l testuser "echo abCd | tr '[a-z]' '[A-Z]'"
ABbD

# remsh badBOX -l testuser "unset LANG;echo abCd | tr '[a-z]' '[A-Z]'"
ABCD


the following setting does not exist on the goodBOX:

> cat /etc/rc.config.d/LANG
#!/sbin/sh
# @(#)B.11.31_LR
# Language preference. See lang(5), hpnls(5)
#
# LANG: Locale name
#
# Note: if using the default C locale, many commands will execute faster if
# LANG is not set.
#
#LANG=C
#export LANG
LANG=en_US.iso88591

Dennis Handly
Acclaimed Contributor

Re: discovered an issue when performing a remote tr (translate) command on hpux 11v3.

>the LANG setting was the root cause for my remote tr issues.
echo abCd | tr '[a-z]' '[A-Z]'

Ah. If you want to do this, you have to be in the American Nerd locale, not this goofy dictionary sort locale.

Basically what you asked for was:
a B b .. Z z converted to:
A AE a ... Z

See /usr/lib/nls/loc/src/en_US.iso1.src LC_COLLATE category.

From the man page, you probably want:
tr '[:lower:]' '[:upper:]'