how to check encoding of a file on hp-ux

[Shan_2]

I would like to know if there is any hp-ux standard command / tools that can tell the encoding of a file, say, in unicode file format, or, in UTF-8 file format.

Many Thanks!
Shan

[Dennis Handly]

There is no simple tool that can do this. Text files are just strings of bytes. And you probably have to know the locale to be able to know the encoding.

You could easily know if the file was ASCII.

[Jose Mosquera]

Hi,

"iconv" command converts the encoding of characters in the input files from the fromcode code set to the tocode code set, and writes the results on standard output.
#man iconv

The full list of supported code sets is in /usr/lib/nls/iconv/config.iconv.

#iconv -f iso[nnn] -t utf8 current_file > final_utf8_file

Rgds.

[Shan_2]

Thanks Dennis for the quick response.

Thanks Jose for the suggestion of iconv.

My problem is,
I have program which will generate a file in UTF-8 format.

When this is run on machine A, the encoding of the output file is correct (i.e. UTF-8) and can be confirmed when this file is downloaded to Windows and open in Notepad.

However, when the same program run on machine B, the encoding of the output file is simply ASCII, and could only be verified after download to Windows and open in Notepad.

Thus, I would like to know if I can verify and confirm the file encoding on the unix machine. As I am afraid the encoding "may be" changed after the file downloaded (ftp) to PC.

Many Thanks!
Shan

[Dennis Handly]

>when the same program run on machine B, the encoding of the output file is simply ASCII,

Have you figured out the difference? Do you need to export LANG to set the locale?

[Jose Mosquera]

Hi again,

Yes, I agree with Dennis about the environmental setting on your machine B, so please check possible differences between them, specially LANG variable.
#env

Rgds.

[Shan_2]

Dear all,

Thanks for the suggested checking.

Machine A's PATH environment variable include /opt/langtools/bin, while this directory does not exist in Machine B!

Do you know which HP-UX software is missing on Machine B ?

Many Thanks!
Shan

[Shan_2]

I have tried to compare fileset on both machines.

I run this command (the last "wc" is just to reduce the output) and found that both machines are installed the same number of filesets that are related to "lang"

Machine A:
$ swlist -l fileset|grep -i lang| wc
76 621 7151
$


Machine B:
$ /usr/sbin/swlist -l fileset|grep -i lang|wc
76 617 7143
$


Does it mean the problem may not be due to missing "/opt/langtools" ?

Many Thanks!
Shan

[Dennis Handly]

>Machine A's PATH environment variable include /opt/langtools/bin, while this directory does not exist in Machine B!

This has nothing to do with character encoding.

>Do you know which HP-UX software is missing on Machine B?

Development tools like lex/yacc and gdb. On machine A:
swlist -l file | grep /opt/langtools

>both machines are installed the same number of filesets that are related to "lang"

Does it also mention locale?

>Does it mean the problem may not be due to missing /opt/langtools?

Exactly. langtools is for computer languages, not language locales.

You want to check for env vars, LANG and LC_* variables:
env | grep -e LANG -e LC_