Unable to display international character in telnet

[slchan]

I am facing problem, which i am not able to view some international character i.e. ñ, ó in telnet but the characters are displayed correctly in vi through telnet connection.

[Matti_Kurkela]

As the characters are displayed correctly in vi, your telnet client and server are probably using the same character sets, so it might not be a character conversion problem.

What kind of a system you're telnetting from?
Is it a Windows workstation or something else?
What's the name of the character set you intend to use? Is it ISO-8859-1 (aka ISO Latin-1), ISO-8859-15, UTF-8 ("Unicode") or something else, maybe ISO-8859-5?

If the problem appears with the command prompt only, your shell is probably filtering out the characters that it considers inappropriate. How are your locale settings defined (use the "locale" command to see)?

The command "locale -a" will display the names of all locales available on your system. If your preferred locale is not available, you may have to install some locale support packages.

For this kind of problems, the most important locale setting is LC_CTYPE. It defines -among other things- which characters are considered readable. If your locale settings are in default values (like C or POSIX), the default character set is US-ASCII, which leaves out the accented characters. The "bash" shell makes quite strict checks, but vi seems to be more accepting.

For a quick test, you can set the locale in your current terminal session with "export LC_CTYPE=". To see how it effects your shell, you probably have to start another shell "inside" the original session: just type "bash" to do that. If this improves the situation, put the correct locale setting in your personal login script (.bash_profile or .profile) or even make it the default for all users in the system (the exact method for this depends on which Linux distribution you're using).

[slchan]

Hi,

Thanks for the suggestion.

I am attaching the output of locale -a.

I tried to export the locale but it seems not working. (i.e. export LC_CTYPE=aa_DJ.iso88591)

Actually, my program is written in Java, which reads and display a file in Spanish.
I need to telnet to run the program remotely. The program is running in Linux, Redhat.

In telnet, the program displays the language well execpt some international character (i.e.ñ, ó). We use ISO-8859-1 (Latin-1, Western)

Apart from that, i tried with ISO-8859-1 or utf-8 setting in putty to run the program in window, i got the some problem as well.

Appreciate for the help. Thanks.

[Matti_Kurkela]

So, it's a Java program. That makes the problem a bit more tricky.

Let's see. You'll have to get *all* the following correct for the characters to work:

- you'll have to have the system's locale settings correct, so that the programs are allowed/expected to output ISO-8859-1 characters.

- Java uses Unicode internally, so there is certainly some conversions involved. Normally Java picks the correct settings from the environment variables, but if your locale settings are complex, there may be quirks: Unix (and Linux) has several locale variables, but Java has a single unified notion of locale. Sometimes the JVM does not do the right thing automatically.

- you'll have to have your telnet program use the correct character set, so that it won't mis-convert the correct characters to something unrecognizable.

- the telnet program must be in 8-bit enabled mode. This should be the default with PuTTY, but some other telnet programs may have other default settings.

- the stty settings should allow the transmission of 8-bit characters. If the characters are displayed correctly when using vi, this is apparently OK.

- and lastly, you'll have to have a screen font that actually has the characters (glyphs) you need to use. Probably this is OK too.

The output of "locale" would have been much more useful than "locale -a", although the latter helps too.

When your program displays an international character wrong, does it seem to display *two* or more garbage characters in place of one international character? This is a clue that the program is actually outputting UTF-8, while your viewer is expecting ISO-8859-1.
The example characters in your today's message look like this might be happening.

The older RedHat Linux distributions used ISO-8859-1, but the newer ones tend to use UTF-8 by default. What is the exact version of your RedHat Linux? You can view the release information using "cat /etc/redhat-release" command. You don't have to be root to execute this.

What is the version of the Java Virtual Machine you're using? You can check this with "java -version".

[slchan]

I am attaching the screen capture for the wrong character output. It seems like not displaying the *two* characters for a single character: ñ .
If it is the case where the program is outputting UTF-8, while the viewer is execting ISO-8859-1. What do you propose to get this fixed? Shall i change the viewer to be UTF-8 as well?

The JVM version is 1.4. I shall be checking on the Red Hat version later.

Thousand thanks.

[Matti_Kurkela]

Hmm, looks like the two characters appear identical.

(Apparently the ITRC reply form does not handle my UTF-8 characters very well, or my UTF-8 setup has some problems. I'll use some descriptive names in for the characters then.)

An UTF-8 mis-interpreted as ISO-Latin-1 characters would produce a and . So that is not what is happening here, or at least not all of it.

This site has a lot of character code tables:
http://www.i18nguy.com/unicode/codepages.html

The characters look like shaded blocks, maybe with two horizontal black lines in them. They might be "filler" characters that the font designer has used to mark the character slots that have no character (glyph) defined in this font.

What happens if you copy & paste the corrupted line to another application? Or change the display font of your telnet program, if possible?

If it looks correct then, it is a display/font problem after all: the program is emitting the correct character codes, but the display system shows the wrong glyph(s) for those characters.

I'm not a Java programmer, but attached is a small Java test program I cobbled together using Sun's Java tutorials and other sources. By default, it will display the JVM's default timezone, locale and character encoding settings. It will also display the current year's EU-style standard/daylight time transition times and makes some UTC -> localtime translations to allow the verification of the JVM's daylight time logic.

Example output:
Timezone ID is: Europe/Helsinki
Timezone name is: Eastern European Time (EET)
Daylight timezone name is: Eastern European Summer Time (EEST)

Last Sun of March, 00:59:59 UTC: Sun Mar 25 02:59:59 EET 2007
Last Sun of March, 01:00:00 UTC: Sun Mar 25 04:00:00 EEST 2007
Last Sun of October, 00:59:59 UTC: Sun Oct 28 03:59:59 EEST 2007
Last Sun of October, 01:00:00 UTC: Sun Oct 28 03:00:00 EET 2007

Default locale is: fi_FI: suomi (Suomi)
Currency value example: 1,00
Default character encoding is: UTF8


If you compile it and start it with "java localetestj4 char", it will additionally list all character encodings your JVM knows about.
This program might help you verify the locale settings of your Java environment.