sh_history file format

[Eric Guerizec]

Hello!

The .sh_history file is not entirely text. There is also special characters in this file. Does anybody know more about the format of the .sh_history?
I write a C prog which read this file...

Thanks.

[Steven E. Protter]

Um,

Its supposed to be all text.

Its a text log file.

All it is supposed to be is a record of keystrokes from the user at the prompt.

Maybe you want to upload this file and show us curious ones whats in it.


Inquiring minds want to know.

SEP

[John Poff]

Hi,

Sometimes it does have some binary characters in it. When I want to view it I do this:

strings .sh_history | more


JP

[Hai Nguyen_1]

Eric,

.sh_history is 100% text. It reflects 100 %what one types at the command line. If there is a typo or a non-readable character, it is logged in .sh_history as well.

Hai

[Jim Mallett]

Your history file should be all text. Do a:
file .sh_history to see the file type.

I do however see some control characters in mine from time to time and it probably just catches them just like it catches commands.

Jim

[David_246]

Hi Eric,

I checked on our systems and indeed there are some special characters inside the file. Doing a more should be no problem but a cat gives you the beeps :)
When reading the file there should be no unexpected end of lines or so, so there should be no real problem when reading the file. I gues you should open it as a normal text-file.

Regs David

[Umapathy S]

Eric,

As previously said, .sh_history logs the command history of the shell. These special characters would have come into the file if you had typed some control chars in the shell followed by enter. This will get recorded into the file.

HTH,
Umapathy

[Jean-Louis Phelix]

Hi,

If "strings" command works for you, then instead of :

fopen(".sh_history", ...)

you could use a :

popen("strings .sh_history")

which would do the filtering for use. It returns a FILE * descriptor like popen, but don't forget the pclose() ...

Regards.

[Eric Guerizec]

hey! many reponses for me :)
If you think that sh_history is only ascii file, use bvi or od and you'll see extra characters like ^A or ^B :))))

My C prog open a user session and trace every user command in a log file. Each time the user press the enter key, the last line in .sh_history is read and copied in anoter file to keep the trace. I wrote a version which use fopen. Even with fopen, I can see some extra characters (not all) in the final log! A solution is to reject all characters < 0x20. I will test...

:)

[Uday_S_Ankolekar]

do a strings on .sh_history and redirect to a file that should be then converted as ascii

strings .sh_history >yourfile
-USA..

[Paula J Frazer-Campbell]

Hi

A string on the file will not display the non printable chars

so:-

strings .sh_history > /tmp/shellhist will produce a clean printable file.

Paula

[Bill Hassell]

Actually, the shell does write special characters into the file to record position. The fc command is a shell built-in and is almost always aliased since most admins never use fc directly (alias history="fc -l"). To see this in action, edit the .sh_history file with vi and remove a few lines. Then logout and log back in again. The history command will now show a bunch of numbers with no commands and command recall doesn't work. The fix is to null the file so it can be re-initialized. Users that are trying to hide their tracks will have this symptom as they try to sanitize their history file.

[W.C. Epperson]

Dang, Bill, I thought when they taught us the secret handshake we had to swear not to tell those sorts of secrets!

But if you're gonna, then....

You can cover tracks without the symptom by:
grep -v {command I want to hide} .sh_history > tmp
mv tmp .sh_history

Now, of course, you end up with the grep and mv in the history, but a little script that removes itself, named after an innocuous command like ls, and a slight path manipulation can leave almost no trace.

We'll let you guys work out the details.

[Anonymous]

you may want to compare
xd $HISTFILE
or
xd -c $HISTFILE
against
head $HISTFILE
or
more $HISTFILE
-that gives enough details about it??s format.

Anyway, there are no details on that in
man sh-posix

[Sridhar Bhaskarla]

Hi,

.sh_history is purposefully embedded with special characters for the purpose of 'fc' comand as stated by Bill.

To see those special characters do

cat -v .sh_history

-Sri

[Eric Guerizec]

This is what my prog does. It opens the history file for reading and reads each new line with fgets(). Null and control characters are removed, then the new string wrote in my log file. It works well but some control characters semm to be associated with some ascii characters, exemple I seen Ctrl-B+W but I never pressed the W key! ... and so I kept the W in my log! I try to investigate more....

[Peter Nikitka]

Hi,

at least the ksh has the option '-s' for the builtin commands read and print, which perform a write to the $HISTFILE.
Perhaps you can look into the pdksh-source to find out what has to be done for a read ...

mfG Peter