topic Re: Cron Schedule in Linux - issue with newline character in Operating System - Linux

Cron Schedule in Linux - issue with newline character

sathis kumar — Tue, 14 Jun 2011 08:09:05 GMT

we have one simple script that works fine as expected while running it manually. However, it does not work the way it should be when scheduled through CRON. Script:
a="Perform"
b="Test"
output="$a\n$b\n"
echo "$output"

while executing the above script, it should print the values of a and b in two lines. However, when scheduled through CRON it gives the following output:
"Perform\nTest\n".

Could you please tell how we can get rid of this newline character issue (while using through CRON) ?

Re: Cron Schedule in Linux - issue with newline character

Steven Schweda — Tue, 14 Jun 2011 11:53:20 GMT

> we have one simple script [...]

Perhaps too simple. Where's the "#!/bin/sh"
(or "#!") line at the
beginning of this script? Without it, you
don't know which shell you're using, and you
don't know which "echo" you're using. It
might be a shell built-in "echo" or some
"echo" program on your PATH, and not every
"echo" works the same as every other "echo".

Then, as a Forum search for "cron" should
make very clear, the environment in a "cron"
job may be very different from your
interactive environment.

If you avoid any shell built-in "echo" by
changing "echo" to, say, "/bin/echo", then
you should be able to get the same behavior
in both cases. (It may be not what you want
in both cases, but it should be the same.)

man echo
man

Re: Cron Schedule in Linux - issue with newline character

Wilfred Chau_1 — Tue, 14 Jun 2011 21:59:45 GMT

echo -e "$output"

Re: Cron Schedule in Linux - issue with newline character

Ralph Grothe — Wed, 15 Jun 2011 06:55:12 GMT

First of all it depends on the shells you are using.
Since under most Linux distros bash is the default (unlike e.g. in the Grml distro which uses zsh as default) I suspect that in your usual login shell, which I assume to be a bash, you possibly have set the shell option xpg_echo.
You can easily check. If you type this

$ shopt -p xpg_echo
shopt -u xpg_echo

your output unlike mine most likely would read an "-s" (as "set") instead of an "-u".
See the effect if I set it.

$ msg="See if our\tshell\nhonours escape sequences"
$ echo $msg
See if our\tshell\nhonours escape sequences
$ (shopt -s xpg_echo; echo $msg)
See if our shell
honours escape sequences
$ unset msg

When your script is run under cron it most likely hasn't set the xpg_echo bash option.
Hence the difference.

Then with the echo command you have to distinguish between the shell built-in (see man builtin, and search for echo) and the "external" command to which the echo manpage refers and which resides in /bin/echo.
Usually for performance sake it is desirable to have shell built-ins executed in favour of externals, which the shell does if not told otherwise (e.g. by prepending the full path to the external command).
On the other hand you can enforce execution of a built-in by issuing e.g. "builtin echo blabla".

Besides, you can avoid the whole echo mess if you replace your echo statements by the (not built-in) printf statement.
But remember that printf doesn't do an automatic newline, just like the GNU/Linux/Bash echo command, and that you always need to provide a "\n" in the printf format string (see man printf).
This may slightly impair your performance depending on the amount of output your script generates though.

Re: Cron Schedule in Linux - issue with newline character

Ralph Grothe — Wed, 15 Jun 2011 07:03:55 GMT

Oops, didn't know this.
Just found in the builtin manpage a reference to printf as well.
So you needn't worry about performance issues if you stick to printf.

Also, the tabstop in my xpg_echo example fell prey to the forum's site rendering (because I forgot to check "Retain format spacing" before submitting.

Re: Cron Schedule in Linux - issue with newline character

sathis kumar — Wed, 15 Jun 2011 09:50:20 GMT

shopt -p xpg_echo - getting permission denied message to run this.

executed the script:
----------------------
#!/bin/sh
a="Perform"
b="Test"
output="$a\n$b\n"
/bin/echo "$output"

However, it gives the same output:
"Perform\nTest\n"

Expected result is:
--------------------
Perform
Test

Could you please let me know why the \n(newline character) is NOT getting recognized while running it through CRON?

Re: Cron Schedule in Linux - issue with newline character

Ralph Grothe — Thu, 16 Jun 2011 06:04:23 GMT

> shopt -p xpg_echo - getting permission denied message to run this.

This is curious because issuing shopt normally doesn't require elevated privileges.

e.g.

$ (($(id -u)!=0)) && echo "we aren't root" && shopt -p xpg_echo
we aren't root
shopt -u xpg_echo

Are you sure your account uses a bash?

What are the following outputting in your shell?

$ ps -p $$ -o args=
-bash

$ echo $-
himBH

$ echo $BASH_VERSION
3.2.25(1)-release

As Wilfred already replied normally you need to pass your echo commands the "-e" option to make them expand backslash escapes.

e.g. $ echo -e $output

The whole shopt stuff was only an assumption on my behalf to explain why backslash escapes without providing the -e option to echo get expended nevertheless.

Maybe someone has set an alias or a func in your user's .bashrc or in /etc/bashrc.
Check if you get a match when issuing

$ alias echo

or

$ declare -f echo

Re: Cron Schedule in Linux - issue with newline character

sathis kumar — Thu, 16 Jun 2011 07:42:10 GMT

The shell that we use is KSH.

command: ps -p $$ -o args=
output : -ksh
--------------------------------
Command : (($(id -u)!=0)) && echo "we aren't root" && shopt -p xpg_echo
Output : we aren't root
-ksh: shopt: cannot execute [Permission denied]
--------------------------------

Re: Cron Schedule in Linux - issue with newline character

Ralph Grothe — Thu, 16 Jun 2011 11:26:23 GMT

You aren't running a Bash.
So we could have saved a lot of guessing.

In man ksh on RHEL I found this entry:

echo [ arg ... ]
When the first arg does not begin with a -, and none of the
arguments contain a \, then echo prints each of its arguments
separated by a space and terminated by a new-line. Otherwise,
the behavior of echo is system dependent and print or printf
described below should be used. See echo(1) for usage and
description.

So if the string in arg does contain backslashes the behaviour of echo is system dependant (and printf should be used ;-)

In man 5 crontab I found this entry:

Several environment variables are set up automatically by the cron(8)
daemon. SHELL is set to /bin/sh, and LOGNAME and HOME are set from the
/etc/passwd line of the crontabÃ Â´s owner. HOME and SHELL may be over-
ridden by settings in the crontab; LOGNAME may not.

...

and a little further down

...

The "sixth" field (the rest of the line) specifies the command to be
run. The entire command portion of the line, up to a newline or %
character, will be executed by /bin/sh or by the shell specified in the
SHELL variable of the cronfile.

So there you have it. cron excutes command under /bin/sh (if not told otherwise by the SHELL env var) which on Linux is a symlink to /bin/bash.

$ ll /bin/sh
lrwxrwxrwx 1 root root 4 Jul 30 2010 /bin/sh -> bash

Though bash behaves slightly different when called as /bin/sh rather than as /bin/bash (i.e. more like the traditional Bourne shell), it still is a Bash and not a Korn shell.
That might be the cause of the differences that you observe.

Maybe you could try to prepend to your command string in the crontab a "SHELL=/bin/ksh "
Or if it's a Vixi crontab rather place this assignment at the top of the crontab on a line of its own.
But this I guess will apply to *all* crontab entries in this respective crontab. So be cautious.

Re: Cron Schedule in Linux - issue with newline character

Dennis Handly — Sat, 18 Jun 2011 22:46:54 GMT

>The shell that we use is KSH.

Is your script executable and as Steven asks, does it have:
#!/bin/ksh

Then it shouldn't matter to cron.

Re: Cron Schedule in Linux - issue with newline character

Steven Schweda — Sun, 19 Jun 2011 00:24:47 GMT

> [...] /bin/sh [...] on Linux is a symlink
> to /bin/bash.

Interestingly, this may be less true now than
it has been. On my Debian system, for
example:

debi# ls -l /bin/sh
lrwxrwxrwx 1 root root 4 Mar 9 08:27 /bin/sh -> dash

debi# man dash
[...]
DESCRIPTION
dash is the standard command interpreter for the system. [...]

debi# ls -lL /bin/bash /bin/dash
-rwxr-xr-x 1 root root 1943176 Apr 10 2010 /bin/bash
-rwxr-xr-x 1 root root 232224 Dec 14 2010 /bin/dash

As you can see, "dash" is more compact than
"bash", making it a better choice where a
basic (stripped-down?) shell is desired (as
for non-interactive use).

debi# uname -a
Linux debi 2.6.32-5-mckinley #1 SMP Wed Jan 12 06:05:21 UTC 2011 ia64 GNU/Linux

debi# cat /etc/debian_version
6.0

Everything's complicated.

Re: Cron Schedule in Linux - issue with newline character

Ralph Grothe — Mon, 20 Jun 2011 06:06:09 GMT

>Interestingly, this may be less true now than
>it has been. On my Debian system, for
>example:
>
>debi# ls -l /bin/sh
>lrwxrwxrwx 1 root root 4 Mar 9 08:27 /bin/sh -> dash

Hi Steven,
that's interesting news to me. I even haven't heard of "dash" yet.
Is it debian-specific? Because it of course the "d" sort of implies this.

Ah yes, it seems so

http://en.wikipedia.org/wiki/Debian_Almquist_shell