Unix Shell Scripting - Best Practises

[Hunki]

What are the Best practises in Unix Shell Scripting . Lets build a list together and also if somebody can share a great shell scripting template.

Thanks,

[Dennis Handly · Accepted solution]

You may want to put this in the languages and scripting forum. They may have links already to refererence material.

1) Using set -u for uninit vars
2) Using proper quoting, handling string with blanks and other specials.
3) Using sh -n to check scripts.
4) Using set -x -v to debugging.
5) Using typeset -i

[Peter Godron]

Hi,
my best pratice advice is:
1. Comment your code
We have all come across a script we wrote 2 years ago, it's now going to take ages to work out again what the script did.

2. Establish the requirements
Analyse the problem propery, rather than going for a quick fix. Temporary solutions have a nasty habit of turning permanent.

3. Pick the right shell/tool
After point 2 decide what is the best implementation to get the job done.

In general the methods are the same for shell scripts as they are for traditional software languages.

And finally test-debug-test and test again !

[Steven E. Protter]

Shalom,

I created a stub years ago from which I started all shell scripts.

It contained prompts for documentation and contained a change log a the top.

It was very helpful when people used it.

http://forums1.itrc.hp.com/service/forums/questionanswer.do?threadId=51050

There are a lot of examples of best practices in this thread and it should be looked to for guidance.

SEP

[A. Clay Stephenson]

The best practice is to practice a whole lot.

1) Typeset everything.
2) Normal output goes to stdout; error/usage messages go to stderr.
3) Use {}'s around each and every variable.
4) Use X=$(commnd) rather than X=`command`.
5) No news is good news. Don't output stupid messages that say "all is well"; instead, produce output when things are bad.
6) Always return a valid exit status with 0 meaning OK.

Nowadays, if I were starting over, I would learn only enough shell to get by and concentrate on Perl.

[James R. Ferguson]

Hi Hunki:

Be efficient! Think about the number of processes and/or temporary files you are using!

> Don't use 'cat' to read a file and simply pipe the output to a 'read'. Instead do:

while read LINE
do
...
done < file

> Don't use 'grep' to find patterns and then pipe the output to 'awk' to extract a field. Use awk's pattern matching and do the operation with one process!

> Use pipes to glue the output of one process to the input stream of another instead of using temporary files.

> Use 'xargs' instead of '-exec' with 'find' *OR* use '-exec' with the "+" terminator instead of the ";" to achieve the same performance gain. See the 'find' manpages.

> Learn to use traps (signal handlers). This is a clean, handy way to automatically remove temporary files when a shell process terminates (abnormally or normally).

> Use shell built-ins instead of spawning new external processes. Use shell parameter substitution instead of 'basename' and 'dirname', for example.

Regards!

...JRF...

[Patrick Wallek]

My recommendations:

1) ALWAYS use the she-bang syntax on the first line of your script to declare what interpreter (sh, ksh, perl, etc) the script is to use.

2) Do NOT use back-ticks to execute commands. Instead use $() syntax. Like DATE=$(/usr/bin/date +%m%d%Y)

3) Document your script. Use lots of comments so whoever comes in later can figure out what you did.

4) Use the FULL path to executables so you always know what you are running (/usr/bin/date rather than just date).

5) Set your environment in the script. You may not always run the script interactively. You may run it via cron which has a sparse environment by default.

6) Use 'set -u' as mentioned above.

[Bill Hassell]

Another very important shell technique:

- NEVER use the existing $PATH value. Export PATH in your script so you know exactly where the shell will look for your commands. Start with:

export PATH=/usr/bin

and build on that only as necessary. This avoids a *LOT* of environment issues and improves security.

Also make sure that common shell and Unix commands in your script are not aliased. It's common to have 'nice' options aliased (like rm -i) so rm in your script suddenly goes interactive with: (y/n). Use unalias to stabilize commands you are using.

And a great shell scripting site:

http://www.shelldorado.com/

[Hunki]

Re-opened thru popular demand !

[Duncan Edmonstone]

My absolute most hated BAD practice in scripts is people doing:

su - user -c cmd

The use of 'su -' in scripts should be absolutely banned! This will always invoke /etc/profile and .profile and who knows what is in there that is inappropriate for a script. The least of it could be a bunch of terminal dependent code (result: you get lots of stty: not a typewriter messages), and the worst of it could be a DBA who implements a handy interactive menu in the .profile for the oracle user (result - none of your oracle scripts work or work on the wrong database).

The correct way to do this is:

export MY_ENV_VAR=xyz
su user -c cmd

for example:

export ORACLE_HOME=/oracle/10g
export ORACLE_SID=mydb
su oracle -c dbshut

One gotcha to watch for though is that some env vars are removed by su (such as HOME SHLIB_PATH, LD_LIBRARY_PATH - see SU_KEEP_ENV_VARS in security(4) for details) and may need to be reset, e.g.

export ORACLE_HOME=/oracle/10g
export ORACLE_SID=mydb
su oracle -c "export SHLIB_PATH=${ORACLE_HOME}/lib;dbshut"

HTH

Duncan