The shell is a central aspect of the Unix paradigm of computing. A modern Unix system will offer a range of shells to please the user, and this diversity can be further enhanced with a diverse range of public domain shells.

Much has been said and written over the past decade about the impending demise of the shell, its pre-eminent role as a user interface being usurped by windowing schemes and pulldown menus. Reality however seems to be somewhat different, however, as the shell has not suffered any loss in popularity. In fact, the proliferation of public domain shells has if anything seen users flock to the likes of tcsh, pdksh and bash, on arbitrary platforms.

Anybody with any serious programming or system admin experience will not hesitate to point out that shells are the most powerful means of user interface ever devised, and the ability to blend shell scripts and direct interactive use of the system provides a competent user with a inexhaustable means of creating potent customised tools.

So great is the programming community's love of shells, that many programmers will loudly dispute the merits of their favourite shell, and the failings of all others. This techno-religious argument can become at times ferocious, but there are good and understandable reasons why users will adhere to their shell of choice.

The principal of these is familiarity, and becoming highly proficient in the use of any particular shell requires for most people, considerable effort and plenty of experience in use. This is often quoted as a central failing of the shell model, in that computer illiterates (techno-peasants in programmer-speak) are unable to learn their use. Whether this is a failing of the shell model depends on one's point of view, but it is fair to say that intellectual laziness under any circumstances does not contribute to productivity. Having to wade one's way through a veritable maze of menus and mouse button clicks to achieve the same result as by typing in a single command line has always struck the author as being wasteful in human time as well as CPU cycles ( and yes, the author does find Windows and Macintosh totally counterintuitive ... ).

Another major reason for preferring particular shells is that some are better attuned to certain types of work. Established practice in much of the community has been to use Bourne shell for scripting, and thus many of those who do a lot of script programming will lean toward Bourne shell derivatives, such as the Korn shell or bash.

A third reason for strong preferences is a degree of platform dependency. Some sites discourage (perhaps unwisely) the use the public domain tools, and hence programmers on such sites will be most familiar with the shells native to the platforms in most common use. In practice, System V sites will lean toward Bourne shell derivatives, whereas BSD sites tend to prefer csh and its derivatives.

A point worth making here, in relation to windowing user interfaces vs shells, is that the typical desktop interface such as MS Windows or the Macintosh scheme is application oriented, as the operating systems in these instances are notably weak in the area of embedded tools. The operating system is really no more than a runtime environment for the applications on the platform. In the Unix environment the operating system and its basic utility suite ARE the tool, and the use of Xwindows merely serves to enhance the potency of the basic paradigm by allowing the user multiple terminals on a single display. Whereas a Unix system can function quite comfortably without a bitmap display, the aforementioned desktop systems are totally dysfunctional without it.

A basic Unix system with a dumb terminal console and suitable shells is by all means a very useful piece of machinery in the hands of a competent user. A proprietary desktop platform without a lot of (paid for) applications has very little utility.

Shells - A Functional Model

A shell is an intelligent process, if we make the distinction that naive processes are those that know only about their input and output streams, and the operations they are to perform. A shell knows not only about its input and output streams (keyboard and screen respectively), but also about the state of the various programs which it manipulates in the course of operation.

The most primitive generic model for a shell is that of a command line interpreter, which has the ability to create other processes and pass these their appropriate arguments. Real shells are somewhat more clever, in that they also manipulate file descriptors, can pipe, provide command line and command history editing facilities, shell environment variables and job control. An important aspect of serious shells is that they use a syntax which includes facilities for program flow control and scripting, which allows their use as programming languages. Whilst shells may not offer all of the features a typical modern programming language does, they are still very powerful, particularly if coupled with filters such as awk and sed. Many system administration tasks would be intolerably difficult if not impossible if it were not for the language like features of the shells in use.

The simplest way of gaining insight into the workings of a shell is to look at the processing and execution of a program from the command line. We will start with something simpleminded, in this instance ls /usr/tmp which lists the contents of the /usr/tmp directory.

We type in ls /usr/tmp and then a , which is the command line delimiter. The shell's command interpreter uses the to separate command lines, and the presence of the is in effect a command which says "process all the preceding characters as a command line".

The command line which has been entered is parsed into the command and it argument list. In this instance the argument list is trivial, as it is a single item. As there are no pipes, tests or control flow constructs involved, the command line processing is equally trivial and the shell will proceed to execute the command.

Execution involves spawning a separate process which will execute the commanded program with its commanded argument list. To achieve this, the shell will typically use one or another flavour of the exec system call.

The new process will be first created using the fork system call. The created, or child, process is a separate but identical copy of its parent. Because the processes are at this point identical, a mechanism is required so that they can tell whether they are either the child or the parent. This distinction is provided by the return value form the fork call, which is 0 for the child process and the child's process ID (PID) number for the parent. What is important here is that the child process also has copies of the parent's file descriptors, which point to the same streams. In this fashion the child in effect can usurp the parents input and output (stdin, stdout) streams.

Once the new process has been created, it is overlaid by the exec system call with the program to be run. This specifically involves loading new text, data, bss and stack areas into the address space of the child process. The argument list passed to the child process lives in a particular reserved area of the process address space, and is pointed to by the program's arguments.

A C language program will be called as main(int argc, char *argv[], char *environ[]), where the argc parameter is a count of the passed arguments, the argv parameter is a pointer to an array of strings, which are the command line arguments, and environ is a pointer to the array of shell environment variables.

The child process program then executes, using the arguments passed to it from the command line by the parent. As the child is using the same open file descriptors as the parent, the input and output from the new process appear at and are taken from the user's prompt and keyboard respectively.

Where a more complex command line construct is used, such as find /home/blogs -name "*.txt" -print | xargs grep blogs | awk '{ print $2 }' | more , the shell will create the individual processes for each program invoked, as well as the pipes required to funnel the data through the commands. Clever manipulation of the stdin and stdout descriptors for each process, such that they point to the appropriate pipes or input/output files then completes the exercise.

This model of shell operation is intentionally trivial. In reality, a modern shell with command line editing features, job control and a complex syntax can be a complex beast indeed internally, and the creation of child processes to execute commands is the last in a relatively complicated series of command line processing and syntax interpreting operations. The result is the command line "black magic" which Unix illiterates can find so perplexing.

We follow with a brief review of some of the better known shells. Most readers will find these comfortably familiar.

The Bourne Shell

Written originally by Steve Bourne in the early days of Unix, the Bourne shell could be described as the most basic of the Unix shells. The Bourne shell is used by virtually all Unix variants for running system startup scripts (rc scripts in /etc) and is the most commonly used of the older shells for script writing.

The principal weakness of the Bourne shell is the lack of interactive command line editing and history processing, available in newer shell types. This makes the Bourne shell a somewhat clumsy beast to drive from the command line, but its widespread use for scripting makes a good knowledge of its syntax a necessary skill for anyone intending to perform system administration.

The Korn Shell

David Korn's Korn shell, distributed with AT&T Unixes since the eighties, could be reasonably described as the most sophisticated of the offspring of the Bourne shell. The Korn shell is largely compatible with its parent, and most Bourne shell scripts will run with little or no modification under the Korn shell.

The Korn shell blends features of the Bourne shell, C shell and a number of unique facilities, which make it the most powerful of the modern generation of shells. C shell features are filename completion, aliasing, job control, and embedded functions. The most important of the Korn shell specific features are command line editing using either emacs or vi editor syntax, shell control structures (select), embedding of programming facilities (test, expr, echo, getopt) to enhance performance, an expanded set of built-in shell variables and support for Unix regular expression syntax.

For those living with non AT&T (SVR3 or SVR4) derived systems, the public domain pdksh is a rather good implementation, which in the author's experience will do most of what the "official" SVR4 Korn shell will do. Whilst the pdksh has its quirks, these can be readily overlooked since the program is freely available on the Internet, and compiles with little intervention on most BSD systems.

The POSIX Shell

The IEEE POSIX 1003.2 shell standard is reputed to have taken six years to ratify, which may be one of the reasons why its use is still quite uncommon. This shell is functionally very close to the Korn shell, and it is expected that future Korn shell implementations will be POSIX.2 compliant.

While the POSIX shell differs from the Korn shell in a number of details, it provides, in the author's experience, quite good portability from the Bourne shell and Korn shell. Most Korn shell scripts will run without modification.

POSIX shells can be most frequently encountered on non-Unix platforms such as HP's MPE or DEC's VMS, in either instance it can make both of these proprietary systems usable by a Unix bred programmer. Care should however be taken with porting scripts to POSIX shells on non-Unix platforms, as the POSIX spec is quite permissive in a number of areas and this means that the idiosyncrasies of the native platform can leak through, from time to time.

The bash Shell

The GNU Bourne again shell, or bash, is like the Korn shell a superset of the Bourne shell, but it differs from its cousin in a number of respects. It uses different arithmetic expressions, supports only one of the function definition syntaxes available in Korn shell, uses Bourne shell style condition test syntax and uses the more powerful type command.

The bash shell does have some nice features of its own. The command, builtin and enable commands provide control of command searching. Escapes using backslashes in the prompt string allow for flexible substitution of the items like date, time, uname in the prompt string. The emacs command line editor mode is quite extensively customisable, including the ability to use the bind facility for setting up keystroke sequences. Included are C shell style pushd and popd, not available in Korn shell, as well as online help.

Like the Korn shell and POSIX shell, bash should run most Bourne shell scripts with little or no changes required. Cross portability with Korn shell and POSIX shell may be however problematic, particularly if a user has exploited a lot of either shells' idiosyncratic features.

The csh Shell

Bill Joy's csh was one of the creations of the Berkeley development group in the early eighties. The csh was in many respects the forerunner of most modern shells, and introduced important technical features such as job control, command line editing, and filename completion. All three functional areas have been appropriately described as creature comforts, and were in their time significant functional advances in the user interface.

The csh syntax has a number of features which are similar to C language syntax, these were specifically introduced to make csh script programming easier for C programmers. The drawback is that csh will not digest Bourne shell scripts, which must be almost wholly rewritten should the script need to be ported between the shells. The csh is standard issue on BSD Unixes, and in the days prior to the proliferation of public domain shells, made the early BSD systems much nicer platforms for interactive use, compared to their then AT&T cousins.

The tcsh Shell

The tcsh can properly be described as the "son of csh", and provides a significant number of improvements over its lineal ancestor. These include command line editing using emacs-style syntax, visual stepping up/down through the history list, terminal mode sanity checking and resetting, interactive command, filename and username completion, spelling correction of command, file, and user names (can be a nuisance), lookup of command documentation in the middle of a typed command (another potential nuisance), an enhanced (timestamped) command history mechanism, automatic locking/logout after long periods of idle time, new builtin functions for the which and where commands and automatic window size adjustment.

The tcsh is in many respects to the csh, what the Korn shell is to the Bourne shell, an enhanced superset of the basic product. Like the Korn shell, the tcsh is a pleasure to drive interactively and would certainly be the shell of choice for the entrenched csh user looking for a bit more bang per keystroke.

Summary

The shell model is without doubt one of the most potent attributes of the Unix paradigm, and will be with us for as long as Unix is. Shells are the glue which allow a competent user to produce elegant and powerful command line constructs, as well as shell scripts which are a staple of the sysadm's technical diet. The diversity of available shells provides users with a rich palette of interactive as well as script programming syntactic features, and this diversity means that almost any user's idiosyncratic tastes can be accommodated.

And if somebody isn't happy with what is available, there is nothing to prevent them from writing their own shell, whatever it may look like!