Public domain Unix variants, encompassing the myriad of Linux incarnations, and a growing number of *BSD flavours, have enjoyed steady growth, some consolidation and much refinement in 2001. In this month's issue we survey the current state of play and explore some of the more interesting developments.

Public Domain Unix and the Market

Public domain Unix variants abound. Linux has spawned so many derivatives and ports that attempting to list all of them is indeed futile - one or more would almost certainly be missed resulting in complaint emails to author! The same is arguably becoming true of the *BSD game, a frequently ignored but technologically no less important player.

The important trend seen in the Linux market over the last couple of years, and one which appears to be gradually gathering momentum, is the used of badged Linux ports as host environments for commercial software products. The Suse port, used to host their productivity tools, seems to have started a trend. With other players like Corel jumping on the bandwagon soon after, the propsect is that this will become an increasingly popular business model for proprietary tool and package vendors to pursue. Rather than do battle with the Windoze environment, or indeed try to maintain a large number of Unix variant ports, the vendor will take a baseline Linux i86 port, port their existing product to go with it, and then sell the commercial product with a defacto freebie Linux copy to run on the host platform.

The business model adopted by many players, especially newer entrants into the Linux (and BSD) markets, of providing a badged product and then producing revenue by support contracts, rather than licencing fees, has in many instances not yielded the margins sought or expected - indeed some players have gone bust in the post-NASDAC crash market. One of the public domain Unix congnoscenti make the very apt comment recently that [Unix] is too good [reliable] and support revenues as a result are well below that which the market is accustomed to with proprietary operating systems, especially desktop systems. Those readers with a depth of experience in the industry will no doubt chuckle upon reading this - it is one of the self-evident truths of the current technology base.

The second big trend we are seeing in the public domain Unix game is an astonishing proliferation of ports, both Linux and FreeBSD, to proprietary hardware platforms. The trend originally set by the NetBSD community, who aimed to cover as many older hardware platforms as possible, has now gripped the two largest OS in the public domain market, and will yield big dividends over the coming 18 months as many of these porting projects reach the user base and begin to mature.

In terms of occupying desktops in the corporate world, the state of the game is largely unclear, since there are no easily gatherable statistical bases on how many installations are out there. A proprietary software vendor can point to licence fee statistics and boldly proclaim however million licences they have sold. With Linux and *BSD installations cropping up quietly and without any centralised means of gathering and compiling statistics, the degree of market penetration and the rate of market penetration may prove to be impossible to track. Commercial IT market analysts in the finance industry would be well advised to observe this phenomenon very carefully, lest they be blindsided in the nearer future when the market crosses some critical mass and we start seeing large scale corporate defections from proprietary OS' to Linux and *BSD.

In a sense, the position held by Microsoft in today's market had much to do with the early period of MS-DOS market penetration, when bootleg copies were produced enmasse to be run on cloned i86 desktops, and Microsoft cleverly holding back on policing this activity to maximise penetration. What we are seeing today in respects emulates that model, but it is street legal in that GPL software can be freely distributed and copied.

A key driver of the rate at which Linux and *BSD proliferate will be the availability of commercial quality software tools for users, be these proprietary or public domain GPL tools. The current base of GPL tools is remarkably large and diverse, but many of these tools have yet to displace their established proprietary equivalents. For many users, there is not a large attraction in getting an 80% capable public domain clone tool, even if it is a freebie. This is likely to change over time, as the user base matures - the large scale drift of professional programmers away from proprietary OS' and tools to GPL product was a reflection of a maturing user base, fed up with being treated poorly by vendors with defacto monopolies in their market segment. Barnum's oft quoted saying about suckers born every day of course now applies to the large base of lay users. Many lay users are now becoming experienced enough to get annoyed with the proprietary offering, so it is not inconceivable that in this decade we will see the drift phenomenon repeated in this user base, following the path taken by a good portion of the industry's professionals.

In a sense, many of the analysts' predictions of either a gigantic Linux/*BSD boom, or a collapse under the weight of Microsoft marketing, have proven to be wrong because the market is quite different to the period when DEC/DG boomed in minicomputers, or MS boomed in Intel desktops. In both prior instances, the market segment was previously empty, and the minicomputer and PC defacto colonised terra nullis in technological terms. Today's market is for all practical purposes saturated, certainly in the Western democracies, where most middle class households have one or more computers, virtually every office desktop has one, and if anything the game is not one of occupying an empty place, but rather one of displacing an existing product. As all marketeers know, displacing the incumbent is often extremely difficult, unless the incumbent has alienated the consumer to a very large degree. Such alienation presupposes the user understands what is acceptable product quality, since dumb is happy.

For the public domain software community, the big challenge for this decade is to effect conversions or convince users of proprietary products that the FUD thay have been fed about Linux/*BSD is exactly that - a defensive strategy by a market segment which is under serious threat from a technology base which evolves faster, is more robust, is cheaper and is inherently more interoperable in a networked world.

Market penetration is about end user perceptions, and in this game the Linux/*BSD has yet to defeat its proprietary competitors.

The Linux World

Given the sheer volume of development activity in the Linux world, any survey of interesting projects can at best skim over what is happening. Nevertheless, even a superficial scan provides some interesting insights.

In terms of hardware ports, there are numerous projects under way, at widely ranging levels of maturity. The effort has three broad directions - porting to current non-i86 architectures, porting to legacy architectures and porting to embedded/PDA class platforms. Notable projects include:

The ARM Linux Project (http://www.armlinux.org/) which aims to port Linux to the ARM processor.

The COLIBREE Project (http://www.colibree.de) targets the Motorola Coldfire processor, with an embedded Linux variant.

The Linux / DEC Alpha (AXP) Project (http://www.alphalinux.org/) is developing a Linux port for DEC/Compaq Alpha/AXP 64-bit platforms, which are particularly popular in clustered high performance computing applications. The current Alpha was preceded in the DEC stable by MIPS processors, covered by the Linux for DECstations Project (http://decstation.unix-ag.org/) which is porting to the legacy MIPS R2000/3000/4000 DECstations. A parallel effort targeting MIPS 4X00 series processors is the Linux / MIPS Project (http://lena.fnet.fr).

Other legacy architectures are being actively targeted - the VAXlinux Project (http://linux-vax.sourceforge.net/) covering the VAXstation 2000/3100 series, while the Linux / sun3 Project covers the venerable Sun 3/50, 3/60, and 3/160 systems.

Motorola's older mainstay, the 68k, is covered by the Linux / m68k (Atari and Amiga) Project (http://www.linux-m68k.org/), targeting the 68020, 68030, 68040 and 68060. Currently cited stable releases exist for the Amiga, Atari and MVME series SBCs. A parallel effort, the Linux / VME Project (http://www.sleepie.demon.co.uk/linuxvme/index.html) targets a range of MVME series VMEbus SBCs.

Going down in size, the Linux CE / HandHeld Computer Project (http://www.celinux.org/) targets WindowsCE hosting PDA devices, the aim being to port to as many such devices as possible. This effort is paralleled by the UK based ELKS: Embedded Linux Kernel Subset Project (http://www.elks.ecs.soton.ac.uk/), targeting 8086/80286, Palmtops, SBCs, embedded controllers and other legacy devices. The The Linux/Microcontroller Project (http://www.uclinux.org/) aims to port to MMU devoid embedded CPUs, and currently supports the 3Com PalmPilot / 68k platform.

The Linux on the Hitachi SuperH(TM) Processor (http://www.linuxsh.com) is targeting the Hitachi 32 and 64-bit SuperH processor chip. WinCE PDAs running the Hitachi SHx 1, 2, 3, 4 CPUs are targeted by the LinuxSH3 Project (http://ezc.ath.cx/redhtlinux/sh3/). The PsiLinux Project (http://linux-7110.sourceforge.net/) targets Psion's Palmtops.

The NeXT, somewhat of a cult machine in some circles, has not escaped the attention of Linux zealots, with the Li/NeXT Project ( http://www.black.linux-m68k.org/) under way.

Apple Computer, in concert with The Open Group - Research Institute, are supporting the Linux for Power Macintosh (MkLinux) Project (http://www.mklinux.org/) which is an effort to target a range of PowerMac platforms with an Open Group Mach (PMK1.1) microkernel based Linux port - making it one of the few serious implementations of the Mach microkernel. Closer to home, Paul Mackerras at ANU is working on the Monolithic Linux for Apple PowerMacs Project (http://www.cs.wisc.edu/~tesch/linux_info/) which aims to put a 2.X kernel on the PowerMac. Cited platforms include the 7200, 7500, 7600, 8500 and some clones.

HP's PA-RISC architecture has not escaped the trend, with the PA-RISC Linux Development Project (http://parisc-linux.org/index.html). Similarly, the Linux / SGI (Silicon Graphics) Project ( http://www.linux.sgi.com/) targets the SGI MIPS based platforms. Sun, the third of the major workstation vendors, is also under a sustained Linux assault, with the UltraLinux Project (http://www.ultralinux.org) targeting the SPARC sun4c, sun4m, sun4d and a range of newer UltraSPARCs.

This effort is not unlike that seen with NetBSD, which every manner of mainstream and oddball host platform being targeted for ports. In some instances the hardware vendor cooperates, in many other instances they actively discourage the porting effort, or ignore it. What is clear is that most proprietary hardware will be supported by a Linux port at some time in the nearer and not so distant future - a Linux enthusiast can relish the idea of a monolithic Linux site mixing every imaginable type of legacy and current hardware.

In terms of direct developments on the operating system, utilities and tools, the level of activity is no less frenetic and equally broad. Some interesting projects include:

The Advanced Configuration and Power Interface (ACPI) Project ( http://phobos.fachschaften.tu-muenchen.de/acpi/index.html) aiming to get ACPI running on the system.

The Advanced Linux Sound Architecture (ALSA) Project (http://www.alsa-project.org/) aims to produce a new sound card API.

The ATM on Linux Project ( http://lrcwww.epfl.ch/linux-atm/) is a focussed effort to provide robust ATM and IP over ATM support under Linux.

A new windowing system is being developed under the Berlin Project (http://www.berlin-consortium.org/), while the BLINUX Project (http://www.leb.net/blinux/) aims to provide support for blind Linux users.

The COAS project ie Caldera Open Administration System Project (http://www.coas.org/) is developing a package of system administration tools.

The Coda Distributed File System Project (http://www.coda.cs.cmu.edu/) is aimed at producing a distributed filesystem with strong support for mobile users.

The numerous haters of the Unix lpd may appreciate the effort of the Common Unix Printing System project team (http://www.cups.org/), who are working on a common printing tool for Unix systems.

Fans of flight simulators will be impressed with the FlightGear Project (http://www.flightgear.org) which is an effort aimed at producing a high quality and hihgly customisable X11 hosted OpenGL/Mesa flight simulator (portable across various Unix platforms).

CAD system users, tired of AutoCAD and its other clones, may be interested in the FREEdraft Project (http://freeengineer.org/Freedraft/index.html) which is producing a 2D drafting package.

The VHDL community, currently tied to licenced products, may be very interested in the FreeHDL Project (http://www.freehdl.seul.org/) which is developing a GPL simulator for the VHDL design language.

Electronic design automation, a layer above VHDL, is being addressed by the GNU Electronic Design Automation (gEDA)Project (http://www.geda.seul.org/) which is producing a GPL EDA toolkit.

A generalised graphics interface is being developed under the GGI (General Graphics Interface) Project (http://www.ggi-project.org/), providing support for multiple displays and a layer to hide hardware dependencies.

The GIMP and GTK toolkit are mainstays in the Linux/*BSD base, and development in both areas remains highly active ( http://www.gimp.org/, http://www.gtk.org).

Dabblers into financial management will be interested in the merger of the X-Accountant and GnoMoney projects into GnuCash (http://www.gnucash.org/).

Software developers targeting Unix environments might like to take a serious look at the Gnu Utilities Integrated Development Environment Project (GUIDE) (http://sunsite.auc.dk/GUIDE/) which is merging GNU and GPL development tools into an integrated toolset/environment. It is intended to provide editor, class browser, debugger, profiler, man generator, code checking, testing, animation, and management tools.

Users who need to interface to Macs will like the Stanford HFS for Linux Project (http://www-sccm.stanford.edu/Students/hargrove/HFS/) which aims to support the HFS filesystem under Linux. Those with the analogous NT problem will be interested in the NTFS for Linux Project (http://www.informatik.hu-berlin.de/~loewis/ntfs/).

IBM have made a serious contribution to the Linux effort by making available their Journalled File System (JFS), which is being ported under the JFS for Linux" (http://oss.software.ibm.com/developerworks/opensource/jfs/index.html).

Another interesting, and potentially useful, effort is the Overlay Filesystem Project (http://members.home.net/artn). This allows multiple filesystems to appears as one, under the overlaying filesystem directory tree.

BASIC, a favourite hate object of most C-programmers, is also invading the Linux/*BSD world under the KBasic Project (http://www.kbasic.de), running in the KDE environment, and the ScriptBasic Project (http://scriptbasic.com/).

Efforts to adapt Linux as an RTOS continue, and the KURT: The KU Real Time Linux Project (http://hegel.ittc.ukans.edu/projects/kurt/) is one example. Another is the Real-Time Linux Project (http://www.rtlinux.org). Yet another is the RED-Linux project (http://linux.ece.uci.edu/RED-Linux/).

The popular (and licenced) Motif 1.2 toolkit is being cloned under the LessTif Project (http://www.lesstif.org/).

Users frustrated with rudimentary PDF generation facilities may be interested in the Panda PDF Generator Project (http://www.stillhq.com/panda/) which is developing an API for PDF generation.

Drivers for devices using Firewire, which is growing in popularity as a peripheral bus, are being developed under the Linux IEEE-1394 (FireWire) Driver Development Project (http://linux1394.sourceforge.net/). Its would be competitor, the USB protocol, is covered by the Linux USB Project (http://www.linux-usb.org/).

Infrared devices, widely used with laptops and other small peripherals, are the focus of the Linux IR Project (http://www.cs.uit.no/linux-irda/) which aims to produce a complete Linux IRDA protocol stack.

Smartcard support is being developed under the MUSCLE - Movement for the Use of Smart Cards in a Linux Environment Project (http://www.linuxnet.com/smartcard/index.html).

Mobile phone SMS support is being developed under the SMSLink Project (http://www.styx.demon.co.uk/smslink/).

Scanner support is being developed under the SANE Project (http://www.mostang.com/sane/).

Another project aiming to support industrial applications is the Puffin Linux Controller Project (http://www.puffinplc.org/). A dedicated effort to adapt Linux for robotics is the RoboLinux - Linux for Robotics applications effort (http://www.robolinux.org).

An MPEG2 decoder to support digitsed video broadcasts is being developed under the VideoLAN Project (http://www.videolan.org), another interesting multimedia project is a GPL VRML browser at the FreeWRL Project (http://www.crc.ca/FreeWRL/).

Perhaps the most interesting project, from a political perspective, is the US National Security Agency sponsored SE (Security Enhanced) Linux Project (http://www.nsa.gov/selinux/).

The diversity of these software projects illustrates that the public domain software effort is developing genuine momentum in many areas. Of particular interest is the number of projects which are focussing on embedded, industrial and peripheral applications. Manufacturers of specialised peripherals should take note - ignore Linux/*BSD market penetration at your own peril - the industrial automation and embedded market has taken a serious interest in these operating systems.

The BSD World

The BSD world is seeing a similar broad and diverse porting and software development effort to that in the Linux world. Indeed, many of the GPL applications are developed for a generic POSIX/X11 environment and are almost immediately available on both Linux and *BSD platforms.

The principal players in the *BSD market are FreeBSD, originally offered for i86 only, the multi-platform NetBSD, and its offshoot, OpenBSD. By far FreeBSD is dominant, but NetBSD is growing in popularity especially for legacy proprietary hardware - if you come into possession of an older DEC or Sun machine, NetBSD is the OS of choice.

FreeBSD is now expanding out into other platforms, directly competing with NetBSD as a multi-vendor OS. The Compaq Alpha is supported concurrently with the i86, with current ports in development including the Intel IA-64, Motorola/IBM PowerPC, Sun 64-bit UltraSPARC and the new AMD i86-64 Hammer architecture. Unlike NetBSD which aims to encompass legacy machines, FreeBSD is targeting only the latest processors in the market.

Current development effort, according the Matt Dillon of the FreeBSD VM and Kernle development team (cited in OSNews - www.OSNews.com), includes a push to produce a robust symmetrical multiprocessing kernel, tuning of the already legendary network stack, more network drivers including Gigabit Ethernet, improved random number generation for cryptography, and the aforementioned ports - all to be released with FreeBSD 5.0 within the next year. Dillon noted that the tuned Gigabit Ethernet driver with a Dell DELL2550 adaptor and 1500 byte MTU delivered over 900 Mbits/s of throughput, in effect saturating the Gigabit Ethernet channel with packets.

In terms of development effort, many of the Linux projects discussed have direct parallels or offshoots in the *BSD world. Interesting FreeBSD projects (which may in turn migrate into NetBSD, OpenBSD and eventually Linux) include a scheduler implementing the Waldspurger lottery scheduling algorithm for load management, DHCP support, ATM, NTP precision timekeeping, ISDN IP capability, the new CAM SCSI layer, SVR4 emulation to run SCO UnixWare and SCO OpenServer executables, floppy bootable PicoBSD for embedded applications, and the Elephant file system.

Space precludes an exploration of *BSD developments in detail, but a future feature will aim to explore this area.

The rate of evolution we are seeing today in the Linux and *BSD environments, which provide native platforms for the growing base of GPL public domain applications, has no historical parallel. While proprietory vendors may argue their case, many of their products are evolving more slowly and evolving in directions driven by marketeers rather than end users.

The long term message for proprietary players is simple - adapt or become extinct.