Linux Today: Linux News On Internet Time.

In Context: The Handwriting's On The Wall; Proprietary Unices Are Morphing into a Future Super-Linux

Sep 22, 2000, 09:05 (23 Talkback[s])
(Other stories by John Wolley)

SGI and IBM have been making major announcements all year about giving proprietary high-end Unix code to Linux; Caldera just bought SCO's UnixWare; even Sun is moving in this direction -- the only thing missing is the official vendor announcements that their proprietary Unices are on the way out.

by John Wolley, Linux Today

Over the past year and a half SGI and IBM have made some major announcements about giving some of the code from their high-end proprietary Unix OSes to the Linux kernel team. Then this summer Caldera bought the source code to SCO Unix, another high-end proprietary version of Unix, when it acquired the Unix and Professional Services units of SCO.

While all three companies have stopped short of saying that they intend to abandon their proprietary Unices in favor of a future version of Linux that has been beefed up to be as good or better, it looks to me like the proverbial handwriting is on the wall: it's just too expensive for anyone but Microsoft to maintain proprietary OS code; the high-end OS of the future will be a version of Linux that has incorporated all the best code from SGI's Irix, IBM's AIX, Caldera's (formerly SCO's) UnixWare, and probably somewhere down the road from Sun's Solaris -- the big vendors will make their money selling services and/or hardware.

If this an accurate forecast, why then have these vendors stopped short of making the big announcement about their intentions? Well, it could be pretty unsettling to their current customer base to hear something to the effect that they're going to be migrated from AIX, Irix, UnixWare, or even Solaris -- rock solid systems with which they're very familiar and quite happy, thanks -- to some future version of Linux which doesn't exist yet, when they're still trying to figure out exactly what the current version of Linux can do.

There are probably a lot of reasons related to the desire of these vendors to retain the revenue streams from their proprietary Unices for as long as they can; I mean, once you've told your customers that you're planning to transition to a free OS, they're likely to tell you to hurry up! But the positive spin here is that these vendors are shifting over to doing things the open source way: they're only announcing what they're actually doing, in the short term -- no grandiose plans with unrealistic release dates that we've come to call "vaporware" -- IBM's initial announcement (VNU Net 1/00) in effect left it to the Linux kernel team to determine exactly which pieces of the AIX code that IBM was offering would be picked up and when!

What's to stop these vendors from implementing their Linux enhancements as "overlays" to the kernel, and then keeping those overlays proprietary? Well, nothing, and that may very well be how it's done, for at least some of the code, especially as a first step. And that may be just fine, OK with the GPL, and a perfectly good way for these vendors to play the open source game -- lots of modules to choose from, some of them hardware-specific, but all operating seemlessly with the same kernel.

In the long run, the fator that determines whether high-end proprietary Unix code that gets integrated with Linux is open sourced or remains proprietary will be the vendor's cost of maintining that proprietary code vs. the benefits. With the propensity of the open source community to "reverse engineeer" or create from scratch proprietary functionality that is needed/wanted in a project, it seems highly unlikely that overlays to Linux would remain proprietary for very long -- and, as the articles referenced below indicate, a lot of the code that vendors are offering to Linux is going to be open sourced from the beginning.

SGI Took the Lead

SGI was the first major vendor to offering a major piece of its proprietary code to Linux. The initial piece SGI announced, in May of '99, was the XFS journaling file system, but subsequent announcements have made it clear that SGI plans much more extensive Linux code enhancements.

XFS journaling file system
"SGI extends an OS crown jewel to Linux effort" was the InfoWorld (5/99) headline when SGI announced at Linux Expo that it will "make the heart of its Irix operating system -- its 64-bit journaled XFS file system -- available to the open-source community. ... "XFS will help Linux get past some of the limitations it has today, like scaling and the 2GB file limit. 'Our goal is to make Linux as capable as [SGI's] Irix in terms of scaling and number of processors it can handle,' said Harris Shiffman, strategic technologist for SGI... XFS will enable Linux to scale high enough to handle file systems as large as 18 million terabytes of data and files up to 9 million terabytes...".

CRN (5/99) reported: "The XFS file system brings some badly needed features to Linux, including better crash recovery and greater I/O throughput...". PC Week (5/99) noted: "A journaled file system is crucial if Linux is to be suitable for large, mission-critical applications such as data warehousing." Other headlines echoed the importance of this move: "SGI Bets Big On Open Source" (CRN 5/99); "SGI Contributes World's Most Scalable File System Technology to Open Source Community" (Linux Today 5/99); "SGI to offer key piece of OS to Linux community" (InfoWorld 5/99).

The source code for Linux XFS was released in March '00 with little fanfare (Linux Today 3/00), and at least one review says it was "worth the wait" (EarthWeb 7/00). For technical details on XFS, see SGI's product white paper.

Scaling to "dozens or even hundreds" of processors
A CNET  (7/99) article titled "SGI counts on Linux for new servers", indicated just how far SGI intends to push the scalability of Linux, and hence its suitability for "enterprise mission-critical" tasks: "...SGI hopes to stand out from the crowd by making its Intel servers able to use dozens or even hundreds of processors, and for that, 'you need an operating system where you can actually muck around with the kernel,' he [Jan Silverman, SGI VP of marketing for computer systems] said."

A second CNET (7/99) article indicated that SGI plans to use ccNUMA to make Linux scale up to hundreds of processors: "The company will contribute work to beef up the Linux operating system so it supports SGI's ccNUMA architecture, SGI's take on a method for designing services that allows a manufacturer to cram hundreds of processors into one machine."

In April SGI announced (The Register 4/00) specific plans to scale Linux up to 64 processors on Intel's 64-bit Itanium chip using ccNUMA. However it was not clear in this report whether SGI intends to open source the code. The ccNUMA enhancements to Linux apparently are to be implemented as an "overlay" (ComputerWorld 4/00), which would permit SGI to keep the code proprietary if it chose to do so.

Giving Linux priority over Irix
"SGI is holding off on porting its own Unix operating system to Intel's Merced chip, preferring to use Linux." Hank Shiffman, strategy technologist at SGI, said: " 'Given the resources we have, we have to focus on just one [operating system] and that one is Linux.' " (PC Week 8/99) Although SGI will also offer Windows NT on its servers, in an earlier CNET (7/99) article, Merrill Lynch analyst Steve Milunovich said: "SGI will lead with Linux but offer Windows NT as an add-on."

IBM's Not Far Behind

In January 2000 IBM announced plans to make available to Linux major portions not only of its AIX code, but also parts of its S/390 OS. Since then there has been a stream of announcements providing more details and adding more pieces of code to IBM's Linux gift list.

The initial offer
"IBM is making good progress on its plans to open source key technologies from its AIX-Monterey and S/390 operating systems, the company said this week. Irving Wladowsky-Berger, vice president of technology and head of IBM's Linux initiatives, said he hoped the first pieces of technology would be made available within the next few months. But he said IBM must first agree with the Linux community as to which technologies they would want to incorporate into the open source operating system. 'We want to work with the Linux community to help make Linux better in areas it is not so strong, such as reliability and scaleability. We will open source limited portions of AIX-Monterey and S390 to help Linux be better,' said Irving Wladowsky-Berger, vice president of technology at the enterprise systems group." (VNU Net 1/00)

Scaling up to 64+ processors
"While IBM's server lines currently support Linux, the company plans to utilize some of the NUMA-Q capabilities to push the upstart operating system into the enterprise, said Fry. 'Linux is typically deployed in smaller machines,' Fry said. 'Our goal is to drive Linux to mission-critical applications.' " (CRN 5/00)

"Company officials also announced their intent to deliver a version of Linux optimized for NUMA servers. They hope to deliver a beta version by the end of this year, with the finished version sometime next year. 'We are devoting engineering resources now to exploring what types of technologies we need and how quickly we can drive up the reliability and scalability that corporate data centers would require of Linux,' Frye said. As part of that effort, company officials said they would try to adapt many of the technologies that make up the X Architecture, typically associated with high-end servers such as its mainframes and RS/6000 servers, to Linux." (ComputerWorld 5/00)

Logical Volume Management System (LVMS)
The next piece of code that IBM announced its plans to open source was its Logical Volume Management System (LVMS) technology, which allows different physical drives to be treated as a single volume by the OS, providing important flexibility for high-end systems that need to use a very large amount of disk space: "The LVMS release may represent a milestone accomplishment for the Community and for the extent of IBM's commitment to it. 'IBM is releasing one of its most advanced architectures for a Logical Volume Management System. This architecture is quite interesting as it completely integrates all disk and volume management into a single, highly extensible, easy to use entity.' " (LinuxMall 7/00)

The "X Architecture"
First alluded to in May (ComputerWorld 5/00), IBM provided some details in August: "...IBM will extend the core elements of its X Architecture to its Netfinity servers operating under Linux. Those elements include such products and technologies as Netfinity Director systems management software, memory chip kill technology, PCI hot-swap capabilities, Light Path Diagnostics clustering, and its "software rejuvenation" technology. 'We see this as a pretty big step forward as the X Architecture represents IBM's server crown jewels.' ...said Sandy Carter, director of Solutions Marketing for Netfinity, in Raleigh, N.C." (InfoWorld 8/00). However, it is not clear whether this will be open sourced.

AFS journaling file system
Where SGI lead with its journaling file system, the release of IBM's journaling file system, AFS, was one of the latest announcements: "IBM furthered its commitment to the open-source movement... announcing it will contribute its AFS Enterprise File System as well as supporting programming talent, to the open-source community. The technology contribution should give open-source software, most notably Linux, another boost of credibility among corporate shop considering implementing enterprise-level applications and integrated solutions." (InfoWorld 8/00)

While most commentary on SGI's Linux moves seems to focus on questions about SGI's financial viability, IBM is very strong financially and so the commentary is left with the question of what IBM is up to -- and then comments from IBM exec's come very close to saying that IBM's strategy is to have AIX be superceded by an enhanced Linux in a few years. From an interview with Irvine Wladawsky-Berger, VP of IBM's Enterprise Server Division: "ITD: So when will Linux enter the AIX space? WB: I don't expect that to happen for a few years yet, but it is certainly on the horizon. ..." (IT-Director 2/00 - italics added)

The GartnerGroup rendered its characteristically cautious opinion: "IBM claims that, with its help, Linux will improve and mature for broad enterprise use within five years. To build its credibility, IBM will form a new unit to identify appropriate technologies from AIX/Monterey and make them available for Linux and the open-source community. IBM will continue to differentiate Monterey and Linux through proprietary intellectual property that will remain unique to Monterey, but IBM predicts some kind of convergence (at the application-programming-interface level) of these two operating environments, although details are vague. ... At this point, IBM has not specified which technologies will be given to the open-source community and which will remain proprietary. IBM, however, is now presenting itself as integrator of Unix and Linux with open-system technologies... So far, no other system vendor (perhaps with the exception of SGI, on a smaller scale) has defined or assumed such a role." (GartnerGroup 2/00)

Bloor Research in the UK weighed in fairly early with an opinion that was considerably less cautious: "All of this suggests that Linux will play an increasingly important role in IBM, eventually displacing some or all of IBM's proprietary operating systems..." (It-Director 3/00)

And recent statements from IBM continue to describe in unambiguous terms a business strategy in which Linux plays so central a role that it would make sense for them to transition all the AIX and Monterey code they can into Linux: "IBM executives today described a technical revolution in IT akin to the steam engine or electricity, and they placed Linux at the core of their strategy for dealing with what's seen as the next generation of computing. Irving Wladawsky-Berger, IBM's vice president of technology and strategy, told developers... 'The reason we are so excited about Linux is we believe Linux can do for applications what the Internet did for networks'..." (eWeek 8/00)

SCO/Caldera Will Probably Follow Suit

Caldera's strategy for leveraging the high-end UnixWare code it acquired from SCO hasn't yet been articulated clearly enough to determine exactly how they plan to use (and license) that code. Caldera has been faulted in several articles for sounding like they are "trying to have it both ways" -- calling what they are planning to do "open source," but hedging so much in the licensing that it looks like they are trying to keep it pretty protiary (The Register 8/00, ChannelWeb 9/00)

The latest statement from Ransom Love, Caldera's CEO, quoted indirectly in a Linux Journal (8/24) article titled "Ransom Love's Secret Master Plan for Linux and UNIX": "So, Linux, UnixWare, Openserver, Monterey (or whatever they're calling it now) — what is the secret master plan? I draw a chart — OSes down the left, years across the top, fill in 'Linux 2.4' in 2001 with a question mark, and ask Ransom to fill in the rest. Arrows sprout from Linux and spread like fungus tendrils into the 'UnixWare' and 'Monterey' areas — that's the compatibility thing — and a big arrow moves forward into the future along the UnixWare/Linux dividing line. This represents the spawn of Linux and UnixWare, an über-OS with a yet-to-be-determined licensing policy. Ransom says you'll be able to see the source code, but parts will be open source, and parts will be 'viewable source' — you'll be able to read it, but not modify and redistribute it."

Sun? -- Don't Hold Your Breath... but Don't Be Surprised, Either!

To put it mildly, Sun Microsystems has been the most reserved of all the major proprietary Unix vendors in embracing Linux. In fact, some statements about Linux coming out of Sun have sounded downright Redmondian (except for the parts slamming MS in the same breath!) -- for example, the CRN (5/99) article titled "Sun CEO Speaks His Mind -- McNealy: No to Linux, no to NT", in which Sun CEO Scott McNeily was quoted as saying: " '[Linux is] a great way to get to the wrong answer' and is certainly better than getting there by paying Microsoft Corp.'s 'usury fees and monopolistic' prices for Windows and using 'CaptiveX'..." -- this was much more of a rebuff than an embrace.

But anyone who has followed the trade press on Sun over the past few years has gotten used to Sun's public statements not always matching what they're planning to do. Looking behind the headlines, SunWorld.com has been carrying Linux-related info. (57 articles in the Linux Today archives at last count), Sun early on provided lxrun to run Linux binaries on Solaris, and Sun had a big presence at the first LinuxWorld Conference & Expo in March of '99.

And looking at recent headlines, it seems that Sun's actions have taken a decided tilt toward Linux and open source. In early September, a Giga Information Group analyst published a report saying that "Sun needs to warm up to Linux," that Sun "...would hurt itself financially in the long-term by failing to recognize Linux as a threat and not supporting the operating system in Sun servers." (Techweb 9/00) Coincidentally, a few days later Sun announced its plan to buy Linux server appliance vendor Cobalt Networks and thus jump into the low-end, Intel-based server market with a big splash (TechWeb 9/00, CNET 9/00).

And just a month earlier, at LinuxWorld in San Jose, Sun made the two closely related major announcements: that it was open sourcing StarOffice in October, under the GPL no less (Smart Partner 8/00); and that it was forming, together with a nice assorment of everybody-but-MS vendors, the GNOME Foundation, to promote the open source GNOME desktop as a standard for proprietary Unices as well as Linux -- the link between the two announcments being StarOffice's modification to run as a GNOME app.

The only thing you can be certain of about Sun's management, based on Sun's recent financial performance, is that they will not hesitate to make whatever decisions that they feel are necessary to ramp up Sun's revenues. Just as soon as they feel that supporting Linux a little more is what's needed for Sun's bottom line, that is exactly what they will do -- no matter what Scott McNeily may have recently said in public!

Related Stories -- SGI Took the Lead:
The Register: SGI frees OpenGL brand to Linux coders(Aug 16, 2000)
The Register: SGI to enable 64-way Linux multiprocessing with ccNUMA(Apr 05, 2000)
EarthWeb: XFS: It's worth the wait(Jul 31, 2000)
The Register: SGI to enable 64-way Linux multiprocessing with ccNUMA(Apr 05, 2000)
ComputerWorld: SGI plans 64-bit overlay to Linux(Apr 04, 2000)
Source code for Linux XFS now available!(Mar 30, 2000)
Linux Today Feature: SGI's XFS Journaling File System on Track for Mid-Year Release to Linux(Jan 31, 2000)
PC Week UK: Irix takes back seat as SGI goes for Linux(Aug 02, 1999)
CNET News.com: SGI chief hints at next steps in comeback(Jul 23, 1999)
CRN: SGI To Support Linux With XFS(May 24, 1999)
SGI: XFS: A Next Generation Journalled 64-Bit Filesystem With Guaranteed Rate I/O(May 21, 1999)
InfoWorld: SGI extends an OS crown jewel to Linux effort(May 21, 1999)
PC Week: SGI goes open source(May 21, 1999)
InfoWorld: SGI to offer key piece of OS to Linux community(May 20, 1999)
SGI Contributes World's Most Scalable File System Technology to Open Source Community(May 20, 1999)

Related Stories -- IBM's Not Far Behind:
IBM developerWorks: IBM releases another proprietary product to the open source community(Sep 16, 2000)
LinuxPlanet: It's Not IBM Linux, but Close: IBM AIX 5L(Sep 03, 2000)
TechWeb: IBM Earmarks $200 Million For Linux In Asia-Pacific(Sep 02, 2000)
LinuxWorld.com.au: IBM invests US$200M in Linux development in Asia-Pacific(Aug 31, 2000)
InfoWorld: IBM makes [AFS] file system open source(Aug 30, 2000)
LinuxPR: IBM Contributes Enterprise File System Technology to Open Source Community(Aug 30, 2000)
LinuxPR: Industry Leaders HP, INTEL, IBM, and NEC Forming Open Source Development Lab for Linux(Aug 30, 2000)
InfoWorld: IBM slowly opens up(Aug 21, 2000)
eWeek: IBM refines its Linux message for developers(Aug 17, 2000)
InfoWorld: IBM to bolster Linux strategy(Aug 09, 2000)
LinuxPR: IBM Launches $200 Million Linux Initiative in Europe... Major Linux Development Centers(Jul 21, 2000)
LinuxMall.com: IBM Throws Linux Community LVMS Bone(Jun 19, 2000)
IBM to release LVM Technology to the Linux Community(Jun 16, 2000)
CRN: IBM: New Servers, And Linux For NUMA-Q Soon(May 25, 2000)
InfoWorld: IBM unveils 64-way NUMA server; promises Linux support(May 24, 2000)
IT-Director: IBM's operating system quandary, and the Linux solution(Mar 31, 2000)
GartnerGroup: The Competitive Impact of IBM's Linux Announcement(Feb 13, 2000)
CNN: IBM leads Linux charge(Feb 08, 2000)
IBM: Journaled File System Technology for Linux(Feb 03, 2000)
IT-Director: IBM's Wladawsky-Berger on Linux and the community.(Feb 03, 2000)
VNU NET: IBM making good progress on open source plan(Jan 26, 2000)

Related Stories -- SCO/Caldera Will Probably Follow Suit:
Ch@nnelWeb: Beware Vendors [e.g., Caldera] Playing Open Source Games(Sep 21, 2000)
VNU Net: SCO and Caldera open to questions(Aug 31, 2000)
Linux Journal: Ransom Love's Secret [Caldera] Master Plan for Linux and UNIX(Aug 24, 2000)
The Register: Scaldera vows a better Linux than Linux(Aug 22, 2000)
CNET News.com: Caldera CEO says Unix buy will help Linux adoption(Aug 17, 2000)
ZDNet UK: Caldera merger bolsters Linux(Aug 07, 2000)
Smart Partner: What's Next For Caldera And SCO(Aug 04, 2000)
CNET News.com: Caldera strategy highlights new role for Unix(Aug 03, 2000)
The Register: Double-spinning Caldera faces open source backlash(Aug 03, 2000)

Related Stories -- Sun? -- Don't Hold Your Breath... but...:
TechWeb: Sun, Cobalt Deal Puts Linux In Spotlight(Sep 20, 2000)
CNET News.com: Sun scoops up Cobalt for $2 billion in stock(Sep 20, 2000)
Techweb: Sun Needs To Warm Up To Linux, Analyst Says(Sep 06, 2000)
The Register: Sun puts dollars and suits behind GNOME(Aug 14, 2000)
Smart Partner: Sun Moves Closer To Embracing Open Source(Aug 08, 2000)
Smart Partner: Sun To Open StarOffice Code In October(Jul 19, 2000)
CRN: Sun CEO Speaks His Mind -- McNealy: No to Linux, no to NT(May 10, 1999)