There has been much written about Sun and its various responses to the Linux market phenomena over the past decade or so. Clearly, Linux was one of the primary challenges to Sun's traditional product offering of proprietary Unix systems on proprietary hardware. Sun had built up the Solaris market by competing toe-to-toe with IBM's AIX and HP's HP-UX (as well as a host of other Unix variants) and the strategy that had always worked for Sun was that of "the best defense is a good offense".
The problem with the Linux phenomena, however, was that it was incubated in the same primordial soup that gave rise to Solaris (and its direct ancestor, BSD). Specifically, both Solaris and Linux were born from a "free and open" backlash against proprietary Unix offerings from the primary players of the day. Since Sun had grown up as the David against the Goliaths of DEC, IBM and AT&T, it put the company in an awkward position to have to defend itself as a Goliath against the up and coming David that was Linux. And it didn't help that Sun's traditional competitors were investing in Linux and using it to attack Sun's install base of customers.
All of these factors, along with the fact that many (perhaps most) of Sun's own developers were at least fans (if not out-right supporters) of the Linux phenomena, resulted in constant tension in Sun's strategic plans. To embrace Linux was to feed the growing monster nibbling away at the low end of the market, while ignoring it made Sun look out of touch with a significant trend in the industry. Internally, pro-Linux camps were sprouting up and taking on the traditional Solaris establishment, much to the chagrin of the Solaris development team. After a prolonged internal battle, the Solaris establishment won out, but not without having to dedicate itself to a Linux lifestyle. Such was the birth, in 2005, of OpenSolaris.org. But up until that point, Sun's primary strategy with regard to Linux was largely to run away from it and hope it would go away.
Let's not forget that long before OpenSolaris, Sun had invested in porting and supporting Solaris on x86 (Intel and AMD) platforms, having acquired the Intel/Unix assets of Eastman Kodak (Interactive Systems) in 1991. But Sun really wanted to keep Solaris x86 in as small a box as possible. The real focus was on providing a developer playground where development could incubate while allowing for fast and easy recompilation for deployment on SPARC. Very few people, especially inside Sun, really considered Solaris x86 as a deployment platform until Sun itself launched x86-based products in 2002 and by then, some would say, it was really too late.
By the time OpenSolaris was a reality three years later in 2005, the Linux phenomena was already entrenched in the market, even in some of Sun's previously unassailable markets such as telco and financial services. With OpenSolaris, Sun's strategy was to market Solaris x86 as a better Linux and try to blunt all the market momentum that was moving away from Sun's core hardware systems (both SPARC and x86). OpenSolaris, the thinking went, had all the open source benefits of Linux, but the both the superior reliability that comes with decades of mission critical deployments and valuable new features like ZFS and DTrace that would take the Linux world years to emulate. In truth, Solaris x86 really was a "better Linux" and it should have pushed Linux back into the primordial soup from whence it came and re-established Sun as the King of the Enterprise. But it didn't.
I like to imagine a different history. I like to imagine that Sun created a robust market for Solaris x86 back around 1999 (we in the Software OEM team were pushing for this at the time) and went whole-heartedly after the Windows NT/Novell Netware market with Solaris. I like to imagine that by doing so, we completely sucked the air out of the nascent Linux market and established ourselves as the alternative to Windows in the PC server space. And I like to imagine that all of the innovations we introduced in Solaris worked either better or exclusively on SPARC hardware. It's not that hard to imagine, is it? And imagine if Sun had used Solaris x86 to promote Sun's growth in the low-end x86 market - not necessarily to replace SPARC, but to augment Sun and Solaris.
Barring an investment in Solaris x86, the only real alternative for Sun (circa 2000-2001) would have been to create our own distribution of Linux (which was tried as part of the Cobalt acquisition and the short-lived Sun Linux 5.0 distribution). A robust Linux distribution from Sun was one of Red Hat's (and later Novell's) marketing nightmares. They told me as much. As one of the sole IP-clean owners of Unix copyrights and patents, Sun was uniquely positioned not only to create such a thing but to defend it against legal challenges from within and without the Linux community.
There were many at Sun who believed in the Solaris x86 and/or Linux vision, but in the internal Sun Linux wars, they did not prevail. The rest, as they say, is history.