nextstopcafe.com

Windows TNG: Use Linux as the Next Microsoft Platform

[Microsoft's New Win-Win Strategy: Post 3 of 5]

My last posting, "Scrap The Windows Codebase" explained some of the reasons why the current Windows codebase might be better off in the trash bin.  My mention of "Linux" caused myriads of knee-jerk reactions from other bloggers (see the Channel 9 forums, for example).  But, this post will only talk a little bit about Linux, and a lot about what might be needed for the future, and why the operating system has become too central a focus for Microsoft.

What IS Microsoft’s Business?

I’m not sure Microsoft ever wanted to be in the operating system
business.  There is been a lot of revisionist Microsoft history
floating around, and the original goals of Gates, Allen, and eventually
Microsoft are sometimes lost in the rhetoric of their successful
business strategies.

Today, Microsoft knows their operating system is the cement that glues their business strategy together. 

Microsoft executives describe how desktop applications "widen the ‘moat’ that protects the operating system business". At the same time, Joachim Kempin in his 1999 testimony said "We are not in the operating system business.  We are in the computing business."  While at first glance this seems like defensive rhetoric to distract Jackson’s team from their OS focus, keep in mind that Kempin was with Microsoft since 1983, and his chief responsibility was developing their relationships with PC manufacturers and OEM distributors.  Kempin, perhaps more than anyone else, saw the operating system as a useful business tool, one which wielded legendary power over extending Microsoft’s computing products to every desktop.

While there is no doubt that this strategy has yielded tremendous financial rewards for Microsoft, it is nonetheless as strategy based, not upon technologies or innovation, but upon tie-ins, bundling deals, and partnerships.  In this sense, it limits Microsoft because, unlike other companies, they do not need to create the best products, only the most viable ones suitable for their (so far) successful business strategies.

But, when Gates is at his best talking about Microsoft’s strategies, there is not a mention of predatory business practices, and the "moat" that protects the operating system.  Gates, in a notable 2003 interview with Fortune, said t "One was our vision, which has not changed since the day the company started."  The vision was, according to the interview, "the idea that you could buy PCs from many different hardware companies, and yet they would all run the same software".

C’mon Bill!  Let’s look back briefly.  Bill originally thought computer languages were the company’s business.  He went to IBM for an Operating System.  Only when IBM couldn’t come to terms with Digital Research for an OS license, did Gates see an opportunity, buy QDOS, and license it back to IBM.  This sounds a bit closer to what Kempin might describe as "the vision": bundling and aggressive licensing.

All in all, I still believe Gates wants to serve the needs of consumers.  One of the advantages Apple has over Microsoft is that their vision of ease-of-use and style is strong by comparison to the muddled vision of Microsoft.  This wavering statement of vision, and inability to reconcile what Microsoft says with what it does is one of the reasons that so many Microsoft-bashers conclude that Microsoft is simply greedy and predatory.  Without clear vision, all successful business might appear so.

At his best, Gates talks of the dream "of a PC on every desk and in every home".  I think he still believes that comptuers are good for people and that the business mission is to do whatever is necessary to enhance the computer experience.

What does all this have to do with operating systems and my Windows TNG idea?  Well, two things.

First, I believe Microsoft has become so distracted by the importance of their operating system as the glue for their predatory business practices and the "moat" that they have stopped innovating.  Worse, the size and complexity of the operating system itself has slowed down even progress on Microsoft’s business strategies.  If ever Microsoft needs innovation, now is the time.

The second point Bill makes better than I can.  In the Fortune interview, when asked "What can Microsoft do for small business?".  His answer, in full, is

Making our software simpler will probably have more dramatic impact with small business than anywhere.

Anyone who is following the current Vista releases knows that Microsoft is not moving toward simpler, but toward a far more complex, multi-faceted operating system.  The OS has taken over the company, and it’s taken over the consumer’s view of the product line.  Windows TNG is one perspective on how Microsoft might change that.

A New Platform

If Microsoft is going to return to the goal of extending the power of the PC and creating greater consumer value, they need a new platform.  What might that platform look like?

For a moment, let’s speculate about the feel of a new product line, and the technical underpinnings.  Consider the profile of the products and technologies I’m suggesting, and don’t get hung up dwelling on a particular flaw or inconsistency, as there will be many  This is, essentially, a "White Paper".

It’s first worth considering exactly what the criteria might be for a new Windows.

Goal 1: Unbundling

Windows may not be too big if you consider all that it does.  And Windows may not be as monolithic as it seems.  Internally, there are many layers and boundaries.  But, it is delivered as a monolithic product, and because of that, organizational dependencies have been allowed to remain.  High level application changes, UI changes, and kernel changes all end up on one huge Gannt chart.  Despite attempts to avoid them, true development dependences exist between layers that simply should be separate.

What if, instead of a 50 million line Vista, we had the following:

1. A Windows TNG Kernel OS which has a separate release cycle, is developer-configurable, and was used in both desktop, embedded, and special-purpose applications by Microsoft and third party developers.  It would have drastically simplified security, a lightweight process model, and could be built in custom configurations by developers (similar to the way Windows CE is delivered).  Small configurations may have only a 400K footprint.  Large ones would have no larger than a 1.5M memory footprint.  (1-3 million lines of code)
2. A Windows TNG UI (essentially "Avalon-in-a-box").  Again, a separate release cycle, and custom-configurable.  This is a developer’s product.  Because it is separate from the kernel, competing UI models can be developed and delivered separately.  Much of the "compatibility" with older Windows applications lies here.  (3-5 million lines of code).
3. A set of UI applications bundled with installers which is what users see as "Windows".  It installs and runs very much like Windows does now.  It is completely separate from the other two components, has its own schedule and can even be purchased in its unbundled form.  This is especially appealing for corporates who may want the important security or functional benefits of a new kernel but do not want to retrain users until later.  (5-10 million lines of code).

The new product has a technology profile shown in the (highly speculative) drawing at right.  Click on it for a bigger version.  Notice that:

1. The OS is unbundled, configurable, and separately shipped across all versions.
2. The Microsoft UI is optional, but shipped with the consumer product as standard.
3. On embedded devices, there is a more compelling case for vendor-specific UIs.
4. Specialized embedded devices, such as wearable devices, benefit from a very lean kernel with no GUI overhead.

Most importantly, all variants use the same kernel.

What happened to the other 32-41 million lines of code?  Well, they may exist somewhere, for example:

1. A product that might be called "Windows Legacy".  It’s a virtual machine that runs a version of "Vista Minus" (or even "XP Minus").  Maybe it comes free with Windows, and is even installed by default for a while.  It will run 100% of all old Windows applications.  Thought it provides compatibility, it also sends a strong signal to customers that there is a dividing line beyond which compatibility may not be guaranteed in the future, and it serves to define where that line is at.
2. Specialized Server applications such as IIS, remote management software, etc.

Most of the savings comes from the next goal…

Goal 2: Reduced Complexity

Windows has become far too complex, and needlessly so.  For example, the past 15 years have seen 4 different phases of graphic support:

1. The pre-NT GDI model present in products such as Windows 95 and Windows 98.
2. The user-mode GDI model present until Windows NT 3.51.
3. The kernel-mode GDI and GDI+ model of NT 4.0 and XP.
4. The new Avalon framework for Vista.

While all of these are improvements, the compatibility requirement means that the legacy of these four models will exist for a long time.  This makes the product tremendously complex, holds back progress, and adds to the number of details applications programmers must learn.

Similarly, the Windows security model is orders of magnitude more complicated than Unix-based systems.  While Unix-based systems clearly are overly simple and not a good model either, the Windows model provides a detailed object-level security model which is far too expensive and yields few real benefits.  Rather than being used by developers, most security settings on most objects created in the Win32 API are left unchanged.  On a more pragmatic level, the weak passwords of most Windows users, coupled with the tendency of most non-corporate users to run as Administrator have rendered any security scheme irrelevant.

Vista is about to add yet futher layers of complexity on top of the already existing layers.  Localization, security, object management, and graphics are all about to be reinvented in Vista.  And yet, the old way will still remain.

Reducing the complexity of features such as these is essential, especially if upward compatibility is important.  The worst thing you can do for future systems is saddle them with complicated features that are superceded and replaced constantly as versions are released.

Reduced complexity can also have performance benefits.  For example, the Windows process model has always been criticized as "heavy".  Creating a process takes at least 10 times longer than creating a thread and frustrations over a complex process model can hamper efficient use of resources.  A new process model where threads and processes have the same weight, and processes can be created cheaply and easily would create greater performance opportunities in server-based applications.  This becomes especially important in real-time and embedded applications where lightweight processes are almost essential to development.

What about Compatibility?

Compatibility is one of the most difficult constraints in moving forward.  While Microsoft wants to say with 100% certainty that "Your application will run", the continual requirement of compatibility hampers progress on newer, superior technologies.  In addition, compatibility makes Windows itself less flexble.  If compatibility issues extend deep into the kernel and user APIs, then trying to deploy the Windows kernel in tiny embedded products will be almost impossible.

Microsoft will have to draw a dividing line in the sand, as Apple has done.  Applications on one side of the line will run with few if any changes, and this should represent about 90% of the applications developed in the past 10 to 15 years.  This means that some kind of "compatibility library" will need to be developed.

Isn’t Microsoft Already Doing This?  Why start over?

Well, yes, and no.  While many features in Vista are targeted at these problems (such as the "Server Core" version of Vista), there is no true unbundling, and Vista will remain a monolithic product.  It will take years, or even decades, to gradually pull the pieces of Vista apart, and while those attempts are made, developers will continue to add more.  Attempts by Microsoft to truly create a layered operating system out of XP with no layering violations have been difficult.

Make or buy?

In many ways, what I’m suggesting is obvious.  Microsoft knows they’ll have to replace Windows.  That’s why projects like Singularity exist.  And, the goals I’ve spelled out above (including unbundling) may already be on the drawing board.

In theory.

In practice, the sheer size of Windows, and the compatibility juggernaut, will make everything take longer.  If Microsoft wants to replace Windows by 2015, it will take until 2025.  If they want it to be "fully compatible", then even Singularity will be hampered by the very same issues outlined in my previous post and this one.

Can Microsoft, and their customers, wait until 2025 until repeated evolutionary steps solve all the problems I’ve mentioned?  Will Microsoft continue to have such dominance that people will wait?  Unless Microsoft acts more quickly, it is inevitable that some competitor, probably Apple, will finally be able to attract large numbers of Windows users with an offering that, to users, appears similar enough to Windows from a purely functional perspective.

Maybe it’s time for Microsoft to do what they’ve done so often before:  Acquire technology which solves the problem.

Here comes the L-word

Everything I’ve said until now has tried to make a case that:

1. The liability of the Windows codebase, including Vista, will slow Microsoft’s progress to the point where vulnerability to competitors becomes threatening to Microsoft within the next 5-10 years.
2. Microsoft is solving these problems, but not fast enough for their users or shareholders.
3. It’s worth considering if there are potential technologies which can be aquired to solve the problem.

Linux has the potential to solve Microsoft’s problems, but it’s important to look at the potential of Linux rather than the current reality.  Consider that Microsoft Visual Basic was originally purchased by Gates as Tripod by Alan Cooper, and SQL Server was written by Sybase originally until Microsoft negotiated exclusive rights to all Windows code.  What those products are today only a slight resemblance to what they were on the day they were purchased.

So, rather than look at Linux the way it is today, imagine what it would be like if Microsoft were to adopt the Linux platform, participate in and fund development, and drive the direction of Linux forward to meet Microsoft’s own needs.

Technically, Linux has the following strengths:

1. It is a successful platform in use today, which is benchmarked and compared side-by-side with Windows.  In server-based applications, it often comes out ahead of Windows in some performance and security benchmarks.  Rather than being an "idea", it is an actual contender and is being used side-by-side with Windows in many corporate production environments.
2. Its security model is remarkably simple and even could be called antiquated.  Yet, for some reason, it has held up very, very well and is considered at least as secure as Windows.  Because it is so simple, it will be easy to upgrade and replace.  But, because it is working adequately now, replacement can be done carefully and cautiously while focusing energy on more important issues such as desktop innovation.
3. It has an efficient lightweight process model that is a superset of the one provided by Windows (that is, Windows process model can be built on top of the Linux process model).
4. It has been competing vigorously with Windows, and there is already a large device driver base.  In fact, vendors of hardware consider Linux to be second only to Windows in their priority for releasing device drives, and many vendors already do.
5. It has an entrenched development model which is popular in universities and many businesses.  thus, it is not necessary to spend significant time or effort on development products, especially for low-level drivers and server applications.
6. It has a configurable kernel which can be used in everything from tiny embedded devices up to very large multiprocessor systems.  The kernel is small, modular, and extremely robust.
7. It is much newer than Windows, and has very little legacy code by comparison.
8. An enormous amount of effort has already been done to create Win32 compatibility layers.  WINE, Crossoffice, and Xen are three specific technologies designed to run MS applications under Linux.  Rather than criticize these as inadequate and lame (they are), consider what would happen if Microsoft were to take over development of one of these.  Progress would be rapid and the problem of compatibility would be cleanly isolated.
9. It has a shared library model which allows multiple concurrent and incompatible versions of software to co-exist simultaneously without the need for extensive additional technology investment or developer education.

Linux has several weaknesses to consider:

1. The X-Windows platform is interesting, but outdated.  While a client-server windowing system has clear advantages, its API is more arcane and complex than Win32.  Microsoft (and the market) would be served well if Microsoft were to build (or buy) a new, more modern and capable graphics application framework for Linux.
2. The Open Source model requires a substantial investment in legal work and planning.  While some of the technologies would clearly remain Open Source, Microsoft would want to engineer as many components to be proprietary as possible.  This may dictate packaging and delivery "mechanics" in some ways.  Since many companies are already combining Open Source with proprietary products in their deliveries (such as Redhat), I am confident Microsoft can negotate this minefield in an aggressive and innovative way that would impress us all.

But, the biggest benefits of adopting Linux aren’t technical at all…

Linux: The Business Reasons

While the technical issues can be argued to death, the business reasons for adopting Linux give Microsoft significant advantage.

Keep in mind that Microsoft has always been excellent at the adoption and assimilation of technologies that are already in the marketplace.  I remember when the Web was something Microsoft said they weren’t interested in!  (Yes, it’s true).  Despite the Java legal debacle (which Microsoft could have avoided, I believe), their adoption of Java was highly successful and if managed properly could have avoided having to "reinvent" replacements such as C#.

So, here are what I believe are the most compelling reasons why Linux is a good "buy" choice:

1. Microsoft completely eliminates open source as a competitor.  By embracing Linux, almost all open source efforts suddenly lose most of their "shared mission" to compete with Microsoft.
2. Microsoft extends dominance of the Office applications onto every desktop.  Efforts like Open Office, Sun Office, etc. become truly jokeware.
3. Competitors like IBM, RedHat and Sun start to shake in their boots as they realise that multi-million dollar investments they’ve been making in Linux have now directly benefited their most feared competitor.
4. Microsoft takes firmer control over how the GPL is applied to products.  Keep in mind that many products you buy contain both open source as well as proprietary components.  Windows TNG would be no exception.  But, because it was bread-and-butter to MS, they would apply their significant legal capability more productively.
5. The Open Source "religion" would become diffused.  Most open sourcers would be horrified to think that MS is "taking over".  As more and more MS successes occurred in the Open Source arena, the "we love Linux and software should be free" crowd become more and more marginal.

The Bottom Line

Sure, it’s a minefield.  But, Microsoft is on a slow-burn right now to creating less and less competitive products while others are creating more innovative products with shorter delivery times.  By unbundling, creating leaner development strategies, adopting some proven technologies, and dominating the open source space, Microsoft can reinvent the entire industry.

That leaves Microsoft more time to outdo the very competitor who is making the greatest advances toward Microsoft’s market: Apple.

The next segment of this post will explore how Microsoft might refocus and use their time to create a truly next generation desktop by creating a proprietary application layer on top of Linux and OS/X.

Being Negative about Windows

Ed Kaim responded to yesterday’s "Scrap the Windows Codebase" post with some good comments and it’s worth a follow-up.  Ed says "I was surprised by the negativity of the tone overall and felt it was very much in the style of Michael Moore".  Well, maybe I deserve that.  It is hard to talk about scrapping one of the industry’s most valuable codebases in positive terms.

Ed has a good point when he says:

All I care about is that the OS does what my customers and I expect it
to do and that the apps we build don’t break. If it takes Microsoft 10
years to ship each new OS, that is better for us because it
means less budget gets spent on migration and more on core projects.
However, if the rug gets pulled out from under billions of users by
drastic changes for questionable improvements, we’re all screwed.

I agree with him 100%.  Microsoft has a overwhelming responsibility to their customers and shareholders not to cause needless market and consumer upheaval.

But here’s the point:  It’s far worse to live in denial.  If you have a problem, you need to face it full-on, even if it’s more severe than you want it to be.  When a business has 10000 employees it can no longer use, it’s not easy to make the decision to have massive lay-offs.  But it’s a mistake if the business ignores the problem.

Re-read my post  on Windows Vista: Past Its Due Date Already, where I talk about this kind of denial in a similar situation with the product of a former software industry market leader:

Then a line is crossed.  You know that something is wrong.  Your
engineers can feel it.  There’s a malaise in the air.  But, nobody says
anything.  At the lunch table, you read PC Week’s scathing criticism.
People stare around the room, some even laugh or scoff. Most say
nothing.   You go back to your work, you immerse yourself in further
enhancements to your product.  You convince yourself everything is OK.
You look at competitive products only for purposes of punching holes in
their strategy.  You find the holes.  You reassure yourself.  Everyone
smiles.

Repeat until fail.

I recognize the pattern.  That’s where Windows is right now.

A few MS employees have told me I’m not far off.  And Robert Scoble, in his short comment to the post, says "I totally agree".  Robert may not be on the team, but he’s at least a close observer.

Ed also makes another good point:

There are smatterings of anti-Windows sentiment in broadly sweeping
statements and quotations taken somewhat out of context that would
indicate that people are fleeing Windows due to the problems Gary
outlines. I don’t see it at all.

He’s right.  I don’t think people are fleeing.  Windows customers want windows to be healthy.  Sure they do.   I was listening to an InfoWeek Podcast yesterday and Mitch Wagner said that the newest Vista Beta and Office 2007 have him ready to "eat his words" about former negative comments.  It’s looking better, and we’re all happy. 

Yes, even I am happy.  Nobody who relies upon Windows wants it to fail.  I’m not a Windows basher, trust me.  I did try Linux as my primary OS for 2 years.  I gave it a good try and ran my Windows apps under Wine or VMWare.  I should blog about it someday, it was an interesting experience in compromise.  When I switched back to XP 18 months ago, I felt like an old friend had returned.

When I was working on the dBASE project I talked about in my earlier post, it was the same way.  Everybody in the market wanted dBASE to be great.  Everybody inside Ashton-Tate felt that.  They wanted to produce the best product for the market.  Nobody was "leaving for other products".  There were no other alternatives!  Very much like Windows.  How can anyone leave?  There are truly no alternatives.

My upcoming posts will be less negative.  The last post was the "gosh we have a problem, Houston" post.  Of course it feels bad to admit the codebase is doomed.  Microsoft must eventually admit it.  But, my next post won’t be "pro-Linux".  It will be pro-Microsoft.

Thanks for the comments Ed.  They’ll keep me on track.

 

Windows TNG: Scrap The Windows Codebase

[Microsoft's New Win-Win Strategy: Post 2 of 5]

As a successful software architect, I’ve learned to recognize the results of a poorly managed design process.  In Windows Vista: Past Its Due Date Already, I gave some insight into why Windows matches a "software implosion pattern" I’ve seen before.   This post explores why Microsoft really should scrap the codebase, the next post will suggest the controversial idea that Microsoft should scrap Windows in favor of Linux.

The recognition that the code base is in trouble is no secret, and for the most part even Microsoft agrees.  Microsoft has a team called "The Windows Code Excellence Team".  They have a program for "driving broad changes efficiently into the Windows codebase".  They call these "Strike Force Efforts" and the very name they chose reveals the adversarial relationship that even Microsoft insiders perceive with their own code base.

But still, Microsoft thinks they can fix it.  This post is about why they can’t.

The Recipe Was Wrong From The Start

The biggest single reason why Microsoft can’t fix the problem is because Windows archtecture is flawed at its very foundation.  They’re sinking money into repairs on an old car, trying to make it fuel efficient, trying to make it conform to current needs.  But as we all know, sometimes you need to buy a new one.

Understanding why this is true is technically challenging for most people.  The real issues are understood only by well-educated computer scientists.  As a result, most of the common discussions around Windows architecture talk about the visible aspects of Windows, rather than the technical underpinnings.  And, because the technical world is so full of geeks that argue pointlessly (a la SlashDot), there are rarely any efforts to bring clarity to Windows most severe problem.

There are ample sources of technical information about this.  While it may seem like a Windows vs. Mac argument, Daniel Eran’s excellent (if lengthy) article "Five Architectural Flaws in Windows Solved in Mac OS X" is one of the best examples.  What can easily be missed about this article is that 4 out of 5 points Dan makes are directly the result of Apple’s decision to use Unix as an architectural basis for OS X.  If Dan were talking about MacOS 9, only the first of his five arguments would apply.

So, by adopting Unix, Apple was able to push their own technology into the next generation.

Both anecdotal evidence and qualified research are available to illustrate the problem.  Security researchers can point squarely at flawed API design as a primary reason why Windows architecture actually "encourages insecure applications".  Those who remember Fredrick Brooks classic book "The Mythical Man-Month" see the same trip-ups in Vista that Brooks warned about in his book.

Because Windows architecture has fundamental design flaws, Microsoft is constantly adding layer after layer of new technologies to achieve what would be implicit in a better architecture.  This increases the amount of work, technology, and conceptual baggage that developers need to learn and master.  Thus, it decreases the relability of overall applications.  Developers have a limited amount of mental energy.  Any Windows developer knows that negotiating Windows "idiosyncrasies" takes up almost as much time as the application itself.

The idiosyncrasies themselves become entire "worlds of new technology"  Knowing how the Windows event loop operates, and arcane topics such as using "assemblies" as a way to avoid "DLL Hell" become distinct new art forms.  The Windows community has become so accustomed to this continual "band aid" approach that they don’t even recognize the original problem.  Instead, the new technologies become job skills, and further serve to separate Windows programming expertise and culture from the larger world of shared computer science knowledge.  (See an earlier post of mine for more on this "disconnect").

Now and then, some Windows developer asks the right questions (from the InfoWorld gripeline):

I am surprised that still, after all these years, that Windows has not
seen the solution that UNIX (and probably many OS’s) takes to DLL Hell
– use versioned DLL files so something linked against an old DLL will
use the old one while something linked against the new one will use the
new one. Viola. Problem solved.

Continual layers of technology to solve architectural problems leads to the next problem with the Windows codebase …

Escalating Complexity

In a recent New York Times article, the following appeared:

Several thousand engineers have labored to build and test Windows
Vista, a sprawling, complex software construction project with 50
million lines of code, or more than 40 percent larger than Windows XP.

Windows is growing beyond even Microsoft’s ability to manage it.    In October 2004, Martin Taylor, then Microsoft GM of Platform Strategy, admitted that changes were needed and introduced a new "role-based" strategy for reducing the size of the code-base:

"Today, it’s still the entire code base. There’s no reduction in the
bits you get; things are just roped off," Taylor said Friday. "We want
to get to a model of role-based deployment where you might just have
the bits you need for that function. … It’s one of our design goals
for Longhorn."

In that ComputerWorld article, Taylor was talking about a new agenda for Longhorn to trim the size of the code base.  The article later says

A Microsoft spokeswoman confirmed that the goals of providing a smaller
Windows "footprint" are to cut maintenance costs and provide a "reduced
surface attack area."

Martin is now part of the Windows Live team.  The new GM of platform strategy, Bill Hilf, hasn’t said a word about it in the last 18 months.  So much for trying.  If a smaller code-base was one of the design goals for Longhorn, it seems Microsoft has decided to put the idea (and Martin Taylor) on the back burner for now.

Windows 3.1 had 2.5 million lines of code, Windows 95 had 15 million, XP has 40 million, and Vista will have at least 50 million.  Microsoft managers are stumped about how to reduce the complexity and size, and Michael Cherry, former Microsoft product manager, says "It’s such a collection of smart people that they’ve started to believe too much in themselves". (MercuryNews)

Not only is the problem growing, but the team is looking the other way.

Reinventing Microsoft Software Culture

It’s not enough to throw out the codebase.  Here’s the real challenge:  You need to retool the very culture which allowed it to happen.  Before starting over, you need to figure out what (mis-)management style allowed Windows XP’s excellent security foundation to be completely disabled by other arms of the organization.  Microsoft already tried to re-invent Windows once with Windows NT.  The problem is, while one set of OS experts in Microsoft is devising an excellent security framework, another set of "experts" is violating all the rules in the interest of "dumbing down the features" for users.  Security guru Steve Gibson (quoted in WindowsITPro) explains the phenomenon:

"With
Windows 2000 you could argue that Microsoft was at least preserving the
original NT security model," Gibson noted. "Regular users would log on
as Administrator only when doing system tasks like installing
applications or bug fixes, and then log on as a regular user to get
work done. This is much like a UNIX machine, where the root account is
tweaked very carefully, not generally used for day-to-day work. But
Microsoft moved that NT security model to the home and gave
Administrator power to users. [The company] discarded the traditional
security model because it was too hard to explain to users…"

Aaron Margosis (a Microsoft employee) has an entire blog dedicated to the topic of trying to help people run Windows as "Non-Admin".  Despite his excellent advice, it is impossible for the average user to comprehend, much less follow such instructions when all of the default settings of Windows, and the expectations of third-party software are that average users will be running as Administrator.  As a result, Microsoft’s excellent security model lies in the background, gathering dust, while clever hackers throughout the world know that it’s open season for attacking the average user’s desktop.  All of this is no accident, and is engineered into the product—introduced by well-meaning product managers attempting to make things simpler while elsewhere in the organization people know better.

It is as if Microsoft is releasing products by "trial and error".  They even recognize the "non-admin" problem and are moving to add yet another layer of complexity called User Account Control to Vista to help solve this problem.  But, even on the second attempt, it’s not looking good.   Beta users are annoyed and claiming it is far too complex and intrusive:

In its current incarnation, too many people are likely to dismiss [User Account Control] completely, and if that happens, everyone loses. [Ed Bott -ZDNet]

That Microsoft has launched a major product with an major introduced security flaw is the most brazen sort of incompetence.  That they are still not getting it right reveals something much worse: ignorance.  While scrapping the codebase is essential, it’s equally essential to establish new rules when moving forward.

Margolis final paragraph in one of his articles is among the most interesting.  Probably the biggest reason why Windows XP is so vulnerable, and that so many people run as root, is cultural:

Hey, y’all!  We need to lead by example.  People look to us for best practices, for the right way to do things. We are trying to convince the world that we are thought leaders in software and in software security. In the Unix world, they never run as root except when necessary. They “su”, do what they need to do, and revert back. We are not leaders when we run as root all the time. Comrades: you need to run as “User”, and your customers need to see you doing it. If you run into issues, don’t add yourself back to the admins group – file a bug against the offending product. Customers: if
you see any MS sales, MCS, Premier, PSS, etc., doing web or email as
admin, please tell them, “You’re not setting a very good example. I am disappointed.”

Spaghetti

While some experts agree that the above flaws are proven facts, I suspect there are more people in the industry that consider these things quite subjective.  As we all know, a picture is worth a thousand words.  In April, a ZDNet article with the dubious title "Why Windows is less secure than Linux" included two diagrams generated by Sana Security and shown below. These diagrams received little attention, but compare graphically how Windows and Linux process the service of a single web page.

The first diagram illustrates how Windows processes the page:

The second illustrates how Linux processes the page:

The orderly arrangement of the Linux traces are no accident.  They are the result of years of thinking which goes back to the origins of Unix itself.  Good operating system design assures that the operating system and application layers are distinct—separated by boundaries which are like immovable walls.  Such walls manage the compexity of systems by isolating operations from one another to minimize dependencies.  To a good system designer, these are not just guidelines, they are dogma.

To any good systems architect, the traces of the Windows diagram are like a giant black spot on your MRI.  They represent an undisciplined and haphazard set of interelationships resulting from years of unsystematic development and support of legacy code and processes.

Little Hope of Repair

There is hardly any hope of repair for such systems.  The new Windows Vista may eventually work, but it will be by brute force testing and a bit of sleight of hand—not good design.  Microsoft, however, is trying to fix it.  On his blog, Microsoft employee Larry Osterman describes the problem:

As systems get older, and as features get added, systems grow more
complex.  The operating system (or database, or whatever) that started
out as a 100,000 line of code paragon of elegant design slowly turns
into fifty million lines of code that have a distinct resemblance to a
really big plate of spaghetti.

This isn’t something specific to Windows, or Microsoft, it’s a
fundamental principal of software engineering.  The only way to avoid
it is extreme diligence – you have to be 100% committed to ensuring
that your architecture remains pure forever.

It’s no secret that regardless of how architecturally pure the
Windows codebase was originally, over time, lots of spaghetti-like
issues have crept into the  product over time.

Larry’s right about the problem.  But unfortunately there is no way to turn back the hands of time and retroactively make sure that the architecture was pure from the start.  Yet, he goes on to describe how Microsoft has developed internal tools "that perform static analysis of the windows binaries and
they work out the architectural and engineering dependencies between
various system components".  The hope is that by knowing which layers should be isolated and why, changes can be put in place which fix the problem and eliminate the spagetti.

Then, unfortunately, Larry goes into the tall grass when he says:

Well, most of the layering issues can be resolved via email, but for
some set of issues, email just doesn’t work – you need to get in front
of people with a whiteboard so you can draw pretty pictures and explain
what’s going on.

Software architecture may be an interesting thing to talk about in email or on the whiteboard.  But such naive attempts will not make the sweeping architectural changes that are necessary to yield noticable improvement.  Only good design, enforced by software tools and disciplined coding practices, can result in well-layered systems with managable complexity.  Much of the windows code itself predates the very tools and practices needed to fix it.  For the fundamental design of Windows to change, you need to go back to the drawing board.

Even if you believed, for a moment, that you could check every line of code and fix every dependency, the math would get you.  As the number of function points increases, the number of side-effects and dependencies increases exponentially.  Even a small software system can have millions of interdependent relationships.  A large system like Vista with 50 million lines of code would have side-effects and causal relationships that would defy analysis.

Confidence Building

In March, when Vista slipped, article after article appeared about the whys and whens of the slip.  The popular jourlalism moved quickly into an editorial stance.  The New York Times article "Burden of the years weighs on Windows" set the stage:

"Windows is now so big and onerous because of the size of its code
base, the size of its ecosystem and its insistence on compatibility
with the legacy hardware and software, that it just slows everything
down," noted David Yoffie, a professor at Harvard Business School.
"That’s why a company like Apple has such an easier time of innovation."

Microsoft was uncharacteristically silent.  I believe, finally, there could be no disagreement.

Whether the problem is as egregious as I say, there certainly is the belief that it has reached a turning point in its lifecycle.  Consumer confidence in Windows behavior has waned, and now the recognition that the underlying operating system is to blame is becoming widely accepted.

If, as part of a bold new strategy, Microsoft announced that the Vista codebase was the end-of-the-line, confidence in future solutions could finally increase.   Instead of fighting the past, the talented teams of Microsoft engineers would be learning from their mistakes.  As it is, there is far too much code and far too many problems for them to do anything other than trudge forward, making it work as best they can.

Conclusions

The Windows codebase is in bad shape.  It’s unlikely that Microsoft, or indeed anyone, can fix it.  I am certain they will create a usable version of Vista.  But, I expect that one year after its release, we will not be looking back, happy that the problems are solved.  Instead, such an albatross of design can only yield new problems, and new challenges for Microsoft.

Get rid of it, and replace it.  But with what?  In the next post (coming in a couple days), I’ll suggest that Linux be a part of a new strategy to revitalize the product line.  It’s good for Microsoft in more ways than you think.

Microsoft’s New Win-Win Strategy

Over the next week or two, I’ll be blogging about Microsoft’s product strategy.  I have received a variety of interesting reactions to Windows Vista: Past its Due Date.  I obviously struck a chord with many people both inside and outside of Microsoft.  In the "Vista" article, I related a tale of another time a major product met its demise: dBASE.  I was there for that one.  But, history repeats itself, and it’s happening with Windows.  Windows is headed down the same path.

Unless Microsoft does something dramatic.  I call it Windows TNG.

Microsoft’s best, and perhaps only opportunity to take their products to the next level involves four simple, bet-the-company steps.  Each follow-up post will go into these steps in detail.  Honestly, I don’t think Microsoft has what it takes to take such bold moves anymore. But, here they are:

1. Scrap the Windows codebase forever.  Release Vista, and announce publicly that it will be the last version of Windows based upon the NT/Win32 platform.
2. Use Linux as the base operating system for the next generation of Windows.  Do not modify it, do not "Microsoftize" it.  Do not try to own it.  Exploit it.
3. Reinvent the Desktop.  Call it Windows.  Windows: The Next Generation.  Outdo Apple, outdo the current platform, outdo every "Linux desktop" effort in existence.
4. Put applications first.  Office TNG, Project TNG, Excel TNG, Outlook TNG.  Do not port.  Rewrite.  Do not create a Win32 compatibility layer.  Do it right.

To some readers, this is an obvious win.  To others, it’s ridiculous.  Some would say it is heresy.  The most analytical would say that it throws away Microsoft’s biggest IP asset, the Windows codebase, and puts Microsoft head-to-head on a level playing field, making them far too vulnerable.  The stock price would plummet.

Maybe so.  But, once you realize that the codebase is the problem, you also realize Microsoft has to devise something better.

My early career was fueled by the exitement that Microsoft brought to the playing field.  They not only created a C compiler for me, they created a better one.  They not only gave me a graphical desktop.  They gave me a better one.   They not only created a better way for the novice to write windows applications, they amazed the world with VB and revolutionized the desktop development environment.  They were hungry, unfettered by legacy strategies and technologies.  They were the underdog, and they were my champion.

So, rather than bitch and moan (which is so easy), I’ll go into details about these steps in a multi-part post.  I hope it’s useful.

Ballmer: Family Values vs. Employee iPods

I know Ballmer’s going to take a lot of flak for this comment from today’s CNN article:

Do you have an iPod?
No, I do not. Nor do my children. My children–in many dimensions
they’re as poorly behaved as many other children, but at least on this
dimension I’ve got my kids brainwashed: You don’t use Google, and you
don’t use an iPod.

Though perhaps an offhand comment, I assume it’s not a joke. 

I think Steve needs to step back and listen to some of his employees.  Maybe they really do know quite a bit that Steve doesn’t.  Let’s see…

Last February’s Wired article "Hide your iPod, Here Comes Bill" reveals that Steve Ballmer is in the minority:

"About 80 percent of Microsoft employees who have a portable music
player have an iPod," said one source, a high-level manager who asked
to remain anonymous. "It’s pretty staggering."
…
So concerned is management, owning an iPod at Microsoft is beginning to
become impolitic, the manager said. Employees are hiding their iPods by
swapping the telltale white headphones for a less conspicuous pair.

A year later, it seems management is not only concerned, but positively paranoid.  Many Microsoft employees see things differently than Steve and Bill.  Tom Harpel, a Microsoft Employee who is not hiding his headphones, understands why understanding competing products can be empowering:

Yes, I use a Mac. I love using a Mac. Yes, I
carry an iPod. I don’t love it, but it works pretty well. I have a TiVo
today, but I’m sure I’ll be adding a Media Center to my living room
sometime in the next year.

I like toys. I like gadgets. Every
product out there tries to use technology to solve a problem. It’s fun
and enlightening to try stuff out, to try to understand how each
company approaches problem solving differently. Using competitor’s
products is one way to get at that understanding.

Today in Paul Kedrosky’s Blog, Robert Scoble also had no hesitation in his open-mindedness about the competition: