All Things Distributed

Amazon Move FAQ #4: Why not Microsoft?

From the many questions I received about why I didn't join Microsoft, you would almost think that people had made bets on how long it would take before I would assimilated by MS. After all it didn't seem like an unfair expectation. In '95 we were the first researchers to switch our operating systems and distributed systems research to Windows NT. SunOs was being phased out and the source license on Solaris was very unfriendly for universities. Microsoft stepped in with a license that made it easy to use kernel source in classes and projects, and we were psyched to get our hands on an OS with a more modern structure. It also felt good to work on a platform with which you could reach many real people, and have an opportunity to have real impact on everyday computing.  The biggest stumbling blocks in retrospect have been our colleagues and their skepticism about ' you can't do research on NT!', or that we would be just front men for the Borg's message. In 1999 I wrote about some of my experiences in an article: "Windows 2000 Research Edition. Where the Academic Knights meet the Evil Empire..." (PDF). In retrospect I think we proved everybody wrong, and there is a lot more exciting systems research happening today with Windows as a platform.

There are a lot of smart people at Microsoft. Over the year I have had dealings with quite a few product groups and I also come away amazed about the depth of knowledge on many related subject areas. I worked for quite some time with the Wolfpack team (clustering), and there were people on that team that were equally smart or smarter than many of my Cornell students. It was fun to work with them. Quite a few of the architects at MS are visionaries who are worth listening to every time they speak (Jim Gray, Pat Helland, Felipe Cabrera, Butler Lampson).

Now that I have professed my love again for the evil empire, I still haven't answered the question at hand. When I started to take Amazon's offer seriously I also asked myself the question: If I was now willing to move to Seattle, why not move to Redmond? In the end there are a number of different reasons, but two fundamental technology ones stick out:

• The challenges that Amazon faces are orders of magnitude different from the solutions that Microsoft is working on. Microsoft and others are developing their software targeting the largest market. Internet giants such as Amazon are not a big market, and their challenges need to be addressed by significantly different architectures, which would take a huge investment by MS and colleagues to deliver on. And it will not sell them millions of additional OS licenses. For someone like me who is interested in scalability and robustness Amazon clearly has the more challenging problem set.
• Related to this is that at Microsoft, and at many other middleware and system software vendors, you develop software that will be used by other developers who will build products for their customers to use. This always puts you two or sometime three steps away from the ultimate requirements. To support this you need to be as generic as possible, as you have to cater to a very large, diverse customer base (developers). At Amazon one has a view over the whole pipeline. The end-to-end requirements are clear from the start, and every step in the software construction process allows you to look for solutions that are specific for the problem, for the service or for the data used.  There is no illusion that one single approach or platform will solve all the problems. You need a rather diverse toolbox to be able to build composable, highly scalable internet services at the scope and scale Amazon wants to offer. For a long time I have worked on generic approaches and the criticism you always get makes it feel that it is never good enough. I am very excited to for a change be able work on very specific solutions for which it is clear when it is good enough, or what exactly is needed to make it better.

These are only two of the reasons why at this point in my professional life Amazon is the more exciting choice. But I am very happy that Pat Helland and Felipe Cabrera are only a few highway crossings away, and that there will more opportunities to meet up with Jim Gray instead of less. These are people that continue to educate me, where ever I will work.

This is entry #4 in the Moving to Amazon - Frequently Asked Questions list