Tuesday, May 5, 2015

A modest proposal: SOSIGCOMMOBIXDI

I have a problem: there are way too many conferences to attend. Even worse, the degree of overlap between the systems, mobile, and networking communities means that I am basically running into the same people at all of these events. You have a problem, too: You are paying money (and time) to attend all of these separate conferences.

Conservatively, there are five "top tier" conferences that are "must attend" events every year: SOSP/OSDI, NSDI, MobiSys, MobiCom, SIGCOMM. (Not to mention excellent venues like USENIX ATC, EuroSys, CoNext, SenSys, the list goes on.) And then all of the "smaller workshops because we don't like how big the conferences are but you pretty much have to go anyway": HotOS, HotMobile, HotNets.

Realistically, nobody makes it to all of these events (unless you're, say, a poor junior faculty member going for tenure and have to show your place in as many places as possible). So you pick and choose based on whether you have a paper accepted, or whether you have enough travel money laying around, or whether you just have to get away from home for a few days.

Consider the costs of running all of these separate events. For the attendees, there is the high cost (and time investment) for travel, registration fees, and taking time away from work and home to attend each conference. A single conference trip probably costs $1500-2000, more if you are traveling overseas, and anywhere from three days to a week of time away from home. Especially for those with young children at home each trip takes a serious toll.

Organizing a conference is also a huge amount of work, regardless of whether it's a workshop for 50 people or a big conference for 500. This is especially true for the poor general chair who has to work out all of the details of the venue, hotel, meals, A/V setup, finances, etc.

You know where this is going: Why don't we have one, big, annual conference spanning the systems, networking, and mobile research communities? (And, while we're at it, why not throw in databases for good measure?) SOSIGCOMMOBIXDI would run for, say, 5 days, with parallel (yes!) sessions covering each of these major areas. It would happen at roughly the same week each year, so people can plan their travel and vacation schedules well in advance. It's like FCRC, for systems!

I can hear the objections now! Let me take them one by one.

But won't it be too big? SIGCOMM and SOSP/OSDI already have something like 600 people in attendance; these are hardly close-knit communities. Given the amount of overlap across these various conferences, I estimate that there would be no more than 3,000 people attending SOSIGCOMMOBIXDI, although I'll grant I might be underestimating -- let's be generous and say 5,000 people. Organizing an event for 5,000 people is no big deal. Most large cities have hotels and convention centers that can comfortably handle events of this size. Hell, medical conferences typically have 10,000 or more (way more) attendees. It is understood how to run conferences at this scale. It's not something a typical professor has experience doing, so best to rely on a professional events organization like USENIX.

I have been to 5,000-person conferences and if anything, it's more energizing -- and there is much more to do -- than these 500-person events where everyone is expected to sit in the same room listening to the same talks all day long. You have room for smaller breakouts and workshops; a larger, more interesting industry presence; and greater draw for really interesting keynote speakers.

But I want a single track! Get over it. The single-track "constraint" is often cited by senior people who remember what it was like in the early days of the field when conferences were 1/5th the size that they are now, and every PC member read every paper. The people who complain about parallel tracks are often the ones who spend most of the conference out in the hall chatting with their colleagues -- they're not listening to every talk anyway. Even if they sit in the room all day they're probably on their laptops pretending to listen to the talk, or writing blog posts (like I'm doing now).

Ever been to the morning session on the third day of a conference? Crickets. Where are all of the "single-trackers" then?

Moreover, the single-track "constraint" severely limits the number of papers a conference can publish every year. Most 2.5-day conference can take no more than 25-30 papers to fit in a single track model. To squeeze more papers in, we've gotten rid of the more memorable aspects of these conferences: keynotes, panels, breakouts. It doesn't scale.

Removing the single-track requirement also opens up a bunch of possibilities for changing up the format of the conference. Sure, you want some large plenary sessions and a few tracks of papers. But hosting a few mini-workshops, working sessions, or BoFs during the day is possible too. Squeeze in poster and demo sessions here and there. Even set some space aside for an industry trade show (these are often really fun, but most academic conferences rarely have them).

Worried you're going to miss something? The papers are all online, and USENIX even posts videos of all of the talks. So, I claim that the single-track model is outdated.

But then there's only one paper submission deadline a year! Not necessarily. We could have rolling submissions for SOSIGCOMMOBIXDI, much like SIGMOD and some other venues do. Since SOSIGCOMMOBIXDI practically consists of multiple federated events, each one can have its own set of deadlines, and they could be staggered across sub-events. Paper submission and evaluation are only loosely coupled to the timing of the conference itself.

But ACM and USENIX won't get as much conference registration revenue if there's only one event! Oh, I hadn't thought of that.

Thursday, April 30, 2015

Flywheel: Google's Data Compression Proxy for the Mobile Web

Next week, we'll be presenting our work on the Chrome Data Compression proxy, codenamed Flywheel, at NSDI 2015. Here's a link to the full paper. Our wonderful intern and Berkeley PhD student Colin Scott will be giving the talk. (I'm happy to answer questions about the paper in the comments section below.)

It's safe to say that the paper would have never happened without Colin -- most of us are too busy building and running the service to spend the time it takes to write a really good paper. Colin's intern project was specifically to collect data and write a paper about the system (he also contributed some features and did some great experiments). It was a win-win situation since we got to offload most of the paper writing to Colin, and he managed to get a publication out of it!

Rather than summarize the paper, I thought I'd provide some backstory on Flywheel and how it came about. It's a useful story to understand how a product like this goes from conception to launch at a company like Google.

(That said, standard disclaimer applies: This is my personal blog, and the opinions expressed here are mine alone.)

Backstory: Making the mobile web fast

When I moved to Seattle in 2011, I was given the mission to start a team with a focus on improving mobile Web performance. I started out by hiring folks like Ben Greenstein and Michael Piatek to help figure out what we should do. We spent the first few months taking a very academic approach to the problem: Since we didn't understand mobile Web performance, of course we needed to measure it!

We built a measurement tool, called Velodrome, which allowed us to automate the process of collecting Web performance data on a fleet of phones and tablets -- launching the browser with a given URL, measuring a bunch of things, taking screenshots, and uploading the data to an AppEngine-based service that monitored the fleet and provided a simple REST API for clients to use. We built a ton of infrastructure for Velodrome and used it on countless experiments. Other teams at Google also started using Velodrome to run their own measurements and pretty soon we had a few tables full of phones and tablets churning away 24/7. (This turned out to be a lot harder than we expected -- just keeping them running continuously without having to manually reboot them every few hours was a big pain.)

At the same time we started working with the PageSpeed team, which had built the gold standard proxy for optimizing Web performance. PageSpeed was focused completely on desktop performance at the time, and we wanted to develop some mobile-specific optimizations and incorporate them. We did a bunch of prototyping work and explorations of various things that would help.

The downside to PageSpeed is that sites have to install it -- or opt into Google's PageSpeed Service. We wanted to do something that would reach more users, so we started exploring building a browser-based proxy that users, rather than sites, could turn on to get faster Web page load times. (Not long after this, Amazon announced their Silk browser for the Kindle Fire, which was very much the same idea. Scooped!)

Starting a new project

Hence we started the Flywheel project. Initially our goal was to combine PageSpeed's optimizations, the new SPDY protocol, and some clever server-side pre-rendering and prefetching to make Web pages load lightning fast, even on cellular connections. The first version of Flywheel, which we built over about a year and a half, was built on top of PageSpeed Service.

Early in the project, we learned of the (confidential at the time) effort to port Chrome to Android and iOS. The Chrome team was excited about the potential for Flywheel, and asked us to join their team to launch it as a feature in the new browser. The timing was perfect. However, the Chrome leadership was far more interested in a proxy that could compress Web pages, which is especially important for users in emerging markets, on expensive mobile data plans. Indeed, many of the original predictive optimizations we were using in Flywheel would have resulted in substantially greater data usage for the user (e.g., prefetching the next few pages you were expected to visit). It also turned out that compression is way easier than performance, so we decided to focus our efforts on squeezing out as many bytes as possible. (A common mantra at the time was "no bytes left behind".)

Rewriting in Go

As we got closer to launching, we were really starting to feel the pain of bolting Flywheel onto PageSpeed Service. Originally, we planned to leverage many of the complex optimizations used by PageSpeed, but as we focused more on compression, we found that PageSpeed was not well-suited to our needs, for a bunch of reasons. In early 2013, Michael Piatek convinced me that it was worth trying to rewrite the service, from scratch, in Go -- as a way of both doing a clean redesign from scratch but also leveraging Go's support for building Google-scale services. It was a big risk, but we agreed that if the rewrite wasn't bearing fruit in just a couple of months that we'd stop work on it and go back to PageSpeed.

Fortunately, Michael and the rest of the team executed at lightning speed and in just a few months we had substantially reimplemented Flywheel in Go, a story documented elsewhere on this blog. In November 2013 I submitted a CL to delete the thousands of lines of the PageSpeed-based Flywheel implementation, and we switched over entirely to the new, Go-based system in production.

PageSpeed Service in C++ was pushing 270 Kloc at the time. The Go-based rewrite was just 25 Kloc, 13Kloc of which were tests. The new system was much easier to maintain, faster to develop, and gave our team sole ownership of the codebase, rather than having to negotiate changes across the multiple teams sharing the PageSpeed code. The bet paid off. The team was much happier and more productive on the new codebase, and we managed to migrate seamlessly to the Go-based system well before the full public launch.


We announced support for Flywheel in the M28 beta release of Chrome at Google I/O in 2013, and finally launched the service to 100% of Chrome Mobile users in January 2014. Since then we've seen tremendous growth of the service. More than 10% of all Chrome Mobile users have Flywheel enabled, with percentages running much higher in countries (like Brazil and India) where mobile data costs are high. The service handles billions of requests a day from millions of users. Chrome adoption on mobile has skyrocketed over the last year, and is now the #1 mobile browser in many parts of the world. We also recently launched Flywheel for Chrome desktop and ChromeOS. Every day I check the dashboards and see traffic going up and to the right -- it's exciting.

We came up with the idea for Flywheel in late 2011, and launched in early 2014 -- about 2.5 years of development work from concept to launch. I have no idea if that's typical at Google or anywhere else. To be sure, we faced a couple of setbacks which delayed launch by six months or more -- mostly factors out of our control. We decided to hold off on the full public release until the Go rewrite was done, but there were other factors as well. Looking back, I'm not sure there's much we could have done to accelerate the development and launch process, although I'm sure it would have gone faster had we been doing it as a startup, rather than at Google. (By the same token, launching as part of Chrome is a huge opportunity that we would not have had anywhere else.)

What's next?

Now that Flywheel is maturing, we have a bunch of new projects getting started. We still invest a lot of energy into optimizing and maintaining the Flywheel service. Much of the work focuses on making the service more robust to all of the weird problems we face proxying the whole Web -- random website outages causing backlogs of requests at the proxy, all manner of non-standards-compliant sites and middleboxes, etc. (Buy me a beer and I'll tell you some stories...) We are branching out beyond Flywheel to build some exciting new features in Chrome and Android to improve the mobile experience, especially for users in emerging markets. It'll be a while until I can write a blog post about these projects of course :-)

Thursday, February 12, 2015

How do you become a conference chair?

I've served as program chair and general chair for a number of conferences, but the conditions under which I got asked to serve in these roles were generally a mystery to me. Now that I've served on the steering committee for several conferences, I've seen how the sausage is made and what factors contribute to an individual being asked to be the PC or general chair.

If you're a junior faculty member or a relatively new industry researcher, you might be wondering how you get yourself in the right position to be tapped for these roles. A few words of advice that might be helpful.

Should you chair a conference?

Being a program or general chair requires a tremendous amount of time and energy. You should only accept such a job if you can really do a good job of it. In the past I have declined invitations for events that I didn't feel that I could really devote the appropriate amount of energy to. It was easy to turn down by saying "I don't feel that I have the cycles to do an incredible job". Doing a poor job of chairing will potentially tarnish your reputation and make it difficult for you to get such gigs in the future.

In general I would recommend that you only chair well-known and high-profile conferences and workshops. Way too many people seem to think that the way to make their mark on the research world is by starting up a new conference or workshop in some area, and finding some sucker to serve as the chair. As a result we have a proliferation of third-tier workshops on various topics which don't generally attract good paper submissions, good PC members, or good attendees. These are usually a waste of time.

Of course as a junior faculty member you don't get asked to chair, say, SOSP in your first or second year on the job. You have to start somewhere. Chairing a few smaller or lower-quality events is fine and gives you experience, but be extremely judicious in selecting which venues to chair. I made the mistake once or twice of agreeing to chair for a brand-new workshop which did not clearly have the buy-in from the rest of the community that it needed to exist. As a result I had a hard time getting good people to agree to be on the PC, and the submissions we got were pretty crappy. I should have said no to this (although serving as chair sounded like a great thing for my career at the time).

Is being general chair worth it?

For junior faculty, generally not. The program chai is the job you want. The general chair's job is to do all of the scut work of organizing the conference -- finding the hotel, dealing with finances, deciding on the banquet menu, begging companies for money. This work really sucks and delivers little intellectual satisfaction (unless you really like negotiating hotel contracts). The only reason to do this job is if the venue is really well known and you will have a chance to be fairly high profile in terms of structuring the whole event and making your mark on it. I usually think it's better for senior, more well-established people to be general chairs since they don't need to do it for career-advancement reasons.

I agreed to be general chair for HotMobile last year. I knew this wouldn't advance my career in any meaningful way, certainly not at Google. I did it because I like HotMobile a lot and wanted to help the community. As a junior faculty member devoting this much time to something like this would have been a mistake.

How do you get tapped?

Most conferences have a steering committee that makes the decision about who will be the program and general chair for the following year's conference. Sometimes the steering committee has standing members and sometimes it rotates (and sometimes a combination of the two).

Getting chosen for a chairship comes down to a bunch of factors.

Overall visibility: You need to be known in the community. This means showing up for the conferences, getting to know people, serving on program committees, and generally being there. If you don't ever come to a conference the chances you'll be asked to serve as its chair are vanishingly low. Note that publishing papers in these conferences is helpful but not strictly a requirement (I don't publish much anymore and keep getting asked to do things...) Being seen is more important than being published in this regard.

Being responsible: Generally you establish your reputation as a community member by serving on program committees and doing other volunteer activities like running poster/demo sessions and the like. If you serve in these roles, it's important to be incredibly responsible: Don't miss paper review deadlines, keep people in the loop, deliver on what you promised. I've known faculty who are perennially tardy with paper reviews, don't show up in person to the PC meeting, and so forth. These people are less likely to be asked to be a chair.

Career stage: Many conferences confer chairships on up-and-coming faculty who are at a certain stage in their career (e.g., about to come up to tenure) who are less likely to say no (and also more likely to do an excellent job).

Knowing the right people: This is part of overall visibility, but if you don't have any kind of personal relationship with the folks who will eventually be making these decisions, it's less likely they'll think of you when the time comes. Going to conferences, going out for dinner and drinks after the conference, and generally being chummy with other senior folks in the field is pretty important. I know plenty of people who do second-rate research but are seen as members of the inner cabal mainly because they always show up to the conference and are up for getting beers. (Indeed this may well explain most of my own career path!)

Thursday, January 22, 2015

Day in the Life of a Google Manager

Not long after joining Google back in 2010, I wrote this cheeky piece contrasting my daily schedule at Google with my previous career as an academic. Looking back on that, it's remarkable how much my schedule has changed in four years, in no small part because I'm now managing a team and as a result end up doing a lot less coding than I used to.

So, now seems like a good time to update that post. It will also help to shed some light on the differences between a pure "individual contributor" role and the more management-focused role that I have now.

By way of context: My role at Google is what we call a "tech lead manager" (or TLM) which means I'm responsible both for the technical leadership of my team as well as the people-management side of things. I've posted more details about the TLM role elsewhere, so I won't repeat that here. Our team has various projects, the largest and most important of which is the Chrome data compression proxy service. We're generally interested in making Chrome work better on mobile devices, especially for users in slow, expensive networks in emerging markets.

The best part of my job is how varied it is and every day is different (and I usually have a lot of balls in the air). The below is meant to represent a "typical" day although take that with a grain of salt given the substantial inter-day variation:
6:45am - Wake up, get the kids up and get them ready and make them breakfast, shower. 
8:30am - Jump on my bike and ride to work (which takes about 10 minutes), grab breakfast and head to my desk. 
8:45am - Check half a dozen dashboards showing various metrics for how our services are doing -- traffic is up, latency and compression are stable, datacenters are happily churning along. 
9:00am - Catch up on email. This is a continuous struggle and a constant drain on my attention, but lately I've been using Inbox which has helped me to stay afloat. Barely. 
9:30am - Work on a slide deck describing a new feature we're developing for Chrome, incorporating comments from one of the PMs. The plan is to share the deck with some other PM and eng leads to get buy-in and then start building the feature later this quarter. 
10:00am - Chat with one of my teammates about a bug report we're chasing down, which gets me thinking about a possible root cause. Spend the next half hour running queries against our logs to confirm my suspicions. Update the bug report with the findings. 
10:30am - I somehow find my morning has not been fully booked with meetings so I have a luxurious hour to do some coding. Try to make some headway on rewriting one of our MapReduce pipelines in Go, with the goal of making it easier to maintain as well as adding some new features. It's close to getting done but by the time my hour is up one of the tests is still failing, so I will spend the rest of the day quietly fuming over it. 
11:30am - Meet with one of my colleagues in Mountain View by video hangout about a new project we are starting up. I am super excited to get this project going. 
12:00pm - Swing by the cafe to grab lunch. I am terrible about eating lunch at my desk while reading sites like Hacker News - some habits die hard. Despite this, I still do not have the faintest clue how Bitcoin works.
12:30pm - Quick sync with a team by VC about an internal summit we're organizing, to plan out the agenda. 
1:00pm - Hiring committee meeting. We review packets for candidates that have completed their interview loops and try to decide whether they should get a job offer. This is sometimes easy, but often very difficult and contentious, especially with candidates who have mixed results on the interview loop (which is almost everyone). I leave the meeting bewildered how I ever got a job here.
2:00pm - Weekly team meeting. This usually takes the form of one or more people presenting to the rest of the team something they have been working on with the goal of getting feedback or just sharing results. At other times we also use the meeting to set our quarterly goals and track how we're doing. Or, we skip it.
3:00pm - One-on-one meeting with a couple of my direct reports. I use these meetings to check in on how each member of the team is doing, make sure I understand their latest status, discuss any technical issues with their work, and also talk about things like career development, setting priorities, and performance reviews. 
4:00pm - Three days a week I leave work early to get in an hour-long bike ride. I usually find that I'm pretty fried by 4pm anyway, and this is a great way to get out and enjoy the beautiful views in Seattle while working up a sweat. 
5:00pm - Get home, shower, cook dinner for my family, do some kind of weird coloring or electronics project with my five-year-old. This is my favorite time of day. 
7:00pm - Get the kids ready for bed and read lots of stories. 
8:00pm - Freedom! I usually spend some time in the evenings catching up on email (especially after having skipped out of work early), but try to avoid doing "real work" at home. Afterwards, depending on my mood, might watch an episode of Top Chef with my wife or read for a while (I am currently working on Murakami's 1Q84).
Compared to my earlier days at Google, I clearly have a lot more meetings now, but I'm also involved in many more projects. Most of the interesting technical work is done by the engineers on my team, and I envy them -- they get to go deep and do some really cool stuff. At the same time I enjoy having my fingers in lots of pies and being able to coordinate across multiple active projects, and chart out new ones. So it's a tradeoff.

Despite the increased responsibilities, my work-life balance still feels much better than as an academic. Apart from time-shifting some of my email handling to the evening (in order to get the bike rides in), I almost never deal with work-related things after hours or on the weekends. I have a lot more time to spend with my family and generally manage to avoid having work creep into family time. The exception is when I'm on pager duty, which is another story entirely -- getting woken up at 3 am to deal with a crash bug in our service is always, um, exciting.

Wednesday, October 15, 2014

You'll be fine in Silicon Valley

Yesterday, a group of well-known professors from a bunch of universities sent an open letter to Microsoft Research's leadership, condemning them for closing the Microsoft Research Silicon Valley lab and leaving the researchers there "jobless". Although I know and respect many of the authors of this letter, I think they get it wrong when it comes to the career opportunities for the Microsoft researchers impacted by the lab's closing, and more broadly for the impact this closing will have on the research community.

The letter's tone is incredibly haughty -- a mixture of "how dare you" and "think of the children!!" What worries me the most is the message the letter seems to be sending to junior researchers who may feel that industry jobs are not a viable career path, because, hey, you might get fired if your lab closes. I think we should correct some of these misconceptions.

(Standard disclaimer: This is my personal blog and the opinions expressed here are mine alone, and most certainly not that of my employer.)

Let's take the letter point by point.

By now, you are no doubt aware of the research community’s shock and disappointment at the sudden and harsh way in which the members of the Microsoft Research Silicon Valley lab were dismissed a few weeks ago.
While the news was sudden, I don't have any evidence that it was "harsh". That's how layoffs sometimes work. Yes, it sucks, but don't imply malice where none is intended.

We are writing to share our perspective on the negative impacts of the shutdown and to open a dialogue about the ways in which Microsoft can try to restore the environment that enabled MSR to produce such great research in the past, benefiting both the company and the world.
The implication here is that by closing the Silicon Valley lab, Microsoft has damaged "the environment" that enabled them to "produce such great research". Let's see here. By my count, about 75 researchers out of a worldwide organization of "more than 1,000 scientists and engineers" were affected. While this is unfortunate, I don't think it's going to seriously hinder Microsoft's ability to do great work in the future. It's still a great place and I expect it will continue to grow.

While layoffs are always unpleasant, the impact of this one has been exacerbated by the fact that many researchers at the Silicon Valley lab worked on long-term, fundamental research of the kind that can be done at very few places outside of academia.
The fact that "very few places outside of academia" allow one to do "long-term, fundamental research" should be teaching us something. The authors of this letter seem to believe that doing academic-style research is some kind of special, protected occupation that people with PhDs are entitled to. While there may have been plenty of industry labs doing pure, academic-style research 20 or 30 years ago, the world has changed. What worries me is that professors still cling to an industry research model that has largely been supplanted by other, better models. I think it's time to evolve our thinking about this in the community and start giving appropriate career advice to students so they are prepared for the reality of what's out there, rather than for an outmoded academic ideal.

It is also a common misconception that one cannot do research in a more product-oriented industry setting. For example, Google does a tremendous amount of research, albeit using a fairly different model than places like MSR. The same is true for other large and small companies as well. We may not call all of these places "research labs" and bestow fancy titles on people like "research scientist", but the research is still happening, and having tremendous impact.

As you know, the academic calendar is such that many of these researchers, including very junior ones who recently completed their PhDs, could be jobless for nearly an entire year. We feel that there should have been a better way to close down this lab, one that would have allowed them to have continuous employment until academic jobs are available again in September 2015. Given that this lab was continuing to produce exceptional — indeed revolutionary — research, we fail to understand why closing it had to be done so suddenly.
This is completely disconnected from reality. First of all, where did anyone get the idea that the only viable job opportunities for the researchers being let go from MSR SV are in academia? From what I can tell, all of these people are being heavily recruited by a number of Bay Area companies. Hell, the MSR SV office is literally off of the same highway exit as the Google headquarters. About the only thing that might change for some of these people is which parking lot they drive to in the morning.

I very seriously doubt that any of these researchers will be "jobless" for any length of time. That's delusional.

Second, as a friend of mine pointed out, it's not as though the "academic calendar" is some kind of law of physics. If universities are serious about recruiting some of these people, I am sure they can find a way to make it happen, even if they have to resort to such lengths as hiring them on short-term contracts as visiting scientists or whatever. The academic calendar is your problem, not Microsoft's.

Over the past two decades, MSR, and indeed all of Microsoft, earned an excellent reputation in academia as an organization that not only valued basic research but also supported the career development of the many researchers that worked in or visited the labs.  That reputation has been significantly damaged, threatening Microsoft’s ability to recruit and retain world-class researchers. 
It's true that shuttering MSR SV will have some impact on how people view Microsoft Research as a career choice. On the other hand, it's not as though they hired a huge number of people every year -- the total number of jobs impacted is relatively small. In comparison, Google hires multiple MSR SVs worth of people every week, and more PhDs per year than all of MSR employs, worldwide. (For that matter, Microsoft, as a whole, no doubt beats Google's numbers.) It's not like the loss of a small fraction of MSR's total headcount has anything but a minor impact on the overall pipeline for computer scientists.

What's really happening here is a bunch of finger-wagging in an effort to publicly shame Microsoft for what was no doubt a very difficult and complex business decision.

As faculty members, we can no longer recommend it as highly to our students as a place to start their careers.  In the long term, this move seems likely to adversely affect Microsoft Research (and the positive contributions it makes to Microsoft as a whole) in more ways than any benefit it may have had.
I think Microsoft should appoint a panel of academics to run the company, since clearly they know more about running a business than MSR leadership does.

Nevertheless, we believe that Microsoft can reduce the damage that has been caused by the shutdown of the Silicon Valley lab.  We understand that Microsoft is considering ways to help care for the researchers who were dismissed, such as defraying the additional costs of the academic organizations who are trying to provide these researchers with temporary homes. This would be an excellent, and highly appreciated, first step.
Of course universities must bear the cost of supporting these poor refugees while they find new jobs! I mean, it's not like the Microsoft stock and severance package will pay the bills for very long until the affected people get jobs at Yahoo or Twitter. Hopefully the universities can pull through with basic rations, perhaps a tattered wool blanket and a cot to sleep on, an office with an aging workstation running XENIX, just until these people can find gainful employment. It would be a real humanitarian disaster otherwise.

Looking forward, we hope that you will open a discussion with us and the community about Microsoft’s vision for industrial research (which has become less clear after the closing of what appeared to be an extremely valuable and successful lab) and concrete commitments MSR can make regarding the career development of its remaining and future researchers. 
Frankly, I don't think that the MSR leadership owes the academic community any kind of explanation or commitments whatsoever. If anything, this was a great wake-up call to anyone who was living under the false impression that being a "researcher" at a company gave you job security. This is how companies sometimes work.

Fortunately, especially in Silicon Valley, there are literally hundreds of great companies that the MSR SV folks can work for, doing all kinds of awesome and groundbreaking work. The tech industry as a whole is going strong, and I have no doubt that there will be all kinds of career opportunities for these folks.

Steps like these are essential to rebuilding the relationship between Microsoft and the academic community, along with all the mutual benefits that it brings.
On the whole, my guess is that the relationship between Microsoft and the academic community will continue to thrive. There are a lot of great people still at MSR and the ties there are incredibly strong. It's too bad that they had to shut down the Silicon Valley lab in the way that they did, but I'm not sure it's productive to cast a dark cloud over everything MSR does and stands for as a result.

Mostly, I'd like the authors of this letter to think about the message they are sending to students and junior researchers, which seems off base to me. We're all sad about the loss of MSR SV, but I think this letter really goes too far when it claims that anybody there will be "jobless" or that academic positions must open up to absorb the affected researchers.

Monday, August 4, 2014

The Fame Trap

The other day I was speaking with a group of interns about the differences between the academic and industrial career paths. One of them mentioned that when you join a company like Google, you lose your identity -- that is, people outside the company may not know much about what you do or work on. I like to think of it like getting assimilated into the Borg. This is a concern I hear a lot from grad students.

First off, this is absolutely true. Unless you work really hard at it, being in industry (not counting industrial research labs, of course) does not afford you as many opportunities to stay visible in the research community. No big surprise here -- at a company like Google, your job isn't to publish papers, it's to build products. You can publish papers, serve on program committees, and go to conferences -- but when academic research not your main job, doing those things isn't necessarily a priority.

I think many grad students get fixated on this idea of cultivating their academic profile and tend to make career decisions that build on that, to the exclusion of some other ways in which they could have a mark on the world. (This absolutely happened to me.) As an academic, your entire career is focused, to some degree, on building and maintaining your personal brand. In my experience, though, the "fame" you enjoy as an academic is on a fairly small scale, and it takes a tremendous amount of energy to perpetuate. That is, there are many reasons for wanting to pursue an academic career, but I would argue that achieving prestige in the research community isn't a particularly good one.

Let's keep things in perspective -- renown in the academic world is on a tiny scale. Take the systems community: a typical systems conference will have no more than about 400 attendees. Let's assume that roughly 10% of the people in the community (that matter!) are actually going to the major conferences -- so we're talking (generously) on the order of 10,000 people or so who might be active in the systems community at any given time.

I'm sorry, but being "famous" in a tight-knit community of O(10K) academics ain't really being famous. That's nowhere near the level of Kanye, or even Neil deGrasse Tyson. In fact, I can't name a single academic computer scientist who enjoys that level of fame. Maybe Alan Turing, but his fame is largely posthumous. There are the rare academics who are well-known across CS -- David Patterson comes to mind -- but even he's not exactly a household name. (Unless your household involves a lot of dinnertime conversations about disk arrays and processor architecture.)

As a grad student, it's pretty easy to fall into the fame trap, and I speak from experience. When I finished my PhD, I was pretty much dead set on having an academic career, because that seemed to be the most glamorous, high-profile career path at the time. (Keep in mind that when I finished my PhD in 2002, I had an offer from pre-IPO Google, and could have been something like employee #400. Now I'm just employee #40,000.) I was enamored with the prestige of being a professor. I dreamt of running my own lab, having my own students, being as "famous" as my advisor. And of course, from having been in grad school for so long, building up my academic profile was pretty much the only world I knew.

What I didn't realize at the time was how much work it would be to maintain an academic reputation. To be visible, and stay visible, in the research community means going to countless conferences, serving on program committees and panels, visiting other universities to give talks, and of course publishing as many papers as you possibly can. Getting tenure is driven in large part by how well known you are and how much "impact" you're having. If you fail to do these things, your stature in the community slips pretty quickly, unless you are really well-established and can afford to just show your face from time to time. (Junior faculty need not apply.)

I see this all the time in my academic colleagues. They travel constantly -- one or two trips per month is typical. They say yes to a ridiculous number of committees, advisory boards, whatever. They like to "complain" about the amount of travel and committee work they do, but I'm sure they all realize that they can't really stop doing it, lest they stop getting invited to do these things. (Kind of a strange vicious cycle, that.) It's like being in a small-time folk band that has to tour constantly to keep paying the bills -- you never get a break.

I applaud my academic colleagues who can still do this. For me, personally, I never really enjoyed it, and I found it wasn't worth the sacrifice just to maintain the status quo of my academic reputation. Once I had kids, I really started to appreciate the toll it was having on my family to be out of town so often, and I started to realize that maybe I had my priorities all wrong.

These days I still enjoy being part of the academic community, but in a more selective way. I like to go to occasional conferences and serve on PCs that are very closely aligned with my area, because I learn a lot about what's going on in the research world. It's also a good way to recruit interns and full-time hires, and of course I have lots of academic friends I like to meet up with and have beers with. I'll admit I still like giving talks from time to time. I've done very little publishing since leaving academia, but that's by choice -- I'm spending my time on things that are more important (to me) than writing more papers.

None of this is to say that academia can't be a wonderful career path, but I think chasing academic fame is not the best reason to go down that path. I wish I had known that when I was finishing my PhD.

Saturday, May 31, 2014

The NCSSM 2014 commencement address

This morning I had the distinct pleasure of delivering the commencement address at the North Carolina School of Science and Math, my alma mater, where I graduated in 1992. NCSSM is a public residential high school in which students attend their junior and senior years, and of course as the name suggests has an emphasis on science and math curriculum (but it is so much more than that). For some reason, the students and administrators asked me to give the commencement address this year, which is one of the strangest things I've ever had to do -- in part because it felt like yesterday that I graduated from there (yet it was 22 years ago)!

NCSSM is dear to my heart and those two years were a transformative time in my life. I've blogged before about how my time at S&M saved my life -- it is difficult to overstate how important the place was to my formative years. (Actually I think I'm still "forming".)

I wore Google Glass to give the speech, thinking this would be a really cool and unique way to show off my "Google-ness", but actually one of the graduating seniors was wearing Glass as well:

So much for being unique.

The school live-streamed the event and I was surprised to find a video of my speech online soon after it was over. Watch here at your own peril.

Below I'm posting my prepared speech (which is not exactly the same as the one I actually gave).

Students, professors, parents, mom, (hi mom!), it is most humbling to stand before you here today. Indeed, I am terrified. Somehow I think I am being punished for all of the misdeeds I was involved with back during my days as a student here so many years ago. Either that, or the fact that I am here must be proof that there is no such thing as a permanent record -- if they had one on me, well, let’s just say they would have thought twice about inviting me back to give a commencement address. So, those of you who stayed out way too late last night -- there is hope for you yet! 
Before we begin, I want to get one thing out of the way. Some of you may be wondering what this thing is that I’m wearing. This is called Google Glass, and basically, it’s a cell phone that you strap to your head. I am told that in some neighborhoods of San Francisco it either makes you utterly irresistible to the opposite sex, or it gets you beat up. Google Glass has all kinds of amazing features, and this little screen up here is feeding me a constant stream of information about each and every one of you … yes, you in the back there… the one who made a B-minus in history last term… Google Glass is telling me that those tennis shoes you ordered off of Amazon last week will be delivered this afternoon. You’re welcome.
It’s really remarkable to be back here, twenty-two years after I graduated from NCSSM, on this very lawn. Of course, I don’t remember much about the graduation ceremony itself, seeing as how I got absolutely no sleep the night before. You see, I stayed up too late with my friends and staggered back to my room at 5 o’clock in the morning and realized I had not started packing my room. At all. And my parents were arriving in just a few short hours, expecting me to be rested, cheery, but most of all packed and ready to go. So I frantically threw everything I owned into boxes, tore all of the Depeche Mode posters off of the walls, packed away my IBM PC and boxes full of diskettes, and tried to make some order out of chaos. I was only partially successful, but my parents understood, and we loaded everything into the minivan, and away we went. Sorry, mom! You guys were great about it though.
Poof. Just like that, my NCSSM experience was over. They really were two of the best years of my life, in so many ways -- and the experiences I had here utterly changed the course of my life. I am sure most of you feel the same way already, but the ripple effect will be felt for years to come. Look around -- look at your friends, your teachers, your RAs. This place has made a deep and enduring mark on who you are and who you will be.
NCSSM taught me the single most important lesson I’ve ever learned in my life, and I want to share it with you.
It’s very simple: Never stop looking. That is, never stop looking for who you are, where you belong, and what you might do with your short time on this planet. Sometimes it takes a really, really big change to figure that out.
Before coming to S&M -- as we called it affectionately in those days -- I grew up in a small town called Wilson, about an hour east of here. I cannot say that I really loved Wilson, and I didn’t fit in well there at all. For one thing, I didn’t have a car, so I could not join the legions of kids who would spend their Friday nights driving in circles around the mall, which was pretty much the only entertainment there. But more than that, with the exception of a few close friends, I never quite felt like I found kindred spirits -- people with the same weird and geeky interests that I had. People who saw the world in the same way.
Fortunately, I was admitted to Science and Math, and I am dead serious when I say that coming here saved my life. Almost as soon as I arrived here, I realized that I had found what I had been looking for all of those years in Wilson, where I was so out of place -- people who cared about the same things that I did. People who were smarter, and weirder, and more interesting than I could ever be. Teachers with an amazing passion and ability to inspire. A cafeteria serving the most amazing food … wait … scratch that … The food was terrible. But at least it was free!
Coming to Science and Math was the first time that I realized that there was a bigger, more incredible, more inspiring world out there than the narrow confines of my experience back in Wilson. I am sure many of you have had the same experiences, over and over again while roaming these halls. Never forget it. It’s the most important thing you can take away from your time here.
This lesson has been reinforced many times in my life. Another time was my first opportunity to travel outside of the developed world. Although I had traveled and lived in Europe during college, it wasn’t until I was a grad student at Berkeley that my wife and I traveled to “difficult” places -- the first such trip being Nepal, where we spent about a month, trekking through the Himalaya. I will never forget the experience of arriving in Kathmandu for the first time, speeding through the night in a taxi, looking out at the eye-popping poverty, the people everywhere, cows and goats and water buffalo roaming in the road, the endless cars and rickshaws and bicycles and motor scooters everywhere -- it might has well have been a different planet. It completely changed my conception of what the world was like, right there, over the course of about ten minutes.
Over the following years, we traveled to many other places as remote as Papua New Guinea, Bolivia, and Laos -- and every time my mind was blown by how strange and big and varied the world is. If I can leave you all with just one thing from today, it would be to encourage you to travel the world. Especially while you are young. Not enough young people do this, and I think it’s the single most effective way to broaden your horizons in life.
I also learned this lesson when I decided to quit my job as a professor at Harvard to join Google. To many people, being a professor, especially somewhere like Harvard, sounds like a dream job. You get tremendous freedom, the opportunity to work with some of the smartest students and faculty anywhere. Mark Zuckerberg, the founder of Facebook, was even one of my students. If you’ve seen the movie “The Social Network”, that scene where Zuck storms out of a Computer Science lecture -- the professor in that scene was supposed to be me -- those were in fact my real PowerPoint slides from class! Of course Mark never stormed out of a lecture -- he wasn’t actually in lecture often enough for that to have happened.
The thing is, being a professor is really, really hard work. You have almost no down time, as there’s always another deadline, another lecture to prepare, another letters to write. It’s grueling. Your teachers here have to work just as hard, so you should really thank them today -- they work their tails off for you.
So after seven years at Harvard, I was ready for a change, and started a sabbatical at Google. And going to Google felt a heck of a lot like what it felt coming to Science and Math for the first time. I mean, nerds everywhere! People wearing socks with sandals! The other day, I went to a meeting, and one of the engineers actually had a live parrot on his shoulder. This is totally normal at Google! And, just like Science and Math, they even have free food! Though you don’t have to work in the cafeteria or rake pine needles. That part’s much better.
I knew almost from the moment I went to Google that that was where I was meant to be -- that I had found my kin. But it took making a big change -- giving up a permanent job, leaving behind my students and colleagues, moving my family across the country -- to figure that out. So it was a big risk, but the reward was huge -- finding out what I really want to be doing with my life, right now. You have to take those risks.
So never stop looking. That’s it. Pretty simple, right? Never stop looking. You never know what you might find.
And with that, I just want to say, congratulations to you all. Today is a big day -- it’s your day. Live it up! Don’t forget it! That is, if you got any sleep last night. If you didn’t, well, you can go back to sleep now because the best part is over.
Thank you very much, and congratulations to the class of 2014!