Thursday, January 22, 2015

Day in the Life of a Google Manager

Not long after joining Google back in 2010, I wrote this cheeky piece contrasting my daily schedule at Google with my previous career as an academic. Looking back on that, it's remarkable how much my schedule has changed in four years, in no small part because I'm now managing a team and as a result end up doing a lot less coding than I used to.

So, now seems like a good time to update that post. It will also help to shed some light on the differences between a pure "individual contributor" role and the more management-focused role that I have now.

By way of context: My role at Google is what we call a "tech lead manager" (or TLM) which means I'm responsible both for the technical leadership of my team as well as the people-management side of things. I've posted more details about the TLM role elsewhere, so I won't repeat that here. Our team has various projects, the largest and most important of which is the Chrome data compression proxy service. We're generally interested in making Chrome work better on mobile devices, especially for users in slow, expensive networks in emerging markets.

The best part of my job is how varied it is and every day is different (and I usually have a lot of balls in the air). The below is meant to represent a "typical" day although take that with a grain of salt given the substantial inter-day variation:
6:45am - Wake up, get the kids up and get them ready and make them breakfast, shower. 
8:30am - Jump on my bike and ride to work (which takes about 10 minutes), grab breakfast and head to my desk. 
8:45am - Check half a dozen dashboards showing various metrics for how our services are doing -- traffic is up, latency and compression are stable, datacenters are happily churning along. 
9:00am - Catch up on email. This is a continuous struggle and a constant drain on my attention, but lately I've been using Inbox which has helped me to stay afloat. Barely. 
9:30am - Work on a slide deck describing a new feature we're developing for Chrome, incorporating comments from one of the PMs. The plan is to share the deck with some other PM and eng leads to get buy-in and then start building the feature later this quarter. 
10:00am - Chat with one of my teammates about a bug report we're chasing down, which gets me thinking about a possible root cause. Spend the next half hour running queries against our logs to confirm my suspicions. Update the bug report with the findings. 
10:30am - I somehow find my morning has not been fully booked with meetings so I have a luxurious hour to do some coding. Try to make some headway on rewriting one of our MapReduce pipelines in Go, with the goal of making it easier to maintain as well as adding some new features. It's close to getting done but by the time my hour is up one of the tests is still failing, so I will spend the rest of the day quietly fuming over it. 
11:30am - Meet with one of my colleagues in Mountain View by video hangout about a new project we are starting up. I am super excited to get this project going. 
12:00pm - Swing by the cafe to grab lunch. I am terrible about eating lunch at my desk while reading sites like Hacker News - some habits die hard. Despite this, I still do not have the faintest clue how Bitcoin works.
12:30pm - Quick sync with a team by VC about an internal summit we're organizing, to plan out the agenda. 
1:00pm - Hiring committee meeting. We review packets for candidates that have completed their interview loops and try to decide whether they should get a job offer. This is sometimes easy, but often very difficult and contentious, especially with candidates who have mixed results on the interview loop (which is almost everyone). I leave the meeting bewildered how I ever got a job here.
2:00pm - Weekly team meeting. This usually takes the form of one or more people presenting to the rest of the team something they have been working on with the goal of getting feedback or just sharing results. At other times we also use the meeting to set our quarterly goals and track how we're doing. Or, we skip it.
3:00pm - One-on-one meeting with a couple of my direct reports. I use these meetings to check in on how each member of the team is doing, make sure I understand their latest status, discuss any technical issues with their work, and also talk about things like career development, setting priorities, and performance reviews. 
4:00pm - Three days a week I leave work early to get in an hour-long bike ride. I usually find that I'm pretty fried by 4pm anyway, and this is a great way to get out and enjoy the beautiful views in Seattle while working up a sweat. 
5:00pm - Get home, shower, cook dinner for my family, do some kind of weird coloring or electronics project with my five-year-old. This is my favorite time of day. 
7:00pm - Get the kids ready for bed and read lots of stories. 
8:00pm - Freedom! I usually spend some time in the evenings catching up on email (especially after having skipped out of work early), but try to avoid doing "real work" at home. Afterwards, depending on my mood, might watch an episode of Top Chef with my wife or read for a while (I am currently working on Murakami's 1Q84).
Compared to my earlier days at Google, I clearly have a lot more meetings now, but I'm also involved in many more projects. Most of the interesting technical work is done by the engineers on my team, and I envy them -- they get to go deep and do some really cool stuff. At the same time I enjoy having my fingers in lots of pies and being able to coordinate across multiple active projects, and chart out new ones. So it's a tradeoff.

Despite the increased responsibilities, my work-life balance still feels much better than as an academic. Apart from time-shifting some of my email handling to the evening (in order to get the bike rides in), I almost never deal with work-related things after hours or on the weekends. I have a lot more time to spend with my family and generally manage to avoid having work creep into family time. The exception is when I'm on pager duty, which is another story entirely -- getting woken up at 3 am to deal with a crash bug in our service is always, um, exciting.


Wednesday, October 15, 2014

You'll be fine in Silicon Valley

Yesterday, a group of well-known professors from a bunch of universities sent an open letter to Microsoft Research's leadership, condemning them for closing the Microsoft Research Silicon Valley lab and leaving the researchers there "jobless". Although I know and respect many of the authors of this letter, I think they get it wrong when it comes to the career opportunities for the Microsoft researchers impacted by the lab's closing, and more broadly for the impact this closing will have on the research community.

The letter's tone is incredibly haughty -- a mixture of "how dare you" and "think of the children!!" What worries me the most is the message the letter seems to be sending to junior researchers who may feel that industry jobs are not a viable career path, because, hey, you might get fired if your lab closes. I think we should correct some of these misconceptions.

(Standard disclaimer: This is my personal blog and the opinions expressed here are mine alone, and most certainly not that of my employer.)

Let's take the letter point by point.

By now, you are no doubt aware of the research community’s shock and disappointment at the sudden and harsh way in which the members of the Microsoft Research Silicon Valley lab were dismissed a few weeks ago.
While the news was sudden, I don't have any evidence that it was "harsh". That's how layoffs sometimes work. Yes, it sucks, but don't imply malice where none is intended.

We are writing to share our perspective on the negative impacts of the shutdown and to open a dialogue about the ways in which Microsoft can try to restore the environment that enabled MSR to produce such great research in the past, benefiting both the company and the world.
The implication here is that by closing the Silicon Valley lab, Microsoft has damaged "the environment" that enabled them to "produce such great research". Let's see here. By my count, about 75 researchers out of a worldwide organization of "more than 1,000 scientists and engineers" were affected. While this is unfortunate, I don't think it's going to seriously hinder Microsoft's ability to do great work in the future. It's still a great place and I expect it will continue to grow.

While layoffs are always unpleasant, the impact of this one has been exacerbated by the fact that many researchers at the Silicon Valley lab worked on long-term, fundamental research of the kind that can be done at very few places outside of academia.
The fact that "very few places outside of academia" allow one to do "long-term, fundamental research" should be teaching us something. The authors of this letter seem to believe that doing academic-style research is some kind of special, protected occupation that people with PhDs are entitled to. While there may have been plenty of industry labs doing pure, academic-style research 20 or 30 years ago, the world has changed. What worries me is that professors still cling to an industry research model that has largely been supplanted by other, better models. I think it's time to evolve our thinking about this in the community and start giving appropriate career advice to students so they are prepared for the reality of what's out there, rather than for an outmoded academic ideal.

It is also a common misconception that one cannot do research in a more product-oriented industry setting. For example, Google does a tremendous amount of research, albeit using a fairly different model than places like MSR. The same is true for other large and small companies as well. We may not call all of these places "research labs" and bestow fancy titles on people like "research scientist", but the research is still happening, and having tremendous impact.

As you know, the academic calendar is such that many of these researchers, including very junior ones who recently completed their PhDs, could be jobless for nearly an entire year. We feel that there should have been a better way to close down this lab, one that would have allowed them to have continuous employment until academic jobs are available again in September 2015. Given that this lab was continuing to produce exceptional — indeed revolutionary — research, we fail to understand why closing it had to be done so suddenly.
This is completely disconnected from reality. First of all, where did anyone get the idea that the only viable job opportunities for the researchers being let go from MSR SV are in academia? From what I can tell, all of these people are being heavily recruited by a number of Bay Area companies. Hell, the MSR SV office is literally off of the same highway exit as the Google headquarters. About the only thing that might change for some of these people is which parking lot they drive to in the morning.

I very seriously doubt that any of these researchers will be "jobless" for any length of time. That's delusional.

Second, as a friend of mine pointed out, it's not as though the "academic calendar" is some kind of law of physics. If universities are serious about recruiting some of these people, I am sure they can find a way to make it happen, even if they have to resort to such lengths as hiring them on short-term contracts as visiting scientists or whatever. The academic calendar is your problem, not Microsoft's.

Over the past two decades, MSR, and indeed all of Microsoft, earned an excellent reputation in academia as an organization that not only valued basic research but also supported the career development of the many researchers that worked in or visited the labs.  That reputation has been significantly damaged, threatening Microsoft’s ability to recruit and retain world-class researchers. 
It's true that shuttering MSR SV will have some impact on how people view Microsoft Research as a career choice. On the other hand, it's not as though they hired a huge number of people every year -- the total number of jobs impacted is relatively small. In comparison, Google hires multiple MSR SVs worth of people every week, and more PhDs per year than all of MSR employs, worldwide. (For that matter, Microsoft, as a whole, no doubt beats Google's numbers.) It's not like the loss of a small fraction of MSR's total headcount has anything but a minor impact on the overall pipeline for computer scientists.

What's really happening here is a bunch of finger-wagging in an effort to publicly shame Microsoft for what was no doubt a very difficult and complex business decision.

As faculty members, we can no longer recommend it as highly to our students as a place to start their careers.  In the long term, this move seems likely to adversely affect Microsoft Research (and the positive contributions it makes to Microsoft as a whole) in more ways than any benefit it may have had.
I think Microsoft should appoint a panel of academics to run the company, since clearly they know more about running a business than MSR leadership does.

Nevertheless, we believe that Microsoft can reduce the damage that has been caused by the shutdown of the Silicon Valley lab.  We understand that Microsoft is considering ways to help care for the researchers who were dismissed, such as defraying the additional costs of the academic organizations who are trying to provide these researchers with temporary homes. This would be an excellent, and highly appreciated, first step.
Of course universities must bear the cost of supporting these poor refugees while they find new jobs! I mean, it's not like the Microsoft stock and severance package will pay the bills for very long until the affected people get jobs at Yahoo or Twitter. Hopefully the universities can pull through with basic rations, perhaps a tattered wool blanket and a cot to sleep on, an office with an aging workstation running XENIX, just until these people can find gainful employment. It would be a real humanitarian disaster otherwise.

Looking forward, we hope that you will open a discussion with us and the community about Microsoft’s vision for industrial research (which has become less clear after the closing of what appeared to be an extremely valuable and successful lab) and concrete commitments MSR can make regarding the career development of its remaining and future researchers. 
Frankly, I don't think that the MSR leadership owes the academic community any kind of explanation or commitments whatsoever. If anything, this was a great wake-up call to anyone who was living under the false impression that being a "researcher" at a company gave you job security. This is how companies sometimes work.

Fortunately, especially in Silicon Valley, there are literally hundreds of great companies that the MSR SV folks can work for, doing all kinds of awesome and groundbreaking work. The tech industry as a whole is going strong, and I have no doubt that there will be all kinds of career opportunities for these folks.

Steps like these are essential to rebuilding the relationship between Microsoft and the academic community, along with all the mutual benefits that it brings.
On the whole, my guess is that the relationship between Microsoft and the academic community will continue to thrive. There are a lot of great people still at MSR and the ties there are incredibly strong. It's too bad that they had to shut down the Silicon Valley lab in the way that they did, but I'm not sure it's productive to cast a dark cloud over everything MSR does and stands for as a result.

Mostly, I'd like the authors of this letter to think about the message they are sending to students and junior researchers, which seems off base to me. We're all sad about the loss of MSR SV, but I think this letter really goes too far when it claims that anybody there will be "jobless" or that academic positions must open up to absorb the affected researchers.

Monday, August 4, 2014

The Fame Trap

The other day I was speaking with a group of interns about the differences between the academic and industrial career paths. One of them mentioned that when you join a company like Google, you lose your identity -- that is, people outside the company may not know much about what you do or work on. I like to think of it like getting assimilated into the Borg. This is a concern I hear a lot from grad students.

First off, this is absolutely true. Unless you work really hard at it, being in industry (not counting industrial research labs, of course) does not afford you as many opportunities to stay visible in the research community. No big surprise here -- at a company like Google, your job isn't to publish papers, it's to build products. You can publish papers, serve on program committees, and go to conferences -- but when academic research not your main job, doing those things isn't necessarily a priority.

I think many grad students get fixated on this idea of cultivating their academic profile and tend to make career decisions that build on that, to the exclusion of some other ways in which they could have a mark on the world. (This absolutely happened to me.) As an academic, your entire career is focused, to some degree, on building and maintaining your personal brand. In my experience, though, the "fame" you enjoy as an academic is on a fairly small scale, and it takes a tremendous amount of energy to perpetuate. That is, there are many reasons for wanting to pursue an academic career, but I would argue that achieving prestige in the research community isn't a particularly good one.

Let's keep things in perspective -- renown in the academic world is on a tiny scale. Take the systems community: a typical systems conference will have no more than about 400 attendees. Let's assume that roughly 10% of the people in the community (that matter!) are actually going to the major conferences -- so we're talking (generously) on the order of 10,000 people or so who might be active in the systems community at any given time.

I'm sorry, but being "famous" in a tight-knit community of O(10K) academics ain't really being famous. That's nowhere near the level of Kanye, or even Neil deGrasse Tyson. In fact, I can't name a single academic computer scientist who enjoys that level of fame. Maybe Alan Turing, but his fame is largely posthumous. There are the rare academics who are well-known across CS -- David Patterson comes to mind -- but even he's not exactly a household name. (Unless your household involves a lot of dinnertime conversations about disk arrays and processor architecture.)

As a grad student, it's pretty easy to fall into the fame trap, and I speak from experience. When I finished my PhD, I was pretty much dead set on having an academic career, because that seemed to be the most glamorous, high-profile career path at the time. (Keep in mind that when I finished my PhD in 2002, I had an offer from pre-IPO Google, and could have been something like employee #400. Now I'm just employee #40,000.) I was enamored with the prestige of being a professor. I dreamt of running my own lab, having my own students, being as "famous" as my advisor. And of course, from having been in grad school for so long, building up my academic profile was pretty much the only world I knew.

What I didn't realize at the time was how much work it would be to maintain an academic reputation. To be visible, and stay visible, in the research community means going to countless conferences, serving on program committees and panels, visiting other universities to give talks, and of course publishing as many papers as you possibly can. Getting tenure is driven in large part by how well known you are and how much "impact" you're having. If you fail to do these things, your stature in the community slips pretty quickly, unless you are really well-established and can afford to just show your face from time to time. (Junior faculty need not apply.)

I see this all the time in my academic colleagues. They travel constantly -- one or two trips per month is typical. They say yes to a ridiculous number of committees, advisory boards, whatever. They like to "complain" about the amount of travel and committee work they do, but I'm sure they all realize that they can't really stop doing it, lest they stop getting invited to do these things. (Kind of a strange vicious cycle, that.) It's like being in a small-time folk band that has to tour constantly to keep paying the bills -- you never get a break.

I applaud my academic colleagues who can still do this. For me, personally, I never really enjoyed it, and I found it wasn't worth the sacrifice just to maintain the status quo of my academic reputation. Once I had kids, I really started to appreciate the toll it was having on my family to be out of town so often, and I started to realize that maybe I had my priorities all wrong.

These days I still enjoy being part of the academic community, but in a more selective way. I like to go to occasional conferences and serve on PCs that are very closely aligned with my area, because I learn a lot about what's going on in the research world. It's also a good way to recruit interns and full-time hires, and of course I have lots of academic friends I like to meet up with and have beers with. I'll admit I still like giving talks from time to time. I've done very little publishing since leaving academia, but that's by choice -- I'm spending my time on things that are more important (to me) than writing more papers.

None of this is to say that academia can't be a wonderful career path, but I think chasing academic fame is not the best reason to go down that path. I wish I had known that when I was finishing my PhD.

Saturday, May 31, 2014

The NCSSM 2014 commencement address

This morning I had the distinct pleasure of delivering the commencement address at the North Carolina School of Science and Math, my alma mater, where I graduated in 1992. NCSSM is a public residential high school in which students attend their junior and senior years, and of course as the name suggests has an emphasis on science and math curriculum (but it is so much more than that). For some reason, the students and administrators asked me to give the commencement address this year, which is one of the strangest things I've ever had to do -- in part because it felt like yesterday that I graduated from there (yet it was 22 years ago)!

NCSSM is dear to my heart and those two years were a transformative time in my life. I've blogged before about how my time at S&M saved my life -- it is difficult to overstate how important the place was to my formative years. (Actually I think I'm still "forming".)

I wore Google Glass to give the speech, thinking this would be a really cool and unique way to show off my "Google-ness", but actually one of the graduating seniors was wearing Glass as well:


So much for being unique.

The school live-streamed the event and I was surprised to find a video of my speech online soon after it was over. Watch here at your own peril.

Below I'm posting my prepared speech (which is not exactly the same as the one I actually gave).

Students, professors, parents, mom, (hi mom!), it is most humbling to stand before you here today. Indeed, I am terrified. Somehow I think I am being punished for all of the misdeeds I was involved with back during my days as a student here so many years ago. Either that, or the fact that I am here must be proof that there is no such thing as a permanent record -- if they had one on me, well, let’s just say they would have thought twice about inviting me back to give a commencement address. So, those of you who stayed out way too late last night -- there is hope for you yet! 
Before we begin, I want to get one thing out of the way. Some of you may be wondering what this thing is that I’m wearing. This is called Google Glass, and basically, it’s a cell phone that you strap to your head. I am told that in some neighborhoods of San Francisco it either makes you utterly irresistible to the opposite sex, or it gets you beat up. Google Glass has all kinds of amazing features, and this little screen up here is feeding me a constant stream of information about each and every one of you … yes, you in the back there… the one who made a B-minus in history last term… Google Glass is telling me that those tennis shoes you ordered off of Amazon last week will be delivered this afternoon. You’re welcome.
It’s really remarkable to be back here, twenty-two years after I graduated from NCSSM, on this very lawn. Of course, I don’t remember much about the graduation ceremony itself, seeing as how I got absolutely no sleep the night before. You see, I stayed up too late with my friends and staggered back to my room at 5 o’clock in the morning and realized I had not started packing my room. At all. And my parents were arriving in just a few short hours, expecting me to be rested, cheery, but most of all packed and ready to go. So I frantically threw everything I owned into boxes, tore all of the Depeche Mode posters off of the walls, packed away my IBM PC and boxes full of diskettes, and tried to make some order out of chaos. I was only partially successful, but my parents understood, and we loaded everything into the minivan, and away we went. Sorry, mom! You guys were great about it though.
Poof. Just like that, my NCSSM experience was over. They really were two of the best years of my life, in so many ways -- and the experiences I had here utterly changed the course of my life. I am sure most of you feel the same way already, but the ripple effect will be felt for years to come. Look around -- look at your friends, your teachers, your RAs. This place has made a deep and enduring mark on who you are and who you will be.
NCSSM taught me the single most important lesson I’ve ever learned in my life, and I want to share it with you.
It’s very simple: Never stop looking. That is, never stop looking for who you are, where you belong, and what you might do with your short time on this planet. Sometimes it takes a really, really big change to figure that out.
Before coming to S&M -- as we called it affectionately in those days -- I grew up in a small town called Wilson, about an hour east of here. I cannot say that I really loved Wilson, and I didn’t fit in well there at all. For one thing, I didn’t have a car, so I could not join the legions of kids who would spend their Friday nights driving in circles around the mall, which was pretty much the only entertainment there. But more than that, with the exception of a few close friends, I never quite felt like I found kindred spirits -- people with the same weird and geeky interests that I had. People who saw the world in the same way.
Fortunately, I was admitted to Science and Math, and I am dead serious when I say that coming here saved my life. Almost as soon as I arrived here, I realized that I had found what I had been looking for all of those years in Wilson, where I was so out of place -- people who cared about the same things that I did. People who were smarter, and weirder, and more interesting than I could ever be. Teachers with an amazing passion and ability to inspire. A cafeteria serving the most amazing food … wait … scratch that … The food was terrible. But at least it was free!
Coming to Science and Math was the first time that I realized that there was a bigger, more incredible, more inspiring world out there than the narrow confines of my experience back in Wilson. I am sure many of you have had the same experiences, over and over again while roaming these halls. Never forget it. It’s the most important thing you can take away from your time here.
This lesson has been reinforced many times in my life. Another time was my first opportunity to travel outside of the developed world. Although I had traveled and lived in Europe during college, it wasn’t until I was a grad student at Berkeley that my wife and I traveled to “difficult” places -- the first such trip being Nepal, where we spent about a month, trekking through the Himalaya. I will never forget the experience of arriving in Kathmandu for the first time, speeding through the night in a taxi, looking out at the eye-popping poverty, the people everywhere, cows and goats and water buffalo roaming in the road, the endless cars and rickshaws and bicycles and motor scooters everywhere -- it might has well have been a different planet. It completely changed my conception of what the world was like, right there, over the course of about ten minutes.
Over the following years, we traveled to many other places as remote as Papua New Guinea, Bolivia, and Laos -- and every time my mind was blown by how strange and big and varied the world is. If I can leave you all with just one thing from today, it would be to encourage you to travel the world. Especially while you are young. Not enough young people do this, and I think it’s the single most effective way to broaden your horizons in life.
I also learned this lesson when I decided to quit my job as a professor at Harvard to join Google. To many people, being a professor, especially somewhere like Harvard, sounds like a dream job. You get tremendous freedom, the opportunity to work with some of the smartest students and faculty anywhere. Mark Zuckerberg, the founder of Facebook, was even one of my students. If you’ve seen the movie “The Social Network”, that scene where Zuck storms out of a Computer Science lecture -- the professor in that scene was supposed to be me -- those were in fact my real PowerPoint slides from class! Of course Mark never stormed out of a lecture -- he wasn’t actually in lecture often enough for that to have happened.
The thing is, being a professor is really, really hard work. You have almost no down time, as there’s always another deadline, another lecture to prepare, another letters to write. It’s grueling. Your teachers here have to work just as hard, so you should really thank them today -- they work their tails off for you.
So after seven years at Harvard, I was ready for a change, and started a sabbatical at Google. And going to Google felt a heck of a lot like what it felt coming to Science and Math for the first time. I mean, nerds everywhere! People wearing socks with sandals! The other day, I went to a meeting, and one of the engineers actually had a live parrot on his shoulder. This is totally normal at Google! And, just like Science and Math, they even have free food! Though you don’t have to work in the cafeteria or rake pine needles. That part’s much better.
I knew almost from the moment I went to Google that that was where I was meant to be -- that I had found my kin. But it took making a big change -- giving up a permanent job, leaving behind my students and colleagues, moving my family across the country -- to figure that out. So it was a big risk, but the reward was huge -- finding out what I really want to be doing with my life, right now. You have to take those risks.
So never stop looking. That’s it. Pretty simple, right? Never stop looking. You never know what you might find.
And with that, I just want to say, congratulations to you all. Today is a big day -- it’s your day. Live it up! Don’t forget it! That is, if you got any sleep last night. If you didn’t, well, you can go back to sleep now because the best part is over.
Thank you very much, and congratulations to the class of 2014!




Wednesday, May 7, 2014

The Google career path, Part 2: Starting new projects

In my last blog post, I talked about what it's like to get started at Google as a new engineer. In this post, I'll talk about how new projects get started.

Standard disclaimer applies: This is my personal blog and nothing I'm saying here is endorsed or approved by Google. This is only based on my personal experience. Take it with a grain of salt.

Google is well known as having a bottom-up engineering culture. That is, new projects typically arise because a group of engineers get together, decide a problem needs to be solved, devise a solution, and convince others to (a) join in the effort and (b) launch it. It is rare -- although not unheard of -- for a Google project to start with a high-level mandate from some executive. (I have heard that this is how Google+ got its start, but I wasn't there at the time, so I don't really know.) More typically, a grassroots effort emerges out of engineers trying to build something that they think is important and will change the world. Or at least scratch an itch.

How my project got its start

My team is responsible for a range of projects to optimize and streamline the mobile web. The main thing we are known for is the Chrome Mobile data compression proxy, which provides users of Chrome on Android and iOS a switch they can flip in the browser to compress web pages by 50%. I thought it might be instructive to talk about how this project came about.

When I started the team in 2011, I was given a very broad mandate: Go fix the mobile web. Initially, we spent a bunch of time just trying to understand the problem. At the time, there were no good tools for even measuring mobile web page performance. Now, we have things like Webpagetest for mobile, but at the time you couldn't get decent measurements without a lot of work. So, we built a testbed for collecting mobile web measurements and bought a bunch of phones and tablets and started collecting data. We spent a huge amount of time analyzing the data and playing around with various technologies, like PageSpeed, to see what kind of gains could be had.

At one point we realized that the only way to move the needle with mobile web performance was to build our own proxy service which could be integrated into a browser. Around the same time, the Chrome Mobile project was ramping up, and we realized that the technology we were building would be a great addition to Chrome when it became available for Android and iOS. So, we went to the Chrome team (which we were not yet a part of) and pitched the idea -- would it make sense to integrate this hypothetical proxy into the browser, provide a setting for users to enable it? At the time, we didn't even have the proxy yet -- just a slide deck and a crude, proof-of-concept demo. But we had a lot of ideas of what it could do and how it would work.

Fortunately, the Chrome team loved the idea and even offered to take our team on as part of the Chrome project. Of course, they hadn't agreed to launch anything yet -- that approval wouldn't come until many months later -- but we had their blessing to develop the idea further and get it to a state where it could be dogfooded within Google. So we built it. We got it to dogfood. We collected data from real users (well, Googlers anyway). The data looked good, so we got approval to launch it externally, first behind a flag, then in the Chrome beta channel, and finally launch it to everyone. Boom!

So the idea for the proxy service and the project as a whole came entirely from within our team. Nobody told us to do this. And it was our job to "sell" it internally, in terms of turning the idea into a fully-baked feature that could launch with Chrome. In a lot of ways, it's like being a startup company (although one that does not have to worry about funding and already has fancy cafeterias and a gym).

Does this generalize?

I can't say for certain that my experience is the standard way of getting new projects off the ground at Google, but it seems to be.

Projects often seem to get their start with one or two engineers who have an itch to scratch, and come up with something compelling that they want to build. They might start the project on their own, in their 20% time (on which see more below), or they might spawn an effort out of their existing work.

A typical example might be where an engineer realizes that a new system needs to be built to solve a problem that they keep running into all the time (measuring mobile web pages, say). So with the blessing of their manager (or not, in some cases), they carve out time to get that project off the ground. If it's successful, it might grow, and more engineers will start to get involved.

New projects often die, too. Sometimes a project will start out with a couple of people, they'll spend a quarter or two trying to get it off the ground, and the effort fizzles out for any number of reasons. One example is that it turns out to be much harder than expected. Another is that other priorities come up and they don't have the time to run as far with the idea as they had hoped. Another is that some other project comes along which is a broader, more general form of what they were trying to do, so they decide to join forces (or just shut down the original effort).

What about 20% time?

Much has been written about Google's "20% time", whereby engineers are allowed to work on projects unrelated to their main job. (You can't do just anything for 20% projects -- taking guitar lessons wouldn't be allowed, for example -- but the definition is pretty broad.) 20% projects are a great way to seed a new effort, since (a) you don't really need any formal approval to work on them, and (b) it gives engineers a chance to do things that they would not otherwise find the time to do.

Gmail famously started as a 20% project. (The original press release was ironically posted on April 1, 2004, but it was not actually an April Fools' joke!)

There have also been many unfounded rumors of 20% time's demise. From where I sit, 20% time is alive and well, and several engineers on my team have active 20% projects. For example, one of them contributes actively to our internal platform for employees making charitable contributions, called "G-Give". Another contributes to the smart contact lens project. My own (unofficial) 20% project is doing outreach activities to the Computer Science research community -- serving on program committees, reviewing research proposals, that kind of thing. (It actually takes a lot less than 20% of my time, but don't tell my boss that.)

That said, 20% time is not used uniformly across the company and some managers are probably allergic to it. I don't think all engineers should do 20% projects, since often people can have more impact if they spend 100% of their time on their "day job" -- the 20% would be a distraction and not necessarily help them get promoted.

But promotions are the subject of my next blog post, so I'll leave it at that.

Thursday, May 1, 2014

The Google career path, part 1: Getting started

Apart from how to get a job at Google (which I have previously blogged about), the most common question I get from people considering taking a job here is what the career path is like -- that is, how you get started, how you grow your career, how promotions work, and so forth. I thought I'd try to capture some of my perspective in a series of blog posts on the topic.

(But first, the obligatory disclaimer: This is my personal blog. Google doesn't condone or support anything I'm saying here. Heck, it's probably all wrong anyway. Take it with a grain of salt.)

This stuff is mostly targeted at new grad hires (at any level -- Bachelors, Masters, or PhD) for software engineering positions. (I don't know how things work in other parts of the company, like sales.) This may apply to a lesser extent to more senior engineers or researchers as well.

Before you join: The team match

Before your start date, you will be assigned to one of the many hundreds of teams at Google. While there are exceptions, teams typically consist of O(10) people who all report to the same manager and have a common project focus. Examples might include the Android Hangouts app, feeding real-time data such as sports scores into Search, or (my team) making mobile Web browsing faster and more efficient.

A lot of people want to work on teams doing things they have already heard of -- such as Search, Chrome, or Android. But keep in mind that the vast majority of teams at Google work on things that are (a) not user-facing and therefore (b) not something you would necessarily know anything about before coming here. My first team at Google focused our content delivery network for YouTube, which I never even knew existed until I started the job. Many project teams are building internal infrastructure and tools, and often the work they do is highly confidential, so we can't (unfortunately) always tell new hires very much about the details of what the team works on until they join.

Google tries to assign you to a team that matches your interests. If you are a Bachelors or Masters new grad hire, the team match is typically done in a "late binding" fashion, a few weeks before the start date (so, after you have accepted the job offer). Given how rapidly projects and hiring priorities shift among teams, it makes sense to do the binding as late as possible. For PhD and more senior candidates, because your skills are more specialized, the recruiters try to make sure that the candidate has a team interested in hiring you even before bringing you in for an interview. As a result, you essentially have a spot "reserved" on that team well before you would begin the job. In those cases you might be able to learn a bit more about what your target team is working on during the process of interviewing and negotiating the offer.

The thing I always emphasize is that when you are hired by Google, you are getting a job at Google, not a specific team at Google. In other words, the initial team match is by no means final, and nor is it locking you down. Engineers move between teams all the time -- it is not uncommon for people to look for a new team every couple of years. Some people prefer the stability of staying in the same area year after year, while others prefer to explore different parts of the company. New projects and teams are always spinning up, and (indeed) teams often reorganize or disband. So there is a fair bit of churn, and I would not stress the initial team assignment much -- it's a starting point.

The starter project

My advice to people starting at Google is to roll up your sleeves, dive in, and start contributing however you can. It's a super steep learning curve, and for the first month or two you won't have the foggiest idea what anybody is talking about, but the more you get involved the faster you'll come up to speed.

The first week on the job is typically spent in orientation, which is usually done at the headquarters in Mountain View. I blogged about my first week at Google a while back -- it is pretty intense, and there is a huge amount to learn. I found it to be super fun (though exhausting) and have great memories of that week. You get your badge, your laptop, your first five or six items of Google schwag, try to figure out where to get lunch, how the videoconference system works, and how to get started on programming in our insanely complex environment. There are a bunch of hour-long courses on topics ranging from how Google Search works to how to use our source code control system.

After the first week you'll start working with your new team. Your starter project is intended to show you the ropes and get you up to speed on our coding environment, build tools, code review process, and so forth, so you can start to be a fully productive member of the team. Starter projects might be something like adding a small feature to your team's code or fixing a straightforward bug. A starter project might take anywhere from a week to a month.

Taking ownership

As you come up to speed, you'll proceed to taking on projects with larger complexity and scope, as you become more familiar with the codebase and conversant in the problems that are being worked on by your team. It's really up to you to drive this process and own your work.

To give a concrete example, one of the (relatively) recent PhD hires on my team, Michael Buettner, started out by investigating the size and quality tradeoff for the image transcoding parameters we were using in the Chrome Mobile data compression proxy. Over time he expanded his expertise of our software and started taking on harder, bigger problems -- including sophisticated optimizations to the proxy's HTTP fetching backend. He is now our team's "latency expert" and has devoted a great deal of his time to making the whole system faster. Since he knows so much of our codebase, he advises on a broad range of new features and bugfixes, and works with other teams to implement functionality to streamline our system's performance.

I want to emphasize that this is not something Michael was specifically tasked with doing. Like most teams, ours has way more problems than people, so engineers tend to gravitate towards the problems that match their interests. My job as the team lead is to coordinate and prioritize our team's work, and fortunately we rarely have cases where we have something really important to do that's not up someone's alley.

In the next couple of blog posts I'll talk about the evaluation and promotion process at Google, as well as how new projects are started.

Friday, February 28, 2014

Taking the "Hot" out of "Hot Topics" workshops

I just got back from HotMobile 2014 (for which I was the general chair). HotMobile is the mobile systems community's "hot topics" workshop, held annually as a forum for (according to the Call for Papers) "position papers containing highly original ideas" and which "propose new directions of research" or "advocate non-traditional approaches". It's a small workshop (we had about 95 people this year) and the paper submissions are short -- 6 pages, rather than the regular 14.

The HotMobile'14 poster and demo session.
Look how happy those mobile systems researchers are!
Overall, the workshop was great -- lots of good discussions, good talks, interesting ideas. And yet, every time I attend one of these "hot topics" workshops, I end up feeling that the papers fall well short of this lofty goal. This is not limited to the mobile community -- the HotOS community has a similar problem as well.

This has bugged me for a long time, since it often feels as though there is no venue for doing "out of the box" work that is intended to look out five or ten years -- rather than just things that are incremental but not yet ready for publication in a major conference like SOSP or MobiSys. I also have fond memories of HotOS in the late 1990s in which it felt as though many of the papers were there to shake up the status quo and put forward a strong position.

What I've now come to realize is that there is a tremendous value in having a small workshop for preliminary (and often incremental) results. The community obviously feels that such a venue is useful, despite its lack of "hotness" -- we had a record number of attendees this year, and (I believe) a near-record number of submissions.

And after all, the main reasons to attend any workshop are the discussions and networking -- not the papers.

The problem is that we insist on calling this a "hot topics" workshop and pretend that it's about far-out ideas that could not be published elsewhere. Instead, I think we should be honest that HotMobile (and HotOS, HotNets, etc.) are really for three kinds of papers:
  1. Preliminary work on a new project which is not yet ready for a major conference. Getting early feedback on a new project is often very useful to researchers, so they know if they are barking up the right trees.

    An example of this from this year is the CMU paper on QuiltView, which proposes allowing users to pose real-time queries ("How is the weather down at the beach in Santa Barbara?") and get back real-time video snippets (from users wearing Google Glass!) in reply. This work is no where near mature enough for a full conference, and I hope the authors gained something from the paper reviews and discussion at the workshop to shape their future direction.
  2. An incremental, and possibly vestigial, step, towards the next major conference paper on a topic. Many such papers are simply not big enough ideas for a full conference paper, but make a nice "short paper" for the sake of getting some idea out there.

    One example from this year is this paper on the dangers of public IPs for LTE devices. This isn't something that's going to turn into a longer, more pithy paper later on, but is probably worth reporting.
  3. The odd wacky paper that falls under the "hot topics" rubric. These are increasingly rare. About the only example from this year is this Duke paper on adding smart capabilities to childrens' toys with smartphones -- but the idea is not that radical.
Last year at SOSP, there was a one-day workshop called TRIOS ("Timely Results in Operating Systems") which was an informal venue for preliminary work -- exactly to provide an outlet for papers in the first two categories above. At least TRIOS was honest about its intent, so nobody attending could be disappointed that the papers weren't "hot" enough.

So, my humble proposal is to rename the workshop "ColdMobile" and, just to be cheeky, hold it at a ski resort in the winter.