Except that Gabe Newell has helped in the creation of one of the best gaming companies in the industry.
Valve: How I Got Here, What It’s Like, and What I’m Doing
April 13, 2012
It all started with Snow Crash.
If I hadn’t read it and fallen in love with the idea of the Metaverse, if it hadn’t made me realize how close networked 3D was to being a reality, if I hadn’t thought I can do that, and more importantly I want to do that, I’d never have embarked on the path that eventually wound up at Valve.
By 1994, I had been working at Microsoft for a couple of years. One evening that year, while my daughter was looking at books in the Little Professor bookstore on the Sammamish Plateau, I happened to notice Snow Crash on a shelf. I picked it up and started reading, decided to buy it, and wound up devouring it overnight. I also started thinking to myself that I had a pretty good idea how about 80 percent of it could work right then, and wanted to implement it as badly as I had ever wanted to do anything with a computer – I had read SF all my life, and this was a full-on chance to make SF real. So I tried to start a project at Microsoft to do a networked 3D engine.
About the same time that it became clear I wasn’t going to be allowed to start that project, John Carmack, fresh from writing Doom at Id Software, came to Seattle to visit his mother, and we went to dinner at Thai Chef. I had met John in person just once before, but he knew me from PC graphics articles of mine he had read when he was first learning to program the PC, and we had exchanged posts on the M&T BBS a few years earlier. During our one previous meeting, he had surprised me by asking if I wanted to come work at Id; I had said no, because I was in the middle of getting Windows NT shipped, and because Microsoft had been generous with stock options. I knew he was going to ask me again on this visit, and I knew I’d say no again, because I was even more entrenched at Microsoft, and because now I had even more stock options. But John didn’t say anything of the sort for the first two hours. He talked about persistent Internet game servers, about people building their own levels and running them on their own servers, and about how it would be possible to connect them together so players could go from one to another, with the virtual world accreting over time. I realized that what he was really talking about was a Metaverse – less glamorous than Neal’s creation, but, amazingly, gloriously, unbelievably real. And when he finally did get around to asking if I wanted to come work at Id, I realized that I couldn’t pass up the chance to be a part of making that future happen.
Working with John was like the sequence in “The Matrix” where Neo has one martial art after another pumped into his brain. I’d stagger out into the parking lot of Id’s Black Cube in Mesquite each evening stunned from a day of trying to keep up with John as we figured out an entirely new programming world – 3D, Internet client-server multiplayer gaming, mods, scripting, everything since we were the only two programmers – and somehow manage the half-hour drive back to Plano. It all happened at lightning speed, with no time to sit back and digest – Quake shipped only 16 months after I arrived. And it was all worth it – not only did I grow tremendously as a programmer, but Quake turned out to be a seminal game; granted, not one of the best games ever, but truly groundbreaking technically (for which, to be clear, John was the brilliant innovator and driving force), and a game that gave rise to a genre and a community that continue strong to this day. As one example, when I started at Valve I found that dozens of people there had started their careers by modding Quake or working on games based on the Quake engine – and in a way I also created my own job 15 years in the future.
In 1996, Id was visited by Mike Harrington and Gabe Newell, who were leaving Microsoft to start Valve, and wanted to license the Quake source code to build their first game on. No one else at Id was particularly interested in licensing them the source code, or even talking to them, for that matter, but I knew Mike and Gabe from my time at Microsoft (in fact, Mike went far out of his way to help me when I started contracting with Microsoft in 1992 – I doubt I would have survived at Microsoft otherwise – so in a sense that’s the start of this story), so I stepped in and facilitated getting the license done. That turned out to be a good thing for almost everyone involved– Id made a lot of money from the license, and Valve built Half-Life from that code – but I didn’t benefit personally (except in the very long term), because I decided to leave Id shortly thereafter. My plan was to return to Microsoft, but Mike and Gabe asked me if I wanted to be the third founder of Valve. After a lot of thought, I decided that I had had enough of small game companies for a while, and joined the Natural Language Group at Microsoft, where I had a great year or two learning about natural language before figuring out that it wasn’t a problem likely to be solved within my lifetime.
Going back to Microsoft was arguably not the best decision I ever made, but neither was it final. Valve has a long-term view; over the years, many people at Valve stayed in touch with me, and periodically one or another of them would ask whether I was ready to join up. Fourteen years later (did I mention Valve has a long-term view?) my work on Intel’s Larrabee project wrapped up. I knew that Valve made cool games and was very successful, and knew a lot of people there that I liked and respected, and that was enough to make it worth a try. So I went to Valve knowing almost nothing about the company’s culture or organization, expecting something more or less like the other places I’ve worked.
I was in for a surprise.
Valve is different
I’ve worked at a lot of companies and in a lot of situations over the last 30 years. I’ve been a programming consultant and magazine and book author. I’ve worked alone and with a partner and at various companies on games. I’ve worked on operating systems, drivers, firmware, natural language parsing, consoles, and processor design. I’ve consulted and worked at a series of startups and small companies, both hardware and software, I’ve worked at Microsoft, I’ve worked at Id, and I’ve worked at RAD Game Tools. They’ve all been interesting, they’ve all been great learning experiences, and some of them have been truly remarkable places. In short, I’ve seen a lot of what tech has to offer.
Valve is different.
Gabe tells it this way. When he was at Microsoft in the early 90’s, he commissioned a survey of what was actually installed on users’ PCs. The second most widely installed software was Windows.
Number one was Id’s Doom.
The idea that a 10-person company of 20-somethings in Mesquite, Texas, could get its software on more computers than the largest software company in the world told him that something fundamental had changed about the nature of productivity. When he looked into the history of the organization, he found that hierarchical management had been invented for military purposes, where it was perfectly suited to getting 1,000 men to march over a hill to get shot at. When the Industrial Revolution came along, hierarchical management was again a good fit, since the objective was to treat each person as a component, doing exactly the same thing over and over.
The success of Doom made it obvious that this was no longer the case. There was now little value in doing the same thing even twice; almost all the value was in performing a valuable creative act for the first time. Once Doom had been released, any of thousands of programmers and artists could create something similar (and many did), but none of those had anywhere near the same impact. Similarly, if you’re a programmer, you’re probably perfectly capable of writing Facebook or the Google search engine or Twitter or a browser, and you certainly could churn out Tetris or Angry Birds or Words with Friends or Farmville or any of hundreds of enormously successful programs. There’s little value in doing so, though, and that’s the point – in the Internet age, software has close to zero cost of replication and massive network effects, so there’s a positive feedback spiral that means that the first mover dominates.
If most of the value is now in the initial creative act, there’s little benefit to traditional hierarchical organization that’s designed to deliver the same thing over and over, making only incremental changes over time. What matters is being first and bootstrapping your product into a positive feedback spiral with a constant stream of creative innovation. Hierarchical management doesn’t help with that, because it bottlenecks innovation through the people at the top of the hierarchy, and there’s no reason to expect that those people would be particularly creative about coming up with new products that are dramatically different from existing ones – quite the opposite, in fact. So Valve was designed as a company that would attract the sort of people capable of taking the initial creative step, leave them free to do creative work, and make them want to stay. Consequently, Valve has no formal management or hierarchy at all.
Now, I can tell you that, deep down, you don’t really believe that last sentence. I certainly didn’t when I first heard it. How could a 300-person company not have any formal management? My observation is that it takes new hires about six months before they fully accept that no one is going to tell them what to do, that no manager is going to give them a review, that there is no such thing as a promotion or a job title or even a fixed role (although there are generous raises and bonuses based on value to the company, as assessed by peers). That it is their responsibility, and theirs alone, to allocate the most valuable resource in the company – their time – by figuring out what it is that they can do that is most valuable for the company, and then to go do it. That if they decide that they should be doing something different, there’s no manager to convince to let them go; they just move their desk to the new group (the desks are on wheels, with computers attached) and start in on the new thing. (Obviously they should choose a good point at which to do this, and coordinate with both groups, but that’s common sense, not a rule, and isn’t enforced in any way.) That everyone on a project team is an individual contributor, doing coding, artwork, level design, music, and so on, including the leads; there is no such thing as a pure management or architect or designer role. That any part of the company can change direction instantly at any time, because there are no managers to cling to their people and their territory, no reorgs to plan, no budgets to work around. That there are things that Gabe badly wants the company to do that aren’t happening, because no one has signed up to do them.
Hardest of all to believe is the level of trust. Trust is pervasive. All of Valve’s source code is available to anyone in Perforce, and anyone at Valve can sync up and modify anything. Anyone can just up and work on whatever they think is worth doing; Steam Workshop is a recent instance of someone doing exactly that. Any employee can know almost anything about how the company works and what it’s doing; the company is transparent to its employees. Unlike many organizations, Valve doesn’t build organizational barriers to its employees by default; it just trusts them and gets out of their way so they can create value.
To be clear, Valve hasn’t magically repealed the realities of developing and shipping products. We’re all human, so teams sometimes argue (and sometimes passionately) about what to do and how to do it, but people are respectful of each other, and eventually get to a consensus that works. There are stresses and more rigid processes when products are close to shipping, especially when there are hard deadlines for console certification (although shipping for the PC is much more flexible, thanks to Steam). Sometimes people or teams wander down paths that are clearly not working, and then it’s up to their peers to point that out and get them back on track.
Also, don’t think that people randomly come in every day and do whatever they feel like doing. It certainly wouldn’t be okay if a programmer decided to move to an empty room and start weaving straw hats (although if they wanted to write a tool to let people weave and sell virtual straw hats, that would be fine). People commit to projects, and projects are self-organizing; there are leads, but they’re chosen by informal consensus, there’s no prestige or money attached to the label, and it’s only temporary – a lead is likely to be an individual contributor on their next project. Leads have no authority other than that everyone agrees it will help the project to have them doing coordination. Each project decides for itself about testing, check-in rules, how often to meet (not very), and what the goal is and when and how to get there. And each project is different.
It’s hard to believe it works, but it does. I think of it as being a lot like evolution – messy, with lots of inefficiencies that normal companies don’t have – but producing remarkable results, things that would never have seen the light of day under normal hierarchical management. The games speak for themselves – and then there’s Steam (and no, Steam doesn’t have formal management either). Valve’s long string of successes, many of them genuinely groundbreaking, is strong evidence that the hypothesis that creative people are the key to success is in fact correct, and that the structuring of Valve around those people has been successful.
And, almost by definition, it’s a great place for the right sort of creative people to work.
What I’m working on
So what has my personal experience been?
As I said earlier, I knew little about how Valve worked when I started here, and my introduction to the company was not at all what I thought it would be. What I expected was that I’d be handed a substantial chunk of technical work to do – something like visibility determination in the Source engine, or fog of war calculation in DotA 2 (which I did in fact work on, but as a sideline – that was a fun bit of optimization that I’ll write about one of these days).
What I got instead was a few suggestions about areas people thought I might find it interesting to look at, and no direct guidance at all. The closest anyone came to giving me direction was when most of the Source engine team was working on Portal 2 optimization; I’ve done a lot of optimization, so I suggested to Jay Stelly that maybe I should work on Portal 2 as well. Jay said, “Yeah, you could do that, but we’ll get it shipped anyway.” After a couple of discussions like that, I realized that he was saying was that I should think about whether that was really the most valuable thing I could be doing – there were plenty of people who were skilled at optimizing the Source engine already working on Portal 2, so it would be more useful to think about what high-impact things I could do that no one else was doing. That, and conversations with various people around the company, kicked me into a different mode of thought, which eventually led me to a surprising place: wearable computing.
By “wearable computing” I mean mobile computing where both computer-generated graphics and the real world are seamlessly overlaid in your view; there is no separate display that you hold in your hands (think Terminator vision). The underlying trend as we’ve gone from desktops through laptops and notebooks to tablets is one of having computing available in more places, more of the time. The logical endpoint is computing everywhere, all the time – that is, wearable computing – and I have no doubt that 20 years from now that will be standard, probably through glasses or contacts, but for all I know through some kind of more direct neural connection. And I’m pretty confident that platform shift will happen a lot sooner than 20 years – almost certainly within 10, but quite likely as little as 3-5, because the key areas – input, processing/power/size, and output – that need to evolve to enable wearable computing are shaping up nicely, although there’s a lot still to be figured out.
Of course, hardware is only as useful as the software running on it, and there’s a vast web of intertwined issues and questions to be resolved about how the combined hardware-software system might work. What does a wearable UI look like, and how does it interact with wearable input? How does the computer know where you are and what you’re looking at? When the human visual system sees two superimposed views, one real and one virtual, what will it accept and what will it reject? To what extent is augmented reality useful – and if it’s useful, to what extent is it affordably implementable in the near future? What hardware advances are needed to enable the software? And much, much more – there are deep, worthy challenges everywhere you look (and I hope to be posting about some of them soon); in fact, what it reminds me of, but on a larger scale, is Quake, where we had to figure out 3D graphics, client-server Internet networking, file formats, pretty much everything from scratch. Indeed, I think this has the potential to be, like Quake, a technological inflection point after which everything has changed.
In fact, this technology is potentially a big step in the direction of Snow Crash. What goes around comes around: I wouldn’t be at Valve doing this – in fact, Valve itself might not be here – if it weren’t for Snow Crash diverting my career to Id in the first place.
After I had thought all this through and done a lot of research, I came to the conclusion that it would be valuable to spend some time to see if wearable computing was an area that Valve should get into as it developed, so I ran my findings and thinking past a lot of people I respect at Valve. The consensus was that investigating wearable computing was an experiment worth running; the main concern was that the experiment needed to be structured so there were clear tests for success and failure, and so that we’d get useful information in either case. But no one told me what to do, and there were no official approvals that I had to obtain; once I had gathered feedback and thought this through to my satisfaction, I just went ahead and started the project.
Think about that for a second, and think about your own job. How cool is it that this project spun up almost overnight, just because I thought it was the most valuable thing I could do at Valve?
To be clear, this is R&D – it doesn’t in any way involve a product at this point, and won’t for a long while, if ever – so please, no rumors about Steam glasses being announced at E3. It’s an initial investigation into a very interesting and promising space, and falls more under the heading of research than development. The Valve approach is to do experiments and see what we learn – failure is fine, just so long as we can identify failure quickly, learn from it, and move on – and then apply it to the next experiment. The process is very fast-moving and iterative, and we’re just at the start. How far and where the investigation goes depends on what we learn.
It also depends on who’s doing the investigation. The team has grown, and we’re making good headway, but there’s a vast amount of stuff to investigate, and we need more smart people. Lots more smart people. Hardware people, software people, firmware people, game people, UI people, just plain great programmers and problem solvers, industrial designers, mechanical engineers, electrical engineers, systems programmers, computer vision people, optics engineers, you name it.
Maybe you
If you’re excited at the idea of diving into wearable computing, and you’re the right sort of person, then you’re why I wrote this post. We want you here – and you should want to be here; read back over this post and see if that isn’t so. Shoot me an e-mail, and we’ll go from there.
Even if my particular project doesn’t excite you, you should think hard about coming to Valve. We’re a great game company, a great digital distribution and community company, and much more as well; we have all sorts of projects going on (the fact that I’m doing wearable computing should give you a hint of the range of things we’re doing), and need all sorts of skills, including but not limited to animators, artists, level designers, sound and music people, writers, web programmers and database programmers and programmers of all sorts, research psychologists, data mining and machine learning specialists, statisticians, user experience designers, account managers and developer relations people, hardware engineers of many sorts, and more. And if you’re a first-class economist, please check us out. You’ll have a sandbox with 40 million users, and I promise you’ll never be bored.
If Valve sounds interesting to you and you think you’re a fit (a future post here will discuss in some detail what Valve looks for in people when it hires), run don’t walk to contact us at
http://www.valvesoftware.com/jobs/job%5Fpostings.html.We can’t wait to talk with you.
I'll post here whenever there's something about what I'm doing or about Valve that seems worth sharing. The initial post is an unusual one - it's long, my attempt to distill the experience of my first year and a half at Valve - but I think it's well worth reading to understand what I'm doing, why I'm doing it, and the context in which it's happening, and just to understand more about Valve in general.
Michael Abrash is the author of several books, including Zen of Code Optimization and Michael Abrash's Graphics Programming Black Book, and has written columns on graphics and performance programming for several magazines, including Dr. Dobb's Journal and PC Techniques. He was the GDI programming lead for the original version of Windows NT, coauthored Quake at Id Software with John Carmack, and worked on the first two versions of Xbox. He is currently working on R&D projects, including wearable computing, at Valve. He can be reached here.