Recently I sat down with John Podlasek on his GameDevAdvice podcast. We covered topics ranging from my personal path through the game industry, starting a game studio Disbelief with my business partner Steve Ellmore, advice for programmers starting out in the industry, projects me or the studio have worked on , anti-crunch culture, and more. Plus we throw in a little Midway reminiscing, both the good and absurd.
You can listen on Apple, Spotify or elsewhere.
Monday, May 1, 2023
Sunday, December 18, 2022
Where to find me online
Since Twitter is setting up a Berlin Wall and banning links to any social media sites, figure I would post where you can find me online here
Mastodon: https://mastodon.gamedev.place/@solidangle
Bluesky: https://bsky.app/profile/solidangle.bsky.social
Not really active, but on these sites as well
Post: https://post.news/solidangle
Cohost: https://cohost.org/solidangle
Monday, May 21, 2018
Open Salaries at Disbelief
Disbelief is a little over four years old. One thing I've wanted to do is write a little more about our experiences of bootstrapping and growing a small services company.
For some background, Disbelief is a tech services company that focuses on problem-solving for game developers. In short, we help people ship their games. Since Steve Ellmore and I founded Disbelief we've grown from five people working out of their apartments to seventeen people spread across two offices in Cambridge, MA and Chicago, IL.
Recently, we completed a transition to open salaries at Disbelief. Everyone at Disbelief knows the salary and responsibilities for each role. We've had a goal of fairness and transparency within the company for a long time, and this is one key part of that.
A problem-solving services company relies foremost on good communication and cooperation, it's literally our business model. If we can't communicate and work well with clients, we are not going to succeed. Building a culture of communication means we have to practice what we preach internally - clearly communicate business goals, current sales activity, and generally be as open as possible so everyone is rowing in the same direction.
We realized part of this would be more openness when it came to expectations of each position. An ad hoc set of criteria locked in our heads was not sufficient and would not scale. We needed more formal definitions of roles.
One principle behind the criteria is they are rooted in business needs. This gives us focus on what is truly important for the business to succeed and what is not.
For example, one column is "External Communication". Junior Programmers are mostly concentrating on learning the craft of systems programming itself - how to take schooling or self-taught lessons and apply them to real systems in the real world, at the quality level Disbelief demands. Reflecting this, their responsibilities for external communication are minimal - they must be able to report status internally while other, more senior engineers handle the bulk of client communication. At the other end of the scale, Senior Programmers are expected to clearly report status to clients, anticipating and managing expectations, and act as ambassadors for Disbelief as a whole.
Defining these roles took a long time, and we thought very carefully about it. It has become a very useful tool. Monthly one on one meetings with managers have a clear set of criteria to discuss performance. Having the deltas between rows allows us to focus our mentorship and training efforts - rather than try to teach everything at once. Promotions can lay out exact responsibilities of the new role, both beforehand and after. New hires can be slotted based on specific criteria rather than gut feelings. Managers can be trained to evaluate using these criteria, which allows us to scale. The 'requirements' section of public job descriptions are mostly pre-written. At the highest level this document lays out a core of what it means to be a Disbelief engineer.
An additional problem we're tackling is adding a non-management, individual contributor track for very senior personnel who do not want to be leads but want a path for advancement. For example, recognizing that someone is a domain expert and can mentor/teach the rest of the company about an area. The last thing we want to do is shove people into a management role who are not going to be effective at it.
We've found this a very useful process that helped us sharpen and define our our approach to our work and business. If we find something that works better, we'll not shy away from change. At the end of the day, whatever route you choose for your company should be rooted in what works for your business. This structure may not work for you, but so far it is working for us.
For some background, Disbelief is a tech services company that focuses on problem-solving for game developers. In short, we help people ship their games. Since Steve Ellmore and I founded Disbelief we've grown from five people working out of their apartments to seventeen people spread across two offices in Cambridge, MA and Chicago, IL.
Recently, we completed a transition to open salaries at Disbelief. Everyone at Disbelief knows the salary and responsibilities for each role. We've had a goal of fairness and transparency within the company for a long time, and this is one key part of that.
The Business Case for Internal Transparency
Our early compensation structure was mostly ad-hoc. Steve and I would get together and discuss a number on a per-case basis. While we'd put a lot of thought into it, we weren't happy with this process. It worked fine when the company was a group of people who we had known for years, but it wasn't going to scale and it wasn't particularly transparent to anyone.A problem-solving services company relies foremost on good communication and cooperation, it's literally our business model. If we can't communicate and work well with clients, we are not going to succeed. Building a culture of communication means we have to practice what we preach internally - clearly communicate business goals, current sales activity, and generally be as open as possible so everyone is rowing in the same direction.
We realized part of this would be more openness when it came to expectations of each position. An ad hoc set of criteria locked in our heads was not sufficient and would not scale. We needed more formal definitions of roles.
Roles and Ladders
What we came up with was a 'ladders' spreadsheet which defines the roles and responsibilities of each position. The rows are each position (Junior Programmer, Programmer 1-3, Senior Programmer 1-3, etc), and the columns describe the responsibilities and criteria for one area of evaluation. The criteria are specific and attempt to minimize the amount of subjective evaluation and ‘gut feelings’.One principle behind the criteria is they are rooted in business needs. This gives us focus on what is truly important for the business to succeed and what is not.
For example, one column is "External Communication". Junior Programmers are mostly concentrating on learning the craft of systems programming itself - how to take schooling or self-taught lessons and apply them to real systems in the real world, at the quality level Disbelief demands. Reflecting this, their responsibilities for external communication are minimal - they must be able to report status internally while other, more senior engineers handle the bulk of client communication. At the other end of the scale, Senior Programmers are expected to clearly report status to clients, anticipating and managing expectations, and act as ambassadors for Disbelief as a whole.
Defining these roles took a long time, and we thought very carefully about it. It has become a very useful tool. Monthly one on one meetings with managers have a clear set of criteria to discuss performance. Having the deltas between rows allows us to focus our mentorship and training efforts - rather than try to teach everything at once. Promotions can lay out exact responsibilities of the new role, both beforehand and after. New hires can be slotted based on specific criteria rather than gut feelings. Managers can be trained to evaluate using these criteria, which allows us to scale. The 'requirements' section of public job descriptions are mostly pre-written. At the highest level this document lays out a core of what it means to be a Disbelief engineer.
Flattened Salaries
What we did next is still controversial in many business circles - we flattened salaries. For each role, every person in that role is paid the same. You can find many business articles that will argue for this, as it can take a lot of arbitrariness out of compensation, and increases fairness. It limits situations where two people are doing the same job and getting wildly different pay. You can also find many that will argue against - it removes incentives for increased performance.
For us, it came down to a growing confidence that we had nailed down the core aspects of what the job was in written form. We're not naive - no document will ever capture everything nor handle every situation. We feel we have enough graduations in roles and coverage to reflect people's day to day contributions. What we found is the harder cases just made us reflect and refine our positions and roles. We have enough flexibility to recognize that not every Senior Programmer is the same, but also have made a lot of effort to write down specific criteria in specific roles rather than go with gut feelings.
Getting to a flattened salary structure required a period of adjustment -- mostly giving people raises to get everyone at their proper level for their role. This took some time as we're completely bootstrapped, and had to make sure our adjustments didn't cause payroll to outstrip our revenue. What made this a little easier is around the same time, we knew we had to do a series of competitiveness raises to get us closer to the market. These raises were broad based, so most people at the company got raises around the same time - some for competitiveness, some also for normalization.
So far we haven't seen many of the downsides or gotten a lot of negative feedback. We took our time with the transition, explaining it to the entire company, and discussing one on one. More often than not we find this a selling point with candidates.
One reason is base salary is only one part of our compensation. Aside from our benefits package, we have a bonus structure that is based on the performance of the company as a whole. If the company does better, everyone does better.
Cost of living adjustments are done yearly and across the board, and completely separate from merit/promotion increases. Additionally, we frequently review our compensation structure and will make competitiveness adjustments to a position's salary if we feel we're not in line with the market.
At the end of the day, compensation is only one factor in job satisfaction. We feel the positives of equitable salaries at each position outweigh the negatives and foster a culture of cooperation. What works for us may not work for you.
Open Salaries
We have had an open 'Ladders' spreadsheet within the company for quite a while, and everyone knew the salary structure was flat at each role. Only recently did we decide to attach salary information - we wanted to make sure we had time to address any negative feedback to the new compensation structure.
So far the reaction internally and externally has been positive. When mentioning we were releasing salary info internally to a couple of friends running their own companies, their immediate reaction was 'oh I've wanted to do that, how did you get there?" That's part of why I wanted to write up our experiences.
The Future
We hardly consider ourselves done - the compensation structure and roles are a living document, continually reviewed. For instance, we only have programmer roles defined in this structure. While we only recently have started to hire non-programmer roles, we need to define and add them to the overall structure.An additional problem we're tackling is adding a non-management, individual contributor track for very senior personnel who do not want to be leads but want a path for advancement. For example, recognizing that someone is a domain expert and can mentor/teach the rest of the company about an area. The last thing we want to do is shove people into a management role who are not going to be effective at it.
We've found this a very useful process that helped us sharpen and define our our approach to our work and business. If we find something that works better, we'll not shy away from change. At the end of the day, whatever route you choose for your company should be rooted in what works for your business. This structure may not work for you, but so far it is working for us.
Join Us
If you've read this and thought "I'd like to know more about Disbelief", check out our open positions and drop us a line.
Friday, November 6, 2015
Saturday, August 15, 2015
Tips for Navigating Large Game Code Bases
(Author's Note: This article is about navigating large game code bases, which can go up to and past the 2 million lines of code mark and are usually mostly C++. I'm sure these tips are applicable to other industries with comparable size projects, but you write what you know)
Freshly hired, you sit down at your desk at your hard won job at Big Game Studio. You're making games! Big ones! Excited to get started, you've gone through the orientation, pulled the source tree to your machine, and fire up the IDE. You wait for it to load... and wait... and wait... just how large is this project? Your producer has already given you a simple task for their editor -- make the FooBaz window remember its position and size between runs of the editor. You have no idea where to start. A sense of panic comes over you as you realize little in your previous experience has prepared you for dealing with a code base this big.
It is not uncommon these days to land a game programming gig and have to deal with a large legacy code base. It could be all in-house code, it could be a licensed engine, or it could incorporate a lot of open source software. Unless you are working on a small game, more than likely the first thing you are going to have to learn is how to navigate this beast.
Freshly hired, you sit down at your desk at your hard won job at Big Game Studio. You're making games! Big ones! Excited to get started, you've gone through the orientation, pulled the source tree to your machine, and fire up the IDE. You wait for it to load... and wait... and wait... just how large is this project? Your producer has already given you a simple task for their editor -- make the FooBaz window remember its position and size between runs of the editor. You have no idea where to start. A sense of panic comes over you as you realize little in your previous experience has prepared you for dealing with a code base this big.
It is not uncommon these days to land a game programming gig and have to deal with a large legacy code base. It could be all in-house code, it could be a licensed engine, or it could incorporate a lot of open source software. Unless you are working on a small game, more than likely the first thing you are going to have to learn is how to navigate this beast.
Monday, March 3, 2014
BioShock Infinite Lighting
Goals
One of our artists best described Infinite's style as "exaggerated reality." The world of Columbia was colorful, high saturation, and high contrast. We needed to handle both bright, sunny exteriors and dark, moody interiors simultaneously. We were definitely not going for photorealism.
The size of the levels were bigger than anything Irrational had attempted before. The previous game Irrational had worked on, BioShock, was more of an intimate corridor shooter. In contrast, we wanted Columbia to feel like a big city in the clouds. This meant much bigger and much more open spaces that still retained the high detail required for environmental story telling, because much of the story telling in a BioShock game was done via the world itself.
The size of the levels were bigger than anything Irrational had attempted before. The previous game Irrational had worked on, BioShock, was more of an intimate corridor shooter. In contrast, we wanted Columbia to feel like a big city in the clouds. This meant much bigger and much more open spaces that still retained the high detail required for environmental story telling, because much of the story telling in a BioShock game was done via the world itself.
Finally, all of this had to perform well on all of our platforms.
 |
| The end result |
Hybrid Lighting System
The lighting system we came up with was a hybrid system between baked and dynamic lighting:
- Direct lighting was primarily dynamic
- Indirect lighting was baked in lightmaps and light volumes
- Shadows were a mixture of baked shadows and dynamic shadows
- The system handled both stationary and moving primitives.
Deferred Lighting
Dynamic lighting was handled primarily with a deferred lighting/light-pre pass renderer. This met our goals of high contrast/high saturation -- direct lighting baked into lightmaps tends to be flat, mostly because the specular approximations available were fairly limited. We went with the two-stage deferred lighting approach primarily because the information we needed for our BRDF and baked shadows would not fit in four render targets. We did not want to sacrifice UE3's per-pixel material parameterization, so something like a material id system to compact the G-Buffers was out of the question. This of course meant two passes on the geometry instead of one, which we dealt with by running render dispatch in parallel, instancing, and clever art.
There's been a ton written on this technique, so I'm just going to point out a few wrinkles about our approach.
Sunday, July 7, 2013
Code reviews
After reading Aras' Reviewing ALL the CODE entry, I was going to reply describing our process, but it was getting long so I decided to write it up here.
Our process is a little more lo-fi but effective. We use perforce's review daemon, and as part of programmer orientation we set new programmers up to subscribe to the source code folder and set up an outlook filter. That's right, every programmer on the team has a stream of emails for every changelist.
The emails are set up with the first line of the changelist description in the subject and the email of the changelist author in the reply-to field. The body of the email contains the full changelist comment and diffs of the change up to a certain size to avoid flooding our email system.
Reviews are handled by replying to the email, and cc'ing a code reviews email list which goes to everyone and is archived. This is so everyone gets the benefit of subsequent discussion.
We have a few senior engineers who at least read every changelist comment. Personally, I find it is something useful to do while waiting the couple minutes for a big compile to finish. But looking at our code reviews email list, quite a few programmers scan at least some of the changelists, usually looking for changes in code they are most familiar with.
This only works if you enforce meaningful changelist comments. "Fixed bug in renderer" would not be an acceptable changelist comment and would garner a review email to make a better one. A changelist comment should describe the problem being solved and how it was solved. It doesn't need to be a novel but it should be enough information so someone going back to this changelist 6 months from now can understand what was done and why.
I've worked on teams where this was the primary code review process, although currently we use it as a second line of defense - each changelist requires a primary reviewer.
We catch all sorts of issues with this review process - minor issues such as code that is unclear all the way to major bugs that were uncaught by both the author and the primary reviewer. This is a good method for orienting new programmers to the code base, teaching "code base lore", or pointing out bad naming. One of the things I particularly look for is badly named functions or variables - code without short, concise and meaningful names is usually an indicator of a larger problem. I could do a whole entry on names.
Beyond the day to day issues, I also find it is a useful toward improving the quality of the code base as a whole. If you see the same mistakes being made or people having trouble with a particular system, it gets you thinking about ways to prevent those mistakes, or make a system easier to use.
All in all, I find it useful for getting a feel for how the team as a whole is operating and also learning about parts of the code base you might not normally delve into. It's difficult to give people advice on how to solve problems if you don't have at least a cursory understanding of what they are working on. It is also low ceremony and a way to communicate what's going on across largish teams. If you're not doing it, give it a try.
Our process is a little more lo-fi but effective. We use perforce's review daemon, and as part of programmer orientation we set new programmers up to subscribe to the source code folder and set up an outlook filter. That's right, every programmer on the team has a stream of emails for every changelist.
The emails are set up with the first line of the changelist description in the subject and the email of the changelist author in the reply-to field. The body of the email contains the full changelist comment and diffs of the change up to a certain size to avoid flooding our email system.
Reviews are handled by replying to the email, and cc'ing a code reviews email list which goes to everyone and is archived. This is so everyone gets the benefit of subsequent discussion.
We have a few senior engineers who at least read every changelist comment. Personally, I find it is something useful to do while waiting the couple minutes for a big compile to finish. But looking at our code reviews email list, quite a few programmers scan at least some of the changelists, usually looking for changes in code they are most familiar with.
This only works if you enforce meaningful changelist comments. "Fixed bug in renderer" would not be an acceptable changelist comment and would garner a review email to make a better one. A changelist comment should describe the problem being solved and how it was solved. It doesn't need to be a novel but it should be enough information so someone going back to this changelist 6 months from now can understand what was done and why.
I've worked on teams where this was the primary code review process, although currently we use it as a second line of defense - each changelist requires a primary reviewer.
We catch all sorts of issues with this review process - minor issues such as code that is unclear all the way to major bugs that were uncaught by both the author and the primary reviewer. This is a good method for orienting new programmers to the code base, teaching "code base lore", or pointing out bad naming. One of the things I particularly look for is badly named functions or variables - code without short, concise and meaningful names is usually an indicator of a larger problem. I could do a whole entry on names.
Beyond the day to day issues, I also find it is a useful toward improving the quality of the code base as a whole. If you see the same mistakes being made or people having trouble with a particular system, it gets you thinking about ways to prevent those mistakes, or make a system easier to use.
Subscribe to:
Posts (Atom)