Want a transparent look into how a startup iterates and makes the people who work there more engaged and more effective? Dave Zwieback of Next Big Sound shared a bit about “the way we work” and the beginning of their iteration process in Part 1 of this series, and in Part 2, he outlines the roadblocks their initial decisions caused and how they overcame them to be more effective. — KateS
In Part 1 of this article, we covered roughly 6 months of iterating on the way we worked at Next Big Sound, during which we introduced self-selection (you get to pick what you work on, whom you work with, and where you work); the BugBusters incident management rotation along with the associated personal hack weeks.
With more experience with self-selection, we realized the importance of accountability and high-quality communications to our way of working, and continued to experiment with ways of improving both of these critical areas.
November 2013: When projects go un-selected
When people hear about self-selection, the first question is usually whether there are projects that don’t get selected. What about that “shit project” that no one wants to work on? What about projects with external customer deadlines?
Yes, it’s true, it happens: sometimes seemingly important projects don’t get selected. When this happens, no one is ever coerced or forced to work on a project that they did not select. Instead, we ask a lot of questions, starting with, not surprisingly, “Why?”.
If the project’s importance is obvious, why did it still not get selected? Was its importance clearly communicated? Does the team have the necessary skills to complete the work? Are we working on other projects that are higher priority? If we think the project is, in fact, important, we have at most two weeks to advocate for its selection for the following iteration. Otherwise, we could be thankful to the “wisdom of the crowd” for showing that the project is not as critical or time-sensitive as initially thought.
With that in mind, we introduced a concept of project advocate, a person who could provide the necessary context to the team during self-selection. Ideally, the advocate would cover things like why this project is important to do during the coming iteration, and how it ties to company themes and goals. In addition, we decided that each project idea should explicitly list the skills required to complete it (e.g., front-end developer, Java, design).
We also noticed that the timing of communication of iteration scopes, updates, and retrospective results was somewhat haphazard. (All these are communicated via e-mail to the entire company).
Because accountability–which, in this case, is literally the responsibility to provide an account of what’s happening to the entire team–is such a critical part of self-selection, we agreed to adhere to a clear communication schedule, which specified the exact timing of initial iteration scopes, mid-iteration progress updates, scope changes, and retrospectives.
In the two months since the last tune up to the way that we were working, it also became increasingly clear that BugBusters was a project that we had to have someone select at all times (not letting it go during a week when no one selected it, as we had done before).
For at least one two-week iteration, no one selected to be on BugBusters, which actually resulted in higher interrupt levels for most engineers. In addition, certain tasks (like managing over 100 data sources) were falling disproportionately to several engineers and client services folks, who were unable to fully dedicate their time to other projects.
Most importantly, we realized that having the rotation was required for self-selection to apply fully to all engineers: we did not want to have a single engineer dedicated to BugBusters/incident management at all time, because she would not be able to fully participate in self-selection. More fundamentally, having someone on ops at all times (not just when someone wants to) is required for us to operate the Next Big Sound service.
After a lengthy and heated all-hands discussion, we agreed to have someone on BugBusters at all times and created a place for engineers to track (and trade) their upcoming rotations. To be clear, this still does not mean that every engineer must do BugBusters (although by this point, every engineer has completed at least one rotation). Simply put, as part of self-selection, we trust everyone to do what’s right for them, their team, and NBS.
March 2014: Doing demo day
With the mechanics of self-selection mostly worked out, we continued what has been (according to the founders) the longest sustained period of high productivity in the company’s 5-year history. We next turned our attention to demo day – a high point of the iteration when everyone gets to show their work and celebrate our progress as a team.
Perhaps the best way to illustrate the issues with demo day was to compare its intention with how it was actually practiced:
- demos are short (under 7 minutes)
- demo day should last about 1 hour
- demos highlight “the difference between what was initially planned and what was accomplished, including identifying any loose ends”
- demos are explicit opportunities to learn from others, and the most salient parts of retrospectives are emphasized
- demo day artifacts (e.g., presentations) should be easily found
- the entire history of any iteration should be easily accessible at any time to anyone in the company
- we intend the software that we write to be tested, documented, and shipped to production during the iteration
- demos are sometimes short, and frequently go over 7 minutes
- demo days have lasted as long as 2 hours
- demos sometimes highlight the difference between the original scope and what was completed; remaining work is sometimes documented as Trello cards
- demos sometimes refer to lessons learned and stop/start/continue items from retrospectives
- there is no central place for demo day artifacts
- only the original and updated scopes and results of retrospectives can be found in e-mail (if you are at the company when it it sent); it’s not always clear what was actually shipped and how that might differ from what was planned
- the status of testing, documentation, and shipping is sometimes mentioned during demos, and not consistently documented
- many demos use PowerPoint presentations, with “the rate of information transfer … asymptotically approaching zero”
The first thing that we nipped in the bud was the (over)use of PowerPoint. Instead, we opted to write out iteration summaries, and store them in a central place (Google Drive). We created an (optional) summary template, and agreed to keep summaries under six pages in length. We then decided to try reading the summaries simultaneously as a group during demo day, in the same way that we’ve done during several all-hands meetings (inspired by the same practice during meetings at Amazon).
With the introduction of the iteration summaries, we now had a rich, historic, narrative record of each iteration. Due to the high-bandwidth communication of the written word (and an occasional animated gif), the team now had a higher degree of awareness about the many projects going on than ever before.
However, because we also limited the actual demos to two minutes, the social aspect of presenting your project in front of the entire team was greatly diminished. The high-energy demo days became subdued. Before we could address this issue, though, we identified another gap between how we thought about the way we worked vs how we actually worked that we had to dive into first.
April 2014: Getting things done
Starting in July of 2013, we settled into the comfortable rhythm of two-week iterations. However, in reality, few big projects really divide neatly into two-week chunks. As a result, either “work [expanded] so as to fill the time available for its completion” (according to Parkinson’s law); or people took on additional work to fill available time within the iteration. Both of these situations were common, but not easily visible (which is why it took us this long to address the situation).
In addition, because we were emphasizing iterations that are exactly two weeks in length, shorter project and leftover items from previous iterations that took less than two weeks to complete were effectively “second-class citizens.”
That is, two-week iterations subtly encouraged the selection of projects that took at least two weeks to complete. As a result, some projects took a long time to finish, staying at the “almost complete” mark for extended periods of time, and incurring a significant context-switching cost to get to 100% complete.
We also had a whole class of projects (namely data science reports and data journalism articles) that were much more dynamic in nature, with constantly changing priorities (driven by customers or partners), and highly variable effort required to complete them. In recognition of this, some team members were already working outside the structure of normal two-week iterations. A recent experience of cramming several smaller research projects into a two-week iteration has further highlighted the awkward fit of rigid iteration lengths to this type of work.
To address the above shortcomings of fixed length iterations, in early April, we agreed to try working in iterations that could vary in length, not to exceed two weeks. To minimize the cost of context-switching, we also encouraged folks to stay on projects for their duration (not just for an iteration). As one engineer put it, “One fully complete project is better than five halfway complete ones.”
We also agreed to make context-switching explicit: recognizing the fact that we can only work on one thing at a time, we can only mark one task as “in progress” at any given time, marking all others as “blocked”. (We use Trello for tracking our work).
We also changed demo day to accommodate the new, variable-length iterations. During the bi-weekly demo day, folks present the results of any iteration that has been completed before that Wednesday. That might be one or 10 iterations. If an iteration was not fully completed before a particular demo day, its results would be presented during the next one:
We’ve also returned to more interactive demo days. While we still ban PowerPoint and generate iteration summaries (which everyone is encouraged to read offline), we’ve shifted the focus of demo days back on actual demos.
What has worked in the past may not work in the future. As we grow, we remain committed to collaboratively iterating not only on our products, but also on the way that we work.
Unwilling to settle for cookie cutter approaches, we’ll continue to experiment until we find methods that best fit our culture and the challenges ahead.
This evolution is not driven solely by management; in fact, many of the changes described above were championed by designers, client services folks, data journalists, data scientists, and engineers. We strongly believe that this is one of our competitive advantages, and one of the conditions that makes people working at Next Big Sound stay highly engaged.
I would like to thank my colleagues Liv Buli and Karl Sluis for their excellent and tireless feedback and advice on writing this story. I would also like to thank the entire Next Big Sound team for so fearlessly embracing the iterative approach to the way we work.
Dave Zwieback has been managing large-scale mission-critical infrastructure and teams for 18 years. He is the VP of Engineering at Next Big Sound. He was previously the head of infrastructure at Knewton, managed UNIX Engineering at D.E. Shaw & Co. and enterprise monitoring tools at Morgan Stanley, and also ran an infrastructure architecture consultancy for 7 years.