Safari’s Content Team has the dubious distinction of having the highest volume of tickets in our company-wide issue management tracking system (we use Atlassian’s JIRA). We easily win this competition, with more than 1,500 open issues on any given day. But do we buckle under the psychic weight of all these tickets? Nah… go ahead, bring ‘em!
Why So Many, You May Ask?
The Content Team has quality-checked 12,729 brand new titles loaded onto Safari Books Online from April 2011 to last week. For the past 6 months, we averaged 753 titles/month, or 177 titles/week. We track only issues that are clearly errors (e.g., a title-cover image mismatch) or issues that seriously impact readability (e.g., all images are random color bitmaps like this one from a real book).
Each time we find an issue like this, we stop the title in the pipeline before it goes live, and follow up one way or another to correct it. We track all of these issues in JIRA, so we can manage the corrections and move each title live as quickly as possible.
At this time, we only check brand new titles, but our publishers are free to update titles at any time without oversight. And, since we only started quality-checking new titles in April 2011, but Safari launched way back in September 2001, there are quite a few titles that we haven’t scrutinized. Various problems get reported: the unavailability of practice files referred to in the text, teeny tiny images too small to make out, or broken links. An average of 200 new content issue tickets are created each month.
That explains where our issues are coming from. So, how do we manage them?
Standardization, Automation, and Elbow Grease
Well, managing these issues has been an evolving process. We are fortunate to have on staff not just one, but several JIRA experts, who are always willing to help us out with custom fields and productivity brainstorming.
We’ve been working our way up to several key improvements, which are now at a point where we are starting to realize the benefits. With >1,500 issues, global improvements don’t happen overnight. It’s easy to add new fields to help us organize and track issues, but then those fields need to be populated – a daunting task. And of course, in order for this system to work, everyone has to use it the same way — which means a bit of documentation, training, and oversight are needed. Here are the keys to managing this type of issue volume:
- Standardization: custom fields, boilerplate language
- Automation: QaQ, automated email
- Elbow Grease: Monthly issues export & follow up
- NEW: Greenhopper
Standardization. Custom JIRA fields help us slice and dice the issues into manageable groups. For example, we added a publisher field, which allows us to export all the open issues for a given publisher. We use a component field, which allows us to sort that publisher’s open issues by whether the issue relates to the source PDF, the source EPUB, the metadata, companion files, etc.
And we have boilerplated the language we use in certain fields, which serves two purposes. First, it saves the ticket writer time – she doesn’t have to consider how to explain a given issue, she can rather just copy/paste the explanatory text from our (constantly updated) JIRA Issue Map. Second, we make sure our boilerplate language is clear enough for publisher-facing communications, even if our primary publisher contact is a rights person who has no need to speak the lingo of CSS or toc.ncx, for example.
Automation. Our stellar engineering team has built us an QA Queue application (we call it the QaQ) to manage our daily load of new titles to quality-check, and this system hooks right into JIRA. After we check a publisher’s new batch of titles, we follow up via email to let the publisher know which titles are live, and which need a little more work before they can go live. The QaQ automates the creation of lovely formatted emails; for titles with associated JIRA tickets, it exports the text from key fields which detail the required fix in easy-to-understand language.
Elbow Grease. We are now rolling out a monthly export of issues for each publisher. When a publisher receives a spreadsheet listing their issues in detail, sorted by issue type, it’s a lot easier for them to follow up en masse, so they can get as many new titles live (or corrected, if they are already live) as quickly as possible. We did a pilot of this new process with a select set of publishers, with very promising results. We don’t want our publishing partners swimming in the JIRA sea, nor should we require them to rely on email alone for making sure all their titles are working well on Safari.
New: Greenhopper. This plug-in to JIRA has us really excited. We are doing a trial run with a Kanban workflow for the subset of Content issues requiring engineering work. In 2010, we were managing the long list of engineering Content issues via JIRA and email alone. Well, that doesn’t work so well once you have more than a handful of issues. So in 2012, we switched to a shared Google doc so we could be sure we were all working off the same songsheet. But even that has its shortcomings – we meant to keep notes in the Google doc and ALSO update each JIRA ticket as we worked. In theory. Often, only one or the other would get updated, and sometimes the priorities in the doc didn’t match the priorities in JIRA.
But with Greenhopper, we plan to kiss the Google spreadsheet goodbye, for the most part. We created a Kanban board with a few key buckets: Pending, In Progress, In SBO QA, and Completed. We are strictly limiting the number of In Progress tickets to 10. (If you go over 10 tickets In Progress, the whole board turns a distressing bloody red.) This way it’s very clear for engineering to know exactly what must be worked on. And the Kanban board is very easy to work with – in our status calls, we can discuss the entire board, and update each individual issue as we discuss it from the same board. No more getting lost in a sea of dozens of browser tabs or windows.
If this Greenhopper experiment works well for our Engineering tickets, we will explore creating boards for other types of Content Issues. The sky seems to be the limit in terms of how you structure your boards; they seem fully customizable based on the fields you want to use.
OK, now that we have these great tools in place and are starting to use them, we can start setting some nice aggressive goals to get our overall numbers down. (The team is going to kill me when they hear this.) Let’s beat our current created-to-resolved ratio by summer, guys!