We’ve made quite a few changes lately to the search engine and interface on Safari Flow. These changes were motivated in no small part by the evolution of the application itself. When Flow launched last summer, it included a highly curated collection of approximately 250 books and videos, primarily aimed at web developers. We’ve more recently decided to expand the scope of that content, and the original 250 has increased 100-fold — almost exactly, in fact: today, we’re rapidly approaching 25,000 titles on a myriad of topics from dozens of publishers.
The evolution of searching in Flow
Searching across 250 titles is not the same as searching across 25,000. We’ve addressed this shift with a number of refinements, some small and some large. Among the smaller changes we’ve made are adding autocomplete, giving users the option to search across specific indexes (author, title, published, and ISBN), and some retuning of how search results are weighted (one example: book titles are now weighted slightly more heavily than chapter titles).
More drastically, we refactored the interface of the results page. When we had fewer titles, the results page was quite simple and emphasized individual chapters over complete titles.
Search results in the new interface are much richer. Because users have so many more results from which to choose, we aimed to provide tools and information to make choosing easier. For example, results are shown in context, and the user has more filtering options. A user can filter out specific types of media, and sort on relevance or publication date. She can drill down further by clicking on a specific author or publisher. And perhaps the best part: search results are returned twice as quickly.
Reviewing the results
Our hypothesis was that providing users with a richer search experience would make the site easier to use and generally more engaging. Roughly 81% of logged in visits include a search of some kind (40% of which originate on the home page), so it’s important to us that we get it right. So how are we doing?
The new search interface has been on the site for a couple of weeks now, and we’re encouraged by the some of the early analytics. For example, the number of times users search again after the initial search has decreased by 7%. So they are finding what they want more often. Great!
We’ve also seen an uptick in engagement. For example, the amount of time a user stays on the site after searching has increased on average by over a minute. The average number of pages a user views after searching has gone up by a full page. We see these data points as signs that we are doing something right.
Not all of the analytics are unambiguously good or bad. One of the more perplexing data points is related to a new autocomplete feature. Specifically, autocomplete has not consolidated the search terms. Before we added autocomplete, we saw 17,824 unique search terms and 26,814 unique searches, which is 66% unique. Since we added autocomplete (using a smaller sampling size), we’ve seen 10,689 unique search terms and 13,262 unique searches, which is 81% unique. We’re not quite sure to make of that!
This is of course not the end of the story. We know we still have a lot of work to do so that Flow’s search serves users accurately and quickly. Our users regularly write in to suggest changes, and we’re listening. And as you can see in this post, we’re making sure to measure the results of the choices that we make. In addition to the usage on Flow, we have the further benefit of over a decade’s worth of search-related data from our Safari Books Online platform.