Quicksearch beta, a Blacklight-powered library search of the Orbis and Morris catalogs, along with Articles+ from Summon, has been available for library-staff testing since October 2014. Recently staff from around the library conducted usability tests of Quicksearch beta with 14 students. Tests took about half an hour to complete, and consisted of a set of typical library search tasks such as finding a book by title and author, looking for a journal, finding books and articles in a topic, and saving/requesting and citing material. Teams of library staff worked on tests, with one person recruiting testers, a second asking the questions and running the tests, and a third person recording the sessions. Test takers were asked to think aloud during testing and their comments and actions were written down by the observer staff person. These types of “think aloud protocol” usability tests are considered an inexpensive and reliable way to uncover problems with digital interfaces, and are conducted as standard parts of interface development cycles.
Usability testing was done in large part to determine if Quicksearch beta is ready for wider release to Yale students and faculty in the spring semester. During testing the only problem uncovered which must be addressed before Quicksearch beta can be more widely disseminated to the Yale community was that response time slowed considerably, to a level which is not acceptable in a search interface. Modern expectations of search is that results will be returned within one second on average. Anything longer than two or three seconds causes users to question if an interface is working. LIT staff are now working to address the slow response times. Culprits may include the SOLR index used in Quicksearch beta, the Blacklight application, the hardware environment, or a combination of these three things. This complexity makes troubleshooting a challenge.
Quicksearch beta includes results from books and articles, which are presented side-by-side in one screen. Labeling in this display was problematic for test takers. The use of Catalogs (to represent material from the Orbis and Morris catalogs) did not seem to mean much to test takers. We recommend that Catalogs be replaced by Books+ to make the material found in Catalogs clearer and to draw a more obvious distinction with Articles. We also recommend changing Articles to Articles+, and to be consistent with this wording throughout the search application.
Test takers had some trouble with complexity of moving between areas for books and articles, and understanding when books were being searched, when articles were being searched, and when both types of materials were being searched. Visual cues need to be added to the books only and articles only areas so that that the material searched there is more obvious.
One task for which Quicksearch was not successful for most users was finding a recent issue of a journal. The search did not do a good job bringing a journal title to the top of search results. The SOLR index and relevancy ranking can be manipulated to make this type of search function better, or other targeted ways of doing a journal title search need to be added to the Quicksearch beta interface.
Another problem was seen when users were asked to limit a set of book results to a more targeted set of books in a subtopic (in this case, books that were about the law or legal aspects of a subject such as CEO compensation, Ebola, or climate change). The Quicksearch beta interface should make such a task straightforward, or at least that the hope, through the use of subject facets. Most users could not, however, complete this task using subject facets. There was a greater rate of success (73% success versus 46% success) when users were asked to do a similar narrowing of results of articles. Users are presented with different sets of facets for books and articles, and there are actually less facets available for articles. It could be that the simpler list of facets makes the topic facet more easy to spot and use in article search results.
Other aspects of Quicksearch beta worked very well and caused few problems for most users. All test takers were able to successfully find a book by a known title, identify where it was in the library system, and navigate to the order form. Most test takers were also able to save article records for later use and email citations.
Probably the biggest win from Quicksearch beta is just the simple fact of the single search of Orbis and Morris in one interface. There was not a lot of name recognition of Orbis demonstrated by these students and even less of Morris. The chance of a student searching one of these resources seems to be decreasing over time, the chance of searching both of them for material seems unrealistic. Quicksearch beta successfully combines both resources in one search, thereby saving time for Yale users and potentially surfacing far more library material in the process.
Read the summary or get the full report.
