New Audit Tool Keeps Quicksearch in Sync with Library Catalogs

While most MARC records transfer smoothly from Orbis and Morris to Quicksearch, sometimes things go wrong and discrepancies between the systems arise. The recent development and implementation of an automated daily audit process ensures that these errors will be identified and routed for correction swiftly.

The audit script runs each time Quicksearch processes a fresh extract of records from Orbis or Morris. The script identifies the record IDs from the processed extract files. It then contacts the Quicksearch Solr index to acquire an up-to-date list of all the record IDs in Quicksearch, as well as the date each record was last updated. By comparing the extract files to the contents of the index, the script checks that all added records are present with a current update date, and that all deleted or suppressed records have been removed. A notification e-mail immediately reports the audit results to the Quicksearch project team, and includes file attachments listing the record IDs that failed to be processed correctly. These errors are collected, analyzed to determine appropriate handling, and routed to Catalog & Metadata Services or Law Library staff for review.

The next phase in development is for the script to send the daily audit results to a local database, which will in turn power a publicly available visualization that updates in real time, illustrating the volume of catalog records that YUL staff create or update on a daily basis, and charting their flow from Orbis and Morris into Quicksearch.

Quicksearch Update June 2015

The Quicksearch project team has been working hard in the past few months to implement several important “behind the scenes” improvements, and I’d like to highlight a few of them for you:

  • Production Quicksearch is now supported by not one, but two separate servers that are mediated by something called an F5 load balancer. An F5 load balancer acts like a traffic light; for each incoming user request, and it will look to see which of our two production servers is least busy, and direct traffic to that server.  This balanced approach has several advantages.  First, Quicksearch will be able to handle the level of traffic currently experienced by Orbis and Orbexpress.  Second, in the event of a server failure, the load balancer will automatically direct all traffic to the server that is currently up, with no visible downtime for users.
  • We are implementing a new approach to Google Analytics that will enable analytics-driven change in Quicksearch. Quicksearch has been upgraded to use both Google Analytics Universal code and the new Google Tag Manager application.  Google Tag Manager will allow us to more easily track user behavior.  We also developed custom code specifically for identifying the broad use of items by location and call number.  The setup was driven by small group discussions between Library IT and Sarah Tudesco, with the goal of setting up analytics using similar approaches in all of the applications supported by Library IT.  These new analytics will be analyzed over the summer and tweaked as necessary.  Our goal is to finalize our setup by September 2015.
  • Daily record extract and load workflows are being finalized.  For example, a daily Audit script of all loading activities has been added.  The Audit script generates a list of records rejected by Quicksearch for clean up in Orbis or Morris.  The statistics associated with each load will be publicly available in the near future in a graphical dashboard.  A full audit workflow also now exists to compare all publicly-available records in Orbis, Morris, and Quicksearch, and generate a list of any records out of sync between the three public OPACs.


What’s up next in the Quicksearch project?

  • Work is in progress to integrate the display of Morris holdings.
  • We’re also creating public-facing Help documentation for Quicksearch users.  A sub-group of the Quicksearch Implementation team has formed to create this documentation, and will be working in consultation with the User Experience Strategy Group.
  • Re-discover Discovery and other staff sessions will be planned for late summer to support library instruction, in consultation with public services staff.
  • Quicksearch is still on target to replace Orbis as the default search on the Library Front Door in January 2016!