Tag Archives: data remediation

New Audit Tool Keeps Quicksearch in Sync with Library Catalogs

While most MARC records transfer smoothly from Orbis and Morris to Quicksearch, sometimes things go wrong and discrepancies between the systems arise. The recent development and implementation of an automated daily audit process ensures that these errors will be identified and routed for correction swiftly.

The audit script runs each time Quicksearch processes a fresh extract of records from Orbis or Morris. The script identifies the record IDs from the processed extract files. It then contacts the Quicksearch Solr index to acquire an up-to-date list of all the record IDs in Quicksearch, as well as the date each record was last updated. By comparing the extract files to the contents of the index, the script checks that all added records are present with a current update date, and that all deleted or suppressed records have been removed. A notification e-mail immediately reports the audit results to the Quicksearch project team, and includes file attachments listing the record IDs that failed to be processed correctly. These errors are collected, analyzed to determine appropriate handling, and routed to Catalog & Metadata Services or Law Library staff for review.

The next phase in development is for the script to send the daily audit results to a local database, which will in turn power a publicly available visualization that updates in real time, illustrating the volume of catalog records that YUL staff create or update on a daily basis, and charting their flow from Orbis and Morris into Quicksearch.