Linking People: Collaborations between Metadata Librarians and Programmers

Presentation given at the 2017 Mashcat event held in Atlanta, GA.

Linking People: Collaborations Between Metadata Librarians and Programmers Jeremy Myntti @jmyntti Anna Neatrour @annaneat Liz Woolcott @lizwoolcott Why? Metadata problems when collections are aggregated: • • • • • • • • • • Savage, C. R. (Charles Roscoe), 1832-1909 Savage, C. R.(Charles Roscoe),1832-1909 Savage, C.R. (Charles Roscoe) Savage, Charles R. C. R. Savage (Charles Roscoe Savage and George Ottinger), Pioneer Art Gallery, East Temple Street, Salt Lake City, Utah Charles R. Savage Savage, Charles Roscoe C. R. (Charles Roscoe) Savage, photographer Charles R. Savage, perplexed by his many name Savage, C. R. variants. Charles R. Savage Charles R. Utah State Historical Society Classified Photograph Collection https://collections.lib.utah.edu/details?id=432953 More metadata, more metadata problems • Vendor based DAMS often not offering good authority control solutions • Libraries in Utah often consult additional regional names sources • Daughters of Utah Pioneers name index: http://www.dupinternational.org/pioneer_index.php. •Hosting collections for many partners means less control over cataloging practices. •Use LC Authorities as best we can Local pilots at Marriott Library ● Pilot project with Backstage Library Works ■Replicate MARC authority control project in digital collection XML records ● Pilot project using OpenRefine ■Reconciling data with LCNAF and VIAF Van de Vanter, Billy's car https://collections.lib.utah.edu/details?id=126695 Overview of grant 1. Investigation a. May 2016 - October 2016 2. Evaluation and Testing a. November 2016 - April 2017 3. Pilot Implementation a. May 2017 - October 2017 4. Assessment and Future Planning a. November 2017 - April 2018 Christmas party at Lafabreques, Topaz Museum https://collections.lib.utah.edu/details?id=341268 Collaborating with programmers • Digital Library Services (DLS) and Digital Infrastructure Development (DID) are distinct departments at the Marriott Library • DLS has digitization staff, metadata librarians, digital preservation • DID has application development, sandbox server support • PIs Consulted with DID on all phases of grant development brief proposal, final proposal, budget Collaborating with other institutions Project partners include: ● Utah State University ● Brigham Young University ● Utah State Archives ● University of Oregon ● University of Nevada, Reno ● University of Denver Advisory Board: • Kevin Ford, Art Institute of Chicago • Gretchen Gueguen, DPLA • Eric Miller, Zepheira • Philip Schreur, Stanford "Looking Across Donner Lake - To the Summit" - Two people visible in small boat Utah State University Western Photographers http://digital.lib.usu.edu/cdm/ref/collection/westernphoto/id/779 Partner Perspectives DAMs: • CONTENTdm • Hydra • Islandora Types of collections & content: • Historical/archival material • Mostly images and text, increasing audio and video Partner Perspectives Outcomes Desired: • Well organized, well managed regional linked data repository • Rich documentation of best practices and workflows for creating/sharing local linked-data compliant authorities • Ability to authorize names through NACO/SACO (in the long run) • Contributing names to the larger library/archival community Phase 1: data model review Data to capture • • • • • Preferred form of name Alternate forms of name Local authority source Institution holdings Relationship information Data Models Explored • • • • SKOS OWL BIBFRAME Authorities/Agent/Role EAC-CPF Diary of B. H. Roberts, 1884-1885, page 1 http://content.lib.utah.edu/cdm/ref/collection/uw/id/3069 Phase 1: data model review • • • • • Discussed possibilities for data modeling with partners Based on data samples received and goals for project, thought EAC-CPF best fit our needs Archival standard, many digital collections come from Special Collections/Archives http://eac.staatsbibliothek-berlin.de SNAC http://socialarchive.iath.virginia.edu /index.html Brigham Young portrait Utah State University Extension, Enterprise, and Education: the Legacy of Co-operatives and Cooperation in Utah http://digital.lib.usu.edu/cdm/ref/collection/coops/id/1708 Phase 1: data model review (EAC record) <?xml version="1.0" encoding="UTF-8"?> <eac-cpf xmlns="urn:isbn:1-931666-33-4" xmlns:xlink="http://www.w3.org/1999/xlink" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:schemaLocation="urn:isbn:1-931666-33-4 http://eac.staatsbibliothek-berlin.de/schema/cpf.xsd"> <control> <recordID>http://wnafvocab.org/ark:/12345/67891011/</recordID> <maintenanceAgency> <agencyName>Western Name Authority File</agencyName> </maintenanceAgency> </control> <cpfDescription> <identity> <entityType>person</entityType> <nameEntry> <part>Savage, C. R. (Charles Roscoe), 1832-1909</part> <authorizedForm>LC</authorizedForm> <authorizedForm>utsu</authorizedForm> </nameEntry> <nameEntry> <part>Savage, Charles Roscoe, 1832-1909</part> <alternativeForm>LC</alternativeForm> </nameEntry> </identity> <relations> <resourceRelation resourceRelationType="creatorOf" xlink:role="" xlink:actuate="onRequest" xlink:href="http://contentdm.lib.byu.edu/cdm/search/collection/Savage/" xlink:show="new" xlink:type="simple"> <relationEntry>Charles R Savage</relationEntry> </relations> </cpfDescription> </eac-cpf> Phase 1: metadata wrangling (data submitted) Phase 1: metadata wrangling (data compiled) Phase 1: metadata wrangling (data deduplicated) Phase 1: metadata wrangling (data reconciled) Phase 1: metadata wrangling (data metrics) • Started with ~500,000 names • 1091 are single words • Deduplicated to 76,360 • 2400+ cross references • 7357 -- 2+ collections (9.6%) • 500+ PN are First Last • 1484 -- 2+ institutions (1.9%) Total names submitted • 271 -- 2+ states (0.35%) • • • 62,381 personal names 10,706 corporate bodies 3,273 unknown • • • • • • • • Brigham Young University - 30,535 University of Utah - 7533 Utah State University - 2067 Utah State Historical Society - 12,138 Utah State Archives - 3657 University of Nevada, Reno - 1277 Oregon Digital - 4170 University of Denver - 16,608 Phase 2: investigating tools • With help of programmers: •Sandbox EnvironmentVirtual Machines, typically 2 cores, 2gb memory, all using ubuntu 14.04 or 16.04, mysql, php, apache, nginx, tomcat, utilizing the git version control system Utah State Fair development: Utah Chapter of the American Institute of Architects: sheet 17 George Cannon Young Architecture https://collections.lib.utah.edu/details?id=865147 Phase 2: investigating tools (evaluation matrix) https://goo.gl/wMQRbY Phase 2: investigating tools What Metadata People care about: Workflow issues: Discovery Issues: ● Batch Import Support ● Data models supported ● Batch editing of terms ● Data visualization ● Collaborative Workflows ● SPARQL Endpoint ● Local URI support ● LOD publishing ● Advanced search capabilities ● Browse capabilities Phase 2: investigating tools What digital infrastructure/programmers care about: ● Open Source Software Development Support - full community or just one person's project ● Ease of Installation for DID ● Ongoing support considerations from DID ● API availability ● Technical Support Requirements ● Software type (backend, middleware, complete solution) Phase 2: investigating tools xEAC ThManager RAMP Opaque Namespace Controlled Vocabulary Manager redis Vitro Phase 2: investigating tools xEAC RAMP (tools ruled out) Phase 2: investigating tools (starting tools) Phase 2: investigating tools (tools to install) Phase 2: investigating tools (need to look at) ThManager Opaque Namespace Controlled Vocabulary Manager redis Vitro Phase 3: pilot implementation • Perform full evaluation of selected software - harvest, standardize, reconcile, and import controlled vocabulary information into the software of choice and make data available as LOD • Hire and train student assistant to facilitate data entry, vocabulary reconciliation and enrichment, research, vocabulary maintenance, and assessment tasks. • Enrich data with relationships and collections holding information. • Explore the possibility of setting up an OpenRefine reconciliation service for the vocabulary • Develop and revise collaborative workflows Phase 4: assessment (workflows) • Managing vocabulary • Reconciliation issues • Many partners on CONTENTdm • Need external system not tied to specific digital library software/infrastructure • Explore impact of WNAF on metadata creators and users • Training opportunities Block Prints of Lennox and Catherine Tierney Private Art Collection [002]: Geisha under Flowering Tree Lennox and Catherine Tierney Photograph Collection https://collections.lib.utah.edu/details?id=329534 Phase 4: assessment (statistics) • Capture and assess data on the percentage of names not in a national authority file, the number of names unique to one institution, and number of relationships we are able to express with the vocabulary. 8. Navaho man and child. 1913 Dr. Elliott, photo Utah American Indian Digital Archive https://collections.lib.utah.edu/details?id=390032 Phase 4: assessment (statistics) Search results in December 2016 Name Creator Subject MWDL DPLA MWDL DPLA Savage, C. R. 2022 2226 44 50 Savage, C. R. (Charles Roscoe) 1830 2023 43 44 Savage, C. R. (Charles Roscoe), 1832-1909 1708 1901 43 43 Savage, C. R., 1832-1909 1708 1901 43 43 Savage, Charles 1936 2165 60 126 Savage, Charles R. 1932 2128 58 65 Savage, Charles R., 1832-1909 1708 1902 43 43 Savage, Charles Roscoe 1833 2034 44 53 Savage, Charles Roscoe, 1832-1909 1708 1904 43 43 Savage, Charles, 1832-1909 1708 1905 43 43 Savage, Chas. R. 1 1 0 1 Savage, Chas. R., 1832-1909 0 0 0 0 Phase 4: assessment (expanding use) • Develop a plan for expanding the controlled vocabulary to more institutions. • Consider what would be needed to move into full production for the project. Thomas Jefferson O'Brien journal commencing Feb. 6, 1895 https://collections.lib.utah.edu/details?id=1042220#t_1042220 Further reading and acknowledgements Full Grant Narrative https://www.imls.gov/grants/awarded/LG-72-16-0002-16 Project Webpage https://sites.google.com/site/westernnameauthorityfile/home This project was made possible in part by the Institute of Museum and Library Services LG-7216-0002-16.