IDS Search

Brief Description

IDS Search is a real-time availability service for the CUNY Aleph Union Catalog. It utilizes z39.50 to discover CUNY bibliographic records via queries by Standard Number (ISBN, ISSN, LCCN, OCLC Number). If matching are found it then utilizes the Aleph REST API to discover the real-time status of items associated with those records and determines if they are available on the shelf, requestable by clics, or contain links to a electronic resources accessible to the users of a library.

Features

  • Tight integration with the CUNY Aleph System. Local cataloging practices for electronic resources and item statuses are taken into account when deciding upon the availability and requestability of resources.
  • Availability Queries are "scope" sensitive.
  • Provides links to full-text resources only when they are accessible to selected library's users. This is enforced regardless of the current search scope.
  • Supports "CLICS" requests in context. Users are directed to use CLICS only when requestable resources are available.
  • If a record is not available in full-text or clics-eligibile the user is presented with an OpenURL leading to their institutions ILLiad request system.

What set of data is each search scope run against?

Items presented to the user as matching records for queries issued by IDS search are based entirely on the Worldcat Database and the holdings libraries have reported to OCLC. Search results in all search scopes always based on Worldcat data. Here is the breakdown of what subset of Worldcat Holdings are searched by each of the search scopes presented to the user when using IDS Search.

  1. My Library = Queries only those records with OCLC holdings for the single OCLC symbol of the library (i.e. ZGM)
  2. CUNY_Libraries = OCLC Holdings attached to any CUNY library OCLC Symbol (includes the ZCY Central Office Symbol)
  3. Quick Delivery Libraries = OCLC Holdings attached to the OCLC for all active Project IDS Libraries
  4. Worldwide = Entire OCLC Worldcat Database

Note on CUNY OCLC Holdings: The CUNY OCLC holdings can be quite out of date when compared to the local Aleph system, particularly for periodical materials.

Note on Custom Search Scopes: It is possible to have another worldcat search scope defined for a given library. The Grad Center has one called "NYC Research Libraries" that includes NYPL, Columbia and NYU. Ask the search team if you need a custom search scope such as this.

Aleph System Availability Lookup Specifications

Availability Lookup Description

After a search is run IDS Search endeavors to find the availability of result items in the local library catalog. The data presented here is based on the data stored in the CUNY Aleph system rather than Worldcat. This is done through queries sent to the Aleph Z39.50 Server. Queries for availability are scope sensitive in the same fashion as the worldcat search scoping.

What set of data is examined in order to find matching Aleph records?

For all instances below if no match is found, the user is presented with a "Get It" button that directs an OpenURL representation of the resource based on metadata in the Worldcat Record to the Library's ILLiad System.

  1. My Library = Searches only the Library's Aleph Local Base (i.e. GRADCENTER or BARUCH). If a lookup of the local base fails to find a matching record, the query is than passed through the CUNY_Libraries lookup before being passed to the GetIt button.
  2. CUNY_Libraries = Searches the unmerged Aleph Catalog (CUN01). Note this is not the CUNY Union Catalog which will return only preferred records. A search of this base returns all matching record.
  3. Consortia = First tries the lookup for CUNY holdings from 2 before presenting a GetIt button. .
  4. Worldwide = First tries the lookup for CUNY availability from 2 before presenting GetIt button.

The CUNY Aleph System does have a direct numeric index to the OCLC numbers stored the MARC 035 currently defined yet the matching in the four steps remains challenging. In lieu of a direct link between a worldcat records and local Aleph records the system uses a search algorithm that utilize both OCLC Numbers and other standard numbers like ISBN, ISSN, and LCCN numbers to match a resource in worldcat with it's local equivalents. The algorithm works in the following fashion:

  1. If either any ISBN or ISSN numbers are present the Aleph system is queried for any matching records.
  2. If no match is found for 1, any LCCN numbers presents are queried against the Aleph system.
  3. If no match is found in either 1 or 2 any OCLC numbers attached to the record in worldcat are queried against the aleph system in the following fashion. Though no direct numeric index exists many OCLC numbers are stored in the 035 field of CUNY marc records using a variety of prefixes prepended to the actual number. Here are the most common prefixes:

ocn
ocm
OCoLC
(OCoLC)
(OCoLC)ocm

So a hypothetical oclc number "12345678" is actually looked up with the following query:
12345678 OR ocn12345678 OR ocm12345678 OR OCoLC12345678 OR (OCoLC)12345678 OR (OCoLC)ocm12345678

Note on Periodicals: Again, the most problematic records to match successfully are periodicals. Many periodical records in Worldcat seem to lack ISSN numbers and many Aleph bibliographic periodical records lack any OCLC number representations.

Availability Processing

After matching records are found the local Aleph Bibliographic ID numbers (i.e., CUN01000825008) are extracted from each matched record's 001 Field. Example record sent back by z39.50 server with 001 field populated by Bibliographic ID:

https://libadmin.cuny.edu:5661/CUN01?version=1.1&operation=searchRetrieve&query=rec.id=000825008&startRecord=1&maximumRecords=1

These Bib Record ID numbers are then used to lookup the current status of all matching records using the Aleph REST API server.

Aleph REST API

The REST API server is a feature Aleph uses to expose holdings and item level data attached to specific bibliographic records to discovery services such as Primo, Summon, EDS. Here are examples of the types of data it can send back (each response is made possible by supplying a valid bibliographic ID. in this case CUN01000825008):

IDS search uses the second format above (Full Item Records) to determine the following display items/features which are displayed to the user after matching bibliographic records have been found. The system prefers locally own library records over those held by other CUNY Libraries. If any "available" local holdings or a local electronic resource is found these are always preferred in the IDS search display to the user. This notion is also scope sensitive. If a user is searching in the CUNY Libraries Scope and locally owned and available items are found they will be presented with the availability of those items first and not be shown a "Request From CUNY Libraries" button that leads to a CLICS request.

What determines an "available" local holding?

  • If a matched record is part of one the sub-libraries attached to a given library (i.e. GC001) and has item status information (i.e. Look on Shelf) that indicate it is available for loan the system deems that that their are locally available. If a locally owned title is found to be present in any search scope than no "Request It From CUNY" or "GetIt" buttons will be shown.

Note on "View Locations": All owned items regardless of item status are displayed when the "View Locations" button are clicked. View locations only shows locally owned items in the "my_library" scope. All CUNY holdings are displayed in other scopes.

What is a CLICS-Requestable Record?

The "Request It From CUNY" button is found when other CUNY Libraries than the active library have an item available for loan. Locally owned titles are not considered as CLICs eligible resources. The following Aleph Loan Types are considered to be "CLICS-Eligible"

 - "Regular Loan"

If a given title has item status (i.e. Look on Shelf) that indicates it is available and has one of the loan types displayed above the resource is considered "CLICs-eligible" and the "Request It From CUNY" button appears directed the user to the Aleph patron request screen. Example CLICS-eligible title

Where do the Full-Text @ My Library Links come from?

These links are extracted from the marc 856 fields stored in the local CUNY system. If a matching record has Loan Type of any of the following:

- 'eBook'

 - 'Online'

 - 'eResource'

 - 'Web Resource'

It is considered to be an "electronic" resource. If the matching record is part of the "AL001" Cuny-wide sub-library or any of the sub-libraries attached to a given library the record is considered to be a "full-text" resource for the users of the currently active library in IDS Search. The full-text links are prefaced by the active library's EZproxy prefix to support remote access.

The "full-text" links also provide special handling for ebrary titles. Links to full-text materials from ebrary avoid using the proxy prefix because the "EBrary SSO" set-up that already invokes the proxy where appropriate. There is also a special filter to make sure that AL001 records owned by CUNY central provide the correct ebrary channel string the active library.

This full-text linking feature was implemented using the 856 Fields because they provide better representation of eBooks available to CUNY than was has been loaded in fits and starts into the SFX system.

Outstanding Issues

  1. OCLC Reclamation Project - Needs to be completed to truly have an accurate profile of CUNY holdings in worldcat
  2. Availability display performance for periodical records with hundreds attached items or that are owned by a majority of CUNY libraries is an issue. This is something that any disovery system (Summon, Primo, etc.) is going to suffer from when viewing periodicals with hundreds or thousands of item records. The duplication of bibliographic records across many libraries is a problem unique to CUNY but puts a similar strain on the system and will likewise strain discovery systems.

Addendum (July 2012)

A recent issue was identified where a search for a specific item was not found under a ‘CUNY Libraries’ scope yet when one searched using the ‘Quick Delivery Libraries’ scope it showed a CUNY holding for that title.

These displays highlight the issue of OCLC holdings being out of sync with the local Aleph Catalog. If a resource does not have an OCLC holding attached to the OCLC symbol for the local library it will not show up in the search results sent back by OCLC. The other scopes find matching records since OCLC holdings for those titles exist somewhere in the scope of OCLC symbols searched by that scope.

The reason you see “CUNY Availability” for these titles in the non CUNY scopes is that the availability service tries to match every single result against the local library system first (irregardless of scope) so it is finding matches for records and displaying the results of the availability query to the user. The availability service follows the following algorithm to find a match:

1. Search by ISBN or ISSN
2. If neither is present try LCCN
3. If none of those is available it tries an expanded OCLC number search that combines the various prefix strings used in the Aleph Catalog 035 field plus the number.

There is going to be some level of ambiguity in these results until the OCLC Holdings are synchronized. If libraries wish to troubleshoot these issues here is what I’d suggest.

1. Look in worldcat to see if there are no holdings listed for a given title.
2. Look in Aleph to see what holdings exist. If they are present take note of any standard numbers or OCLC numbers attached to the record.
3. Look at the record in worldcat to see if any standard numbers or the oclc numbers attached to the record have a match in the records found in the previous step.

Problems are going to be for some specific examples yet each should be considered independently if one wishes to identify a more general pattern.

The alternative of not invoking the CUNY availability service when a user is outside of the “my_library” and “cuny_libraries” scopes would open the possibility of users submitting ILL requests for books CUNY really owns or provides full-text access to.” This would seem of less practical use for patrons.