home | prev | next

Configuring for OAI-PMH-based Search

Some Background on OAI-PMH-based Search

In the case of OAI-PMH-based search, administrators must configure a remote system as well as the current system before OAI-PMH-based search is possible.

Thus, this configuration requires at least two large steps:

1. Configure remote system (the system which will be searched).
2. Configure local system (the system initiating a search on one or more remote systems).

OAI-PMH-based search is based upon a protocol called OAI-PMH.

OAI-PMH stands for "Open Archives Initiative Protocol for Metadata Harvesting". More information about it can be found here.

This protocol supports data search and exchange which uses a small API (having 6 functions) and a common XML schema for data-sharing. The protocol presumes that each system participating in a network of OAI-PMH data sharing systems supports the ability to perform these functions on demand.

These functions give participating systems a way to ask another system to:

1. identify - provide descriptive information about what data they provide, what abilities they have, and what general kinds of information they contain
2. get (i.e, obtain) an individual metadata record from a given data-providing system
3. list identifiers - get abbreviated (header) information for metadata records of a data-providing system
4. list metadata formats - obtain a list of available/supported metadata formats (XML schemas) from a data-providing system
5. list records - obtain (“harvest”) a list of metadata records (XML files) from a data-providing system
6. list sets – obtain the set structure (types of information) available from a data-providing system

The system implements these functions and supports the OAI-PMH schema.

• Thus, it is able to play the role of a data provider (OAI-PMH server) or of a data harvester (OAI-PMH client) in a network of OAI-PMH-compliant data sharing systems.

This type of data sharing and searching is used most typically by registry systems in order to discover, share, and search data located at other OAI-PMH-aware systems.

Comparison of OAI-PMH and Federated Search Capabilities

When a given user initiates an OAI-PMH-based search on the local (current) system, similar to the federated search scenario, the local (current) system runs OAI-PMH-specific search processes across its registered OAI-PMH data providers (and they, likewise, propagate these searches to their providers, respectively).

The OAI-PMH-based search differs from federated search in several important ways:

1. It is based around the OAI-PMH protocol, API, and schema.
2. Its searches are always local.

That is, OAI-PMH:

• always performs the search on the local (current) system
• rather than on a remote system.

Whereas, federated searches:

• are always performed on the remote system
• rather than on the local (current) system.

OAI-PMH operates by, first locating data-providers that have information it needs and then copying that information locally from the remote system back to the local (current) system. Then, secondly, applying local search operations (e.g., search by example, keyword, etc.) to the locally-copied OAI-PMH information.

In summary:

• The OAI-PMH harvesting process: This process of “copying” the informational records from a remote system to a local system in order to facilitate search, is called “harvesting”. Harvesting is the foundation upon which registry systems share data, search data, and keep shared data up-to-date across a network of such data sharing systems.
• The federated-search remote querying process: In contrast, federated search operates by always asking remote search systems to perform search operations (e.g., search by example, keyword, etc.) remotely on their own systems, not needing to copy their results back to the local system as does OAI-PMH.

Configure Remote System (Data Provider/Server) for OAI-PMH-based Search

The steps below describe what must be done to configure a network of two-or-more OAI-PMH-compliant systems in order to support OAI-PMH-based searches. As with federated search configurations, only administrators are permitted to create and edit OAI-PMH-based search configurations.

1. Login to the remote system using an administrative account.
2. Navigate to the OAI-PMH Data Provider Identity configuration page:
   • On the top navigation bar, click the drop-down-menu (whose title is the user’s username) -> Administration menu-item

3. Support the various commands/functions that must be implemented by various OAI-PMH-Data Providers:

   1. Support Identify command

      • Click "Identity" (on left panel) -> "Edit" button.
      • This leads to an "Edit Curator OAI-PMH Server" input form.
      • Enter the following information:
      • Name: set to a unique name (e.g., “Curator OAI-PMH Server”)
      • Repository Identifier: set to a unique name (e.g., “server-x’)
      • Enable Harvesting: set to default (default=checked = True)
      • Click the "Save" button at the bottom of the page.

   2. Support List Metadata Formats command

      • Click "Metadata Formats" (on left panel)
      • Under the "Templates as Metadata Formats" add one or more templates which will be used to support data sharing under OAI-PMH.
      • Click the "Add" button. This leads to an input form for adding a metadata format template.
      • Enter the following information:
        • Metadata Prefix: (e.g., “oai_dc”)
        • Template: enter a template name
      • Click the "Add" button
        • NOTES:
          • Templates may only be listed/added if the templates have already been entered or uploaded somehow into the system.
          • If no templates exist yet in the system, add at least one template to the system before performing this step.

   3. Support List Sets command

      One is not required to specify sets in order for an OAI-PMH data search to become possible. Indeed, one may specify and perform OAI-PMH-based searches without specifying any sets of templates for OAI-PMH at all.

      • To provide OAI-PMH Data Harvesters (clients) with a way of organizing the different templates available in a given Data Provider:
        • Click "Sets" (on left panel) -> "Add Sets" button, which leads to an "Add Sets" input form for adding set information.
          • Enter the following information:
            • Set spec
            • Set name
            • Templates
            • Description
          • Click the "Add" button
          • NOTE: The remote system is now configured to behave as a data provider in OAI-PMH-based interactions with systems configured to behave as data harvesters.

   4. The administrator for the remote system configuration must share the following information with the administrator for the local (current) system:

      • Remote system IP-address/URL

The remote system is now configured.

Configure Local System (Data Harvester/Client) for OAI-PMH-based Search

1. Login to the local (current) system using an administrative account.
2. Obtain information from remote system administrator for the remote search system configuration: Remote system IP-address/URL
3. Navigate to the Add New Data Provider page:
   • On the top navigation bar, click the drop-down-menu (whose title is the user’s username) -> Administration menu-item -> Data Providers link, which leads to the "Data Providers" page.
   • Click "Add" button to open the "Add New Data Provider" input form.
   • Enter information for the following fields in the form:
     • Enter provider URL – enter remote system (data provider) ip-address/URL
       • i.e., http://127.0.0.1/oai_pmh/server/
     • Harvest Rate (seconds) – (default = 60)
     • Enable automatic harvesting – (default = checked = True)
   • Click the "Add data provider" button at the bottom of the page, which updates the OAI-PMH data providers list.
4. Check all of the registered data providers to ensure they are able to provide data by clicking the "Check All" button.
5. Update all of the registered data providers to ensure they are configured and ready to support OAI-PMH-based searches by clicking on the "Update All" button.
6. Manually force harvesting (local data copying) from all registered data providers by clicking the "Harvest All" button.

The local (current) system is now configured.

If both the remote and local systems are configured, then the users of the local (current) system may now perform OAI-PMH-based searches on this set of registered data providers.

• Over time, an administrator may register many such remote systems for carrying out searches initiated by the local (current) system’s users.

  • Thus, when users initiate searches from this local system, they can be distributed across all such remote search systems (and, likewise, across all of their, respective, remote search systems as well).