CDI: How to contribute?

 

The CDI system connects the SeaDataNet portal and the databases of the SeaDataNet distributed data centres. It enables that registered users can search for data sets, submit requests for data sets and if ok, can download data sets from the distributed data centres via a unique interface at the SeaDataNet portal. In practice, the user is downloading data sets directly from the relevant data centres, but all communication is facilitated through the SeaDataNet portal.

Governance

The maintenance of the Common Data Index (CDI) system is undertaken by the distributed data centres that each are connected to the SeaDataNet infrastructure and each maintain their CDI references to the data sets, that they manage. For the connection they manage and operate the local components of the CDI system for giving access to their data sets in communication with the CDI portal service and the users. The National Oceanographic Data Centre / Marine Data Centre, belonging to the SeaDataNet partnership, can give guidance and assistance to marine data centres in their country for installing and configuring the CDI connection and for generating and submitting CDI entries. The pan-European CDI system is managed and operated by the SeaDataNet partner MARIS, who also runs a CDI-support desk together with IFREMER to give support.

 Formats

For the CDI, a content model has been defined, based upon the ISO 19115 content model. Recently, considerable effort has been applied in establishing the XML coding of CDI on the basis of ISO 19139  and making it fully INSPIRE compliant. Detailed information about the formats and XML schema's can be found here.

Maintenance modality

DOWNLOAD the USER MANUAL compiling the CDI metadata, coupling table and associated data

The CDI directory is maintained by an XML export from the connected data centres to the pan-European directory:

  •  Using the MIKADO software, provided by SeaDataNet, to generate new and updated CDI XML files from locally maintained databases

XML files, generated using the latest MIKADO software, will be valid and should parse to the associated Schema. However partners not making use of MIKADO but generating XML entries for CDI should perform a validation before they can prepare and submit regular contributions. The Schema for CDI includes Schematron rules which allow to validate both the Syntax and the Semantics of CDI XML files, using an XML editor (e.g. Oxygene, XML Spy, ..) and the related Schema, which can be found at the SeaDataNet portal in the Standards and Formats section. If you are online, the Schema should be found by the editor automatically at the SeaDataNet namespace.

Including CSR references in CDI entries

It is encouraged to include Cruise Summary Report (CSR) references in existing and new CDIs in order to establish a direct relation between the CSR and CDI resources of SeaDataNet. The document bellow gives guidance on how to do that in practice in an efficient way:

 System components and configuration

The CDI system consists of a number of components, centrally at the portal and locally at each connected data centre. The central components are:

  • CDI Data Discovery and Access service: for searching and browsing of metadata of data sets and requesting access to data sets via a shopping basket; operated at the portal, while the CDI entries are imported from the XML CDI file deliveries by the data centres (see above)
  • Central User Register: contains details of users, their organisations and addresses, Id-Passwords, and their SeaDataNet roles; operated at the portal
  • Shopping Basket: part of the user interface for preparing a user request of multiple data sets from multiple data centres in one go, handling login validation of users, and routing requests to the Request Status Manager; operated at the portal
  •  Request Status Manager (RSM): for processing and administration of all requests and data deliveries (downloads); for users to handle the communication with data centres; for data centres to oversee all transactions; operated at the portal

For connecting to the CDI system there are 2 modalities possible for data centres, both requiring local system arrangements:

  • Connecting as a full data centre: this means that the processing of data set requests is done in an automatic way, for as far as possible. This requires that a data centre installs locally a java component ‘ Download Manager (DM), that handles all communication between the data centre system and the Request Status Manager at the portal and that takes care that requested files are made ready for downloading by users (if OK) via their personal download pages at the data centre.
  • Alternative is not to install the Download Manager, but to process all data set requests as registered in the Request Status Manager by data centre staff. This is the interim solution. It is intended as interim, because it is strived that all data centres become fully connected  data centres.

The data sets might be locally managed as files in a file management system, possibly supported by a local metadatabase, or managed in a relational database management system. It is required that data sets are delivered to users via the CDI system in standard SeaDataNet Data Transport Formats. This implies that a data centre, in case of a file management system configuration, must arrange that the data files are also available in the SeaDataNet formats. This might require pre-processing via a conversion routine. SeaDataNet provides useful software tools for that purpose:

  •  NEMO software for converting from any kind of ASCII format to the SeaDataNet ODV4 ASCII format and new SeaDataNet NetCDF (CF) format
  •  OCTOPUS software for splitting and conversion of SeaDataNet files

In case of a relational database management system it might not be needed to arrange pre-processed files in the SeaDataNet formats, because the Download Manager software includes functionality to generate the requested files in the SeaDataNet data formats. However in case of an Interim solution and a relational dbms, the data centre itself has to write software to arrange the required output.

In both situations, full and Interim, a number of configuration parameters have to be agreed and set at the system of the data centre and at the SeaDataNet portal. Therefore it is recommended that new data centres contact the CDI system coordinator MARIS  for detailed instructions and documentation.  MARIS will then provide guidance and validate the initial process of getting connected.