Guidelines for Archiving

Golden Rule of Archiving: Consult with your PDS Discipline Node early and often.
Communication through the archiving process is a critical step in the successful creation of a usable, discoverable, accurate, and valid PDS archive. For more details on why this is important, see this whitepaper on why you should start the archiving process early.

The following section provides an overview of some of required tasks and responsibilities for archiving data in the PDS.

For more details on archiving in the PDS, see the appropriate Proposer’s Archive Guide:

Additionally, the PDS Discipline Node you are archiving with can be contacted for additional assistance.

Overview

The purpose of this section is to provide Principal Investigators of funded NASA ROSES proposals a set of PDS archiving milestones to reference when preparing annual progress reports for NASA Program Officers. When reporting on PDS archiving progress, Data Providers should indicate which of the milestones listed below have been reached, including additional details of accomplishments related to the milestone(s). Note that the order of the activities listed for each milestone is flexible and some activities will likely occur concurrently.

Important Reminder: While all projects vary in time and complexity, it is vital to start the PDS data archiving process early and to distribute the effort across the length of the project, even if you expect to acquire or generate data through the entire duration of the award period. Consult with your PDS Discipline Node early and often. Communication throughout the archiving process is a critical part of creating a usable, discoverable, accurate, and valid PDS archive.

Additional data proposer and provider help materials can be found at Information for Data Proposers.

PDS Data Archiving Milestones

Indicated broad timeframes are notional, assuming a three-year proposal, and will vary depending on the project.

  1. Initial PDS Contact (pre-proposal and upon award funding):
    1. Before proposing, the Data Provider makes initial contact with the relevant PDS Node and supplies information about the data to be archived, including types and volume of data, processing levels, supporting products, and estimated schedule. This may be done by providing the Data Management Plan, if relevant. Instructions for requesting a letter of support can be found on the PDS website (https://pds.nasa.gov/home/proposers/). PDS Node contact information can also be found on the PDS website (https://pds.nasa.gov/contact/contact.shtml).
    2. PDS Node determines if the data plan is appropriate for the PDS Node and, if so, provides a written letter of support for the Data Provider to use in the proposal response(s).
    3. If funded, the Data Provider makes contact with the relevant PDS Node to inform them of funding and to begin scheduling the archiving effort.
  2. Archive Schedule and Design (Year 1):
    1. Data Provider works with the PDS Node to determine a suitable schedule for the archiving work, including setting specific milestone dates, and for regular communication.
    2. Data Provider works with the PDS Node to decide on the archive bundle hierarchy structure and file naming convention. The layout of the Bundle-Collection-Product structure allows the creation of bundle-specific Logical Identifiers (LIDs).
  3. Sample Products and Labels (Year 1 to Final Year):
    1. Data Provider reformats data products, if necessary, to meet PDS4 standards. To avoid having to reformat completed products, it is recommended that the Data Provider discuss data formatting plans with the PDS Node early in the project.
    2. Data Provider develops prototype XML labels for the data products and archive bundle(s) with assistance from the PDS Node. Note that each PDS Node varies in its approach to label and bundle development, so the Data Provider should be in frequent contact with the archiving node for proper guidance.
    3. PDS Node and Data Provider identify any necessary updates to PDS4 Local Data Dictionaries (LDDs), such as any updates to the imaging discipline dictionary or appropriate mission dictionary.
    4. PDS Node creates, manages, and shepherds appropriate LDD change requests through the PDS Data Design Working Group (DDWG).
    5. Data Provider submits sample products for review and feedback from the PDS Node. Multiple iterations may be necessary.
  4. Documentation and Bundle Creation and Validation (Year 2 to Final Year):
    1. Data Provider generates a user guide and other documentation to be included in the bundle. Documentation should be written by the knowledgeable science lead on the project for scientific accuracy and completeness. Note that it is often useful to start drafting documentation early in the archiving process to track important archive decisions and details.
    2. PDS reserves a Digital Object Identifier (DOI) for the bundle.
    3. Data Provider creates the bundle using the label templates along with the agreed upon structure.
    4. Data Provider submits the bundle to the PDS Node for internal review.
    5. Data Provider iterates with the PDS Node until the bundle can pass validation.
  5. Peer Review (Final Year or earlier):
    1. PDS Node solicits reviewers for the Peer Review with input from the Data Provider.
    2. PDS Node conducts the Peer Review, receives and collates reviewers' comments, and delivers the list of recommendations ("liens") to the Data Provider.
    3. Data Provider responds to each lien stating how it will be resolved.
    4. PDS Node and Data Provider determine a schedule for lien resolution.
  6. Lien Resolution and Certification (Final Year or earlier):
    1. Data Provider revises the bundle as specified in the response to the liens.
    2. If necessary, the PDS Node hosts a telecon with the Data Provider and reviewers to resolve any difficult issues.
    3. Once lien resolution is complete, reviewers are satisfied, and validation is complete, the bundle is accepted into the PDS and becomes PDS Certified Data. The bundle may be certified and cleared for immediate release even if a few liens remain to be resolved, as long as the liens are minor. Any residual liens must be resolved before being officially archived.
  7. Archive Complete and Released (Final Year or earlier):
    1. PDS submits the newly certified bundle to the PDS Central Registry.
    2. PDS creates a landing web page for distribution of the bundle.
    3. PDS advertises/announces the new bundle(s) (e.g., Planetary Exploration Newsletter, other newsletters, PDS data release notifications and pages, etc.).
Back to top

Detailed Archiving Checklist

  • Contact the applicable PDS Discipline Node to inform them of funding and begin scheduling your archiving effort.
  • Provide information about your data to PDS – Types of data, processing levels, supporting products.
  • Work with the PDS on developing XML Label Templates for your bundle(s).
  • Bundle hierarchy structure decided – Layout of the Bundle-Collection-Product structure allows creation of bundle-specific Logical Identifiers (LIDs).
  • Create the bundle – Use label templates together with decided structure to create your complete bundle.
  • Complete Documentation – Adequate documentation shall be included in the bundle.
  • Iterate with PDS until your bundle can pass validation.
  • Provide a list of potential reviewers to the relevant Node for your peer review.
  • PDS solicits reviewers for the Peer Review.
  • PDS conducts the Peer Review online and receives/collates reviews.
  • Concluding telecon presents results of the Peer Review to Data Provider(s) and determines schedule for any Lien Resolution.
  • Lien Resolution and Validation – Once lien resolution is complete, reviewers are satisfied and validation is complete the data are accepted into the archive.
  • Data Harvest and Registration – PDS pushes the newly certified bundle to the PDS Central Registry to be discoverable across the PDS.
  • PDS creates a landing webpage for distribution of the bundle.
  • PDS advertises/announces new bundles in PEN and/or other appropriate planetary science newsletters.

Download Checklist

Back to top

Getting Started with PDS4

The PDS recognizes that archiving can seem like a daunting process, disconnected from your science and required as some mythical last step in your award period. We hope that by breaking this archiving process down into manageable steps, to be done throughout your grant period, archiving can be a natural consequence of all the work you do.

In order to successfully complete an archive with the PDS, it is necessary to understand how the PDS archive works. Working knowledge of the archive itself – how data are organized, what kinds of metadata are useful, etc. – will allow you, the data provider, to more efficiently organize your efforts in producing viable archive products. Within this task, it is important to understand the workings of the PDS4 Archiving Standard. This will include all aspects of the structure of your archive – the basic hierarchy of the archive, creating the system of unique identifiers, utilizing context products to ‘build’ your reference system within your archive.

Communication will be key to your success. Early and frequent communication with the relevant PDS Discipline Node personnel can be the difference in understanding your responsibilities as a data provider and successful completion of the archiving process. The PDS Discipline Node personnel are always here to help in this process by facilitating your archiving effort and providing answers to any of your archiving questions.

PDS4 Training

In-person PDS4 Training opportunities exist at several yearly conferences, including AGU, LPSC, and DPS, among others. We are also working to develop online-training modules and tutorials you can do from home. See our PDS Training website for more details. In addition, contact your PDS Discipline Node representative for additional training opportunities.

PDS4 Documentation

For more detailed information on the PDS4 standard and appropriate documentation, please see the PDS4 Standards website at https://pds.nasa.gov/datastandards/documents/.

PDS4 Data Provider Tools

For software tools to help you build your PDS4 archive, head to the PDS Tool Registry to search for the applicable tool that meets your needs. If you are unsure of which tools to use, contact your PDS Discipline Node representative and they can help guide you in the right direction.

Back to top