Planetary Data System

DDICT 4.3 User’s Guide (UG)

 

 

 

 

 

 

Draft Release Version 1

May 1, 2004

 

 

 

Prepared by:  Dale Schultz

 

Document Custodian:  Valerie Henderson                                          

 

 

 

 

 

Approved by:

 

 

________________________________________________

Laverne Hall

PDS Project Manager

 

 

 

 

 

 

 

 

 

 

 

 

 


Jet Propulsion Laboratory

Pasadena, California

 

 


CHANGE LOG

                       

 

 

Revision

Date

Description

Author

Start Draft

April, 2004

Initial draft

D. Schultz

Release 1

 

Draft Release 1 – Edited all

V. Henderson

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 


Table of Contents

1.0       Introduction.................................................................................................. 1

1.1       Software Identification....................................................................... 1

1.2       Purpose......................................................................................................... 1

1.3       Document Scope...................................................................................... 1

1.4       Audience........................................................................................................ 1

1.5       Overview....................................................................................................... 1

1.6       Applicable Documents......................................................................... 2

2.0       Getting Started........................................................................................... 3

2.1       PDS DDICT Setup Process.................................................................... 3

2.2       Installing DDICT......................................................................................... 3

2.3       Starting DDICT............................................................................................ 4

2.4       Getting Help................................................................................................ 4

2.5       Using DDICT – Examples......................................................................... 4

2.5.1 Extracting Keyword Definitions From One File............................................ 5

2.5.2 Using DDICT to Verify a Group of Files........................................................ 8

3.0       What DDICT does........................................................................................... 9

3.1       Function........................................................................................................ 9

3.2       Caveats and Limitations....................................................................... 9

4.0       Command Reference................................................................................. 9

4.1       -a  Enable Aliasing................................................................................... 9

4.2       -d  Enable Data Dictionary................................................................ 10

4.3       -f  Specify Label Files(s).................................................................... 10

4.4       -ivf <file name> Specifies name of file containing a list of file extensions to skip................................................................................................................. 11

4.5       -na   Disable Aliasing............................................................................ 11

4.6       -nv  Disable Verbose Output........................................................... 11

4.7       -r  Specify Report File Name............................................................ 11

5.0       DDICT Messages.......................................................................................... 12

Appendices................................................................................................................ 13

Appendix A - Release Notes............................................................................. 13

Version 4.2 Release notes............................................................................ 13

Version 4.2 Known Bugs.................................................................................. 13

Appendix B - Acronyms....................................................................................... 13

 

 

 

 

 

 


1.0           Introduction

 

1.1             Software Identification

 

Project name:                        Planetary Data System (PDS)

Program set name:              PDS DDICT

Abbreviation:                         ddict

Version number:                    4.2

Platforms supported:            Windows, Solaris, Linux

 

 

1.2             Purpose

This document provides information on the use of the PDS DDICT software set. This software will extract the data dictionary definition for every keyword used in a specified PDS label file, a specified list of PDS label Files, all of the labels in a directory, or all of the files on an entire volume.   DDICT will also list those keywords that are not in the data dictionary.

 

1.3             Document Scope  

This document provides:

 

 

1.4             Audience

This document is intended for those who are responsible for ensuring that labels submitted to the PDS conform to PDS standards. This document assumes familiarity with the PDS data preparation process and label design requirements. The data preparation process is described in the PDS Data Preparation Workbook (JPL Document D-7669, Part 1); label file requirements are described in full in Chapter 5 of the PDS Standards Reference (http://pds.jpl.nasa.gov/documents/sr/Chapter05.pdf ). See Section 1.6.

 

 

 

1.5             Overview

 

PDS labels are written in the Object Description Language (ODL). ODL consists of a series of lines of the form "keyword = value", with certain keywords (for example, OBJECT) being used to delimit named groups of keywords within the label. For a description of the PDS implementation of ODL, see Chapter 12 of the PDS Standards Reference.(http://pds.jpl.nasa.gov/documents/sr/Chapter12.pdf ).

 

The keywords and objects are themselves defined in the Planetary Science Data Dictionary (PSDD). The Dictionary exists as a Word file for human readers, as a text file for use by the software, and is available via an online look up tool at (http://pds.jpl.nasa.gov/tools/data_dictionary_lookup.cfm).

 

 

 

1.6             Applicable Documents

 

 

·        Planetary Science Data Dictionary Document, August 28, 2002, Planetary Data System, JPL D-7116, Rev. E. (http://pds.jpl.nasa.gov/documents/psdd/psdd.pdf )

 

·        Planetary Data System Data Preparation Workbook,  JPL D-7669, Part 1. (http://pds.jpl.nasa.gov/documents/dpw/index.html)


 

2.0           Getting Started

2.1             PDS DDICT Setup Process

The PDS DDICT software set consists of the following files:

 

Windows

ddict.exe - executable

 

SOLARIS AND LINUX

ddict – executable

 

PDSDD.FULL and MAKE_INDEX are included in the ddict distribution package. PDSDD.FULL will change often and should be downloaded frequently from:  http://pds.jpl.nasa.gov/tools/software_download.cfm  as can all of the central node validation software tools.  Each time PDSDD.FULL is downloaded it is important to run MAKE_INDEX again to create a new index.  Once at that url select “for data producers”.  When the new url opens scroll down the Software Id/Version ID window until the desired software tool is highlighted.  Click on the selected tool once and select the submit button.  When the new information is displayed click once on the appropriate download.

 

PDSDD.FULL can be obtained by scrolling through the Software Id/Version ID window to pdsdd(CAT1R40):PDS Toolbox Data Dictionary Version *****(the version number will be changed from time to time and is not shown here).  Select MULTIPLE from the download list.

 

MAKE_INDEX can be obtained by scrolling through the Software Id/Version ID window to make_index, selecting it, and then selecting the submit button.

Click once on the appropriate download.

 

2.2             Installing DDICT

Download the zip or tar file for ddict, make_index, and data dictionary file to the hard drive.  Ddict is a command line program and must be executed from a DOS window or from a unix command line prompt.

 

Use make_index to make pdsdd.idx by typing:

 

make_index pdsdd.full

 

 

2.3             Starting DDICT

The PDS DDICT program is a command line tool and will need to be run from a command prompt in Windows or from a terminal/console in unix.

 

To start DDICT from the same directory in which ddict and default Data Dictionary files are located, enter the command:

 

ddict –d pdsdd.idx -f <input-file> -r <output-file>

 

where <input-file> is the name of the label file to verify, and <output-file> is the name of the file of the verification report.  If the Verifier cannot locate the Data Dictionary files it needs, it will generate a message.  Pdsdd.idx is the name of the data dictionary index that was created when running make_index.

 

DDICT may also be started from a different directory that the one in which DDICT and Data Dictionary files are located.  For example, on a Unix system, if DDICT and dictionary files are located in directory /usr/local/pds, enter the command:

 

/usr/local/pds/ddict –d pdsdd.idx -f <input-file> -r <report-file> -d                    

                                                                                                 usr/local/pds/pdsdd.idx

 

The -d option tells DDICT the path (if different than the current directory) and name of the PDS Data Dictionary index file. 

 

2.4             Getting Help

To get a brief description of the command line parameters, at the command prompt type:

 

            ddict

 

The parameters are also explained in detail in the COMMAND REFERENCE section of this document.

 

For further help or information, contact the PDS Operator at pds_operator@jpl.nasa.gov

 

 

2.5             Using DDICT – Examples

The following examples will help in using DDICT immediately.

 

While reviewing the following examples, refer to the “Command Reference” chapter, which explains the command line options available for DDICT, and the “DDICT Messages” chapter, which explains the error and warning messages generated by DDICT.

 

The following examples assume that both DDICT and the Data Dictionary reside in the current directory.

 

2.5.1       Extracting Keyword Definitions From One File

 

Assume that the following label file, EXAMPLE1 is to be processed:

 

                                                                             

                                                                              

                                                                             

/* File Format and Length */                                                 

RECORD_TYPE = FIXED_LENGTH                                                   

RECORD_BYTES = 1000                                                          

FILE_RECORDS = 806                                                           

                                                                             

/* Pointers to Data Objects */                                               

^IMAGE_HEADER = ("EXAMPLE.IMG",1)                                            

                                                                             

/* Description/Catalog Keywords */                                            

SPACECRAFT_NAME = "GALILEO ORBITER"                                          

INSTRUMENT_NAME = SOLID_STATE_IMAGING                                        

                                                                              

/* Image Object */                                                           

OBJECT = IMAGE                                                               

LINES = 800                                                                  

LINE_SAMPLES = 800                                                           

SAMPLE_BITS = 8                                                              

SAMPLE_TYPE = UNSIGNED_INTEGER                                               

INVALID = "N/A"                                                               

END_OBJECT                                                                   

END                                                                          

 

 

Enter the command:

 

ddict –d pdsdd.idx -f example1.lbl -r myreport.rpt

 

This tells ddict: “Extract all keywords from  ‘example1.lbl’, look up the keyword definitions from pdsdd.idx, and write the report to file ‘myreport.rpt’.”  After this command completes, the “myreport.rpt” file will look as follows:

                                                           

_____________________________________________________________


 

************************************************************************

*                                                                      *

*               Planetary Data System Data Dictionary                  *

*                     Extractor Version 4.2                            *

*                                                                      *

*                      Sun May 02 16:16:28 2004                        *

*                                                                      *

*       Data Dictionary Name: pdsdd.idx                                *

*    Data Dictionary Version: OPS                                      *

*  Data Dictionary Generated: October 20, 2003                         *

*                                                                      *

*                  Data Dictionary Validation is ON                    *

*                                                                      *

************************************************************************

 

------------------------------------------------------------------------

*****            The following keywords are not present            *****

*****                   in the Data Dictionary                     *****

------------------------------------------------------------------------

 

    WARNING: Not in Data Dictionary -- INVALID

 

------------------------------------------------------------------------

*****        The following set of keywords and definitions         *****

*****           were extracted from the Data Dictionary            *****

------------------------------------------------------------------------

 

FILE_RECORDS

     The file_records element indicates the number of physical

     file records, including both label records and data

     records.

     Note:  In the PDS the use of file_records along

     with other file-related data elements is fully described in

     the Standards Reference.

 

INSTRUMENT_NAME

     The instrument_name element provides the full name of an

     instrument.

     Note: that the associated instrument_id element

     provides an abbreviated name or acronym for the instrument.

     Example values:  FLUXGATE MAGNETOMETER, NEAR_INFRARED

     MAPPING SPECTROMETER.

 

LINES

     The lines element indicates the total number of data

     instances along the vertical axis of an image.

     Note:  In PDS label convention, the number of lines is

     stored in a 32-bit integer field.  The minimum value of 0

     indicates no data received.

 

LINE_SAMPLES

     The line_samples element indicates the total number of data

     instances along the horizontal axis of an image.

 

RECORD_BYTES

     The record_bytes element indicates the number of bytes in a physical

     file record, including record terminators and separators. When

     RECORD_BYTES describes a file with RECORD_TYPE = STREAM

     (e.g. a SPREADSHEET), its value is set to the length of the longest

     record in the file.

     

     Note:  In the PDS, the use of record_bytes, along with other

     file-related data elements is fully described in the Standards Reference.

 

RECORD_TYPE

     The record_type element indicates the record format of a

     file.

     Note:  In the PDS, when record_type is used in a

     detached label file it always describes its corresponding

     detached data file, not the label file itself.

     The use of record_type along with other file-related data

     elements is fully described in the PDS Standards Reference.

 

SAMPLE_BITS

     The sample_bits element indicates the stored number of

     bits, or units of binary information, contained in a

     line_sample value.

 

SAMPLE_TYPE

     The sample_type element indicates the data storage

     representation of sample value.

 

SPACECRAFT_NAME

     The spacecraft_name element provides the full, unabbreviated

     name of a spacecraft.  See also: spacecraft_id,

     instrument_host_id.

 

************************************************************************

*                                                                      *

*        This is the end of the Data Dictionary Extractor Report       *

*                                                                      *

*                      Sun May 02 16:16:28 2004                        *

*                                                                      *

************************************************************************

 

                        Report File “myreport.rpt “

 

_____________________________________________________________

 

The report file is divided into 2 sections:

 

·        Section 1 is a list of all keywords not found in the data dictionary.

 

·        Section 2 is a list of all keywords that were found in the data dictionary and the definition for the keyword contained in the data dictionary.

 

2.5.2       Using DDICT to Verify a Group of Files

 

If there are several label files in the directory, (e.g. “example1.lbl”, “example2.lbl”, and “example3.lbl”), enter the command:

 

            ddict -f *.lbl -r myreport.rpt

 

This tells DDICT: “Process all the files in the directory that have extension ‘.lbl’ and write all the results to file ‘myreport.rpt’”.  The report file will look the same as it did in the first example.  All keywords will appear in alphabetical order in whichever section is appropriate.  The keywords in the report will not be separated by label, simply by not found and found.

 

 


 

3.0           What DDICT does

 

 

 

3.1             Function

Ddict parses the odlc label into a list containing each object and all of the keywords in each object.  It then simply goes through the list and looks up each keyword in the data dictionary.  Those keywords that are not found in the data dictionary are listed in the first section of the report.  Keywords that are found in the data dictionary are listed in the second part of the report along with the definition that was found in the data dictionary.

3.2             Caveats and Limitations

This tool performs no validation of the odl semantics.

 

4.0           Command Reference

4.1             -a  Enable Aliasing

Default: -na (no aliasing)

 

The -a command line option enables the aliasing feature of DDICT.  This option is useful when processing a label which was developed according to another Data Dictionary, or to identify out-of-date keywords and objects in the label.

 

When this option is turned on, DDICT will compare the object classes in the label with the list of object class aliases in the Data Dictionary, and replace them with their official names.  Object classes that were changed may be identified by examining the label hierarchy in the report file.  For example:

 

OBJECT = TABLE_STRUCTURE

 

will be replaced by

 

OBJECT = TABLE

 

if TABLE_STRUCTURE is defined to be an alias for object class TABLE in the Data Dictionary.

 

DDICT will then compare the keywords in the label with the list of keyword aliases in the Data Dictionary and replace them with their official names.

 

After aliasing is finished, DDICT will continue checking for key word definitions, using the official names instead of the aliases when it searches the Data Dictionary.

 

Note that only full class names will be replaced.  For example, EDR_TABLE_STRUCTURE will not become EDR_TABLE just because TABLE_STRUCTURE is defined as an alias for TABLE.  The same is true for the keyword replacement.  Remember that DDICT does not change the actual label file at all.  The changes to the label remain in effect only for the duration of DDICT’s run.

 

Enabling the aliasing option significantly slows processing, and so the default is

-na.

 

4.2             -d  Enable Data Dictionary

 

Default: -d pdsdd.idx

Off switch: -nd

 

The -d command line parameter is mandatory.  The information following the -d is the name of the index into the data dictionary being used.  Typically it will be pdsdd.idx which is the index for pdsdd.full.  The following is an example of this parameter:

 

ddict  -d pdsdd.idx –f mylabel.lbl –r myreport.txt

 

4.3             -f  Specify Label Files(s)

 

Default:  None.

Off Switch: None. Specify a label file for processing.

 

The -f command line option specifies one or more files to be processed.  DDICT will search all subdirectories of the pathname of the file specified in this option.

 

To specify just one filename:

 

ddict  -f mylabel.lbl . . .

 

Or a wildcard pattern:

 

ddict -f /home/joeuser/*.lbl . . .  (in unix -f /home/joeuser/”*.lbl”)

 

Or a series of filenames and / or wildcard patterns:

 

ddict -f /hme/joeuser/*.lbl *.fmt catalog.lbl

 

In the last case, all files with extension “.lbl” in the directory /home/joeuser, all files in the current directory with the extension “.fmt”, and the file “catalog.lbl” in the current directory will be processed.  The directory specifications and wildcards should be those expected by the particular operating system.

 

4.4             -ivf <file name> Specifies name of file containing a list of file extensions to skip

-ivf <file name>

 

Any file with an extension in this list will not be processed.  Some types of file extensions that might be included in this list are: c, h, txt, img. The list is a string of file extensions that are comma separated. The program is case sensitive.  The list may not contain blanks and can only be 200 characters in length.  Lower case characters can be included in the string.  A sample string is:

            C,H,TXT,IMG,c,h

This string of characters tells DDICT not to process any file having an extension of C (or c), H (or h), TXT, or IMG.  The string is saved in an ASCII file with a name of the choice.

 

4.5             -na   Disable Aliasing

Default: -na

 

This option disables alias substitution.  To turn on alias substitution use the -a option.

 

 

4.6             -nv  Disable Verbose Output

Default: This is the default

 

Verbose output may be invoked with the -v option.

 

4.7             -r  Specify Report File Name

 

The -r command parameter is mandatory, it is followed by the name of the report file to which DDICT will write its report.

 

ddict -r /home/joeuser/label.rpt ...

or

ddict –r c:\pds\testarea\label.rpt…

 

 

5.0           DDICT Messages

TBD


 

Appendices

Appendix A - Release Notes

Version 4.2 Release notes

 

 

Version 4.2 Known Bugs

The following command line arguments are no longer valid and should not be used even though the program will still display and allow them: -nd, -t, -nt, -p,

-np, -nv, -v.

PDSDD.FULL and MAKE_INDEX were not distributed as part of this release of DDICT.

 

Appendix B - Acronyms

TBD

Appendix C- Glossary

TBD