PHOENIX MET/LIDAR EDR ARCHIVE VOLUME SOFTWARE INTERFACE SPECIFICATION Version 1.1 18 May 2007 Lyle F. Huber TABLE OF CONTENTS 1. Introduction 1 1.1. Purpose and Scope 1 1.2. Content Overview 1 1.3. Applicable Documents and Constraints 1 1.4. Relationships with Other Interfaces 1 2. Archive Volume Contents 2 2.1. Root Directory Contents 2 2.2. Data Directory Contents and Naming 2 2.3. Index Directory Contents 2 2.4. Document Directory Contents 3 2.5. Catalog Directory Contents 3 3. Archive Volume Format 4 3.1. Disk Format 4 3.2. File Formats 4 3.2.1. Document File Format 4 3.2.2. Tabular File Format 4 3.2.3. PDS Label Format 5 3.2.4. Catalog File Format 6 3.2.5. Science Data File Formats 6 4. Archive Volume Generation 7 4.1. Data Transfer and Validation Methods 7 4.2. Data Product Sizes and Delivery Rates 7 4.3. Interface Media Characteristics 7 4.4. Backup and Duplicates 7 4.5. Labeling and Identification 8 5. Support Staff and Cognizant Persons 8 TABLES AND FIGURES Table 1 - Standard Product Sizes and Delivery Rates 7 DOCUMENT CHANGE LOG Change Date Affected Portions Draft 5/18/07 TBD ITems 4/11/08 TBD ITEMS Section Description 4.2 Complete delivery schedule section. 4.5 Correct VOLUME_ID? ACRONYMS AND ABBREVIATIONS ASCII American Standard Code for Information Interchange ISO International Standards Organization JPL Jet Propulsion Laboratory NSSDC National Space Science Data Center PDS Planetary Data System SIS Software Interface Specification TBD To Be Determined 0. 1. Introduction 1.1. Purpose and Scope This Software Interface Specification is intended to be used by those who wish to understand the format and content of the Phoenix Meteorology (MET or P&T) and LIDAR EDR Archive. Typically, these individuals would be software engineers, data analysts, or planetary scientists. The specifications in this document apply to all MET and LIDAR standard product archive volumes that are generated by the Phoenix Project. 1.2. Content Overview The MET/LIDAR archive includes products from both the Pressure & Temperature (P&T) meteorology experiment and the LIDAR experiment. The data products are generated by the MET/LIDAR team at York University and Dalhousie University and delivered to the PDS Atmospheres Node at New Mexico State University. This Software Interface Specification (SIS) describes the format, content, and generation of the MET/LIDAR Archive. Section 2, Archive Volume Generation, describes the procedure for transferring data products to archive media. Section 3, Archive Volume Contents, describes the structure of the archive volumes and the contents of each file. Section 4, Archive Volume Format, describes the file formats used on the archive volumes. Finally, Section 5, Support Staff and Cognizant Persons, lists the individuals responsible for generating the archive volumes. 1.3. Applicable Documents and Constraints This Archive Volume SIS is intended to be consistent with the following documents: 1. Mars Exploration Program Data Management Plan, R. E. Arvidson et al., Rev. 3.0, March 20, 2002. 2. Phoenix Project Archive Generation, Validation and Transfer Plan, JPL D-29392, Rev. 1.0, December 20, 2004. 3. MET Lidar EDR and RDRs Software Interface Specification. 4. Planetary Data System Standards Reference, March 20, 2006, Version 3.7. JPL D-7669, Part 2. 5. ISO 9660-1988, Information Processing - Volume and File Structure of CD-ROM for Information Exchange, April 15, 1988. 1.4. Relationships with Other Interfaces This Archive Volume SIS could be affected by changes to the design of the MET/LIDAR standard data products (Applicable Document #3). 2. Archive Volume Contents This section describes the contents of the MET/LIDAR Archive volumes, including the file names, file contents, file types, and organization responsible for providing the files. 2.1. Root Directory Contents Files in the Root Directory include an overview of the archive, a description of the volume for the PDS Catalog, and a list of errata or comments about the archive. The following files are contained in the Root Directory. File Name File Contents File Provided By AAREADME.TXT Volume content and format information Instrument Team ERRATA.TXT A cumulative listing of comments and updates concerning all archive volumes published to date Instrument Team VOLDESC.CAT A description of the contents of this volume in a PDS format readable by both humans and computers PDS Atmospheres Node or Instrument Team 2.2. Data Directory Contents and Naming The Data Directory will be two archive volumes - one for each data set: PT and LIDAR These directories will contain subdirectories (if necessary) of the form Xannn where X refers to the MET (M) /LIDAR (L) instruments, a will be a character indicating the "location" of the observation: S = Surface, T = Testbed, C = Cruise; and nnn will be a three-digit Mars day number (or day of year number for testbed or cruise). Within each of these subdirectories will reside the data files as indicated in the Data Product SISs. 2.3. Index Directory Contents Files in the Index Directory are provided to help the user locate products on this archive volume and on previously released volumes in the archive. The following files are contained in the Index Directory. File Name File Contents File Provided By INDXINFO.TXT A description of the contents of this directory Atmospheres Node INDEX.TAB A table listing all data products on this volume Atmospheres Node INDEX.LBL A PDS detached label that describes INDEX.TAB Atmospheres Node CUMINDEX.TAB A cumulative listing of all data products on this volume and on previous volumes in this set Atmospheres Node CUMINDEX.LBL A PDS detached label that describes CUMINDEX.TAB Atmospheres Node 2.4. Document Directory Contents The Document Directory contains documentation to help the user understand and use the archive data. The following files are contained in the Document Directory. File Name File Contents File Provided By DOCINFO.TXT A description of the contents of this directory Atmospheres Node DPSIS.TXT or .HTM The Data Product SIS as text or hypertext Instrument Team DPSIS.PDF The Data Product SIS as a PDF file Instrument Team DPSIS.LBL A PDS detached label that describes both DPSIS.TXT(HTM) and DPSIS.PDF Instrument Team ARCHSIS.TXT or .HTM The Archive Volume SIS (this document) as text or hypertext Atmospheres Node ARCHSIS.PDF The Archive Volume SIS (this document) as a PDF file Atmospheres Node ARCHSIS.LBL A PDS detached label that describes both ARCHSIS.TXT(HTM) and ARCHSIS.PDF. Atmospheres Node other files Other files as deemed necessary by either Instrument Team or Atmospheres Node - TBD Instrument Team 2.5. Catalog Directory Contents The files in the Catalog Directory provide a top-level understanding of the mission, spacecraft, instruments, and data sets. The files in this directory are coordinated with the PDS data engineer, who is responsible for loading them into the PDS catalog. The following files are found in the Catalog Directory. File Name File Contents File Provided By CATINFO.TXT A description of the contents of this directory Atmospheres Node LIDARDS.CAT LIDAR Data set information for the PDS catalog Instrument Team PTDS.CAT PT Data set information for the PDS catalog Instrument Team INSTHOST.CAT Instrument host (i.e., spacecraft) information for the PDS catalog Phoenix Project INST.CAT Instrument information for the PDS catalog (one each for PT and for LIDAR) Instrument Team MISSION.CAT Mission information for the PDS catalog Phoenix Project PERSON.CAT Personnel information for the PDS catalog (Team and PDS personnel responsible for generating the archive. One each for PT and for LIDAR) Instrument Team REF.CAT References mentioned in other *.CAT files catalog (one each for PT and for LIDAR) Instrument Team 3. Archive Volume Format This section describes the format of MET/LIDAR Archive Volumes. Data that comprise the Archive will be formatted in accordance with Planetary Data System specifications [Applicable Document 4]. 3.1. Disk Format Archive Volumes follow a compact disk format that is compatible with the computer operating systems MS-DOS, Macintosh, and SunOS. The volume format is in accordance with ISO 9660 level 2 Interchange Standard [Applicable Document 5]. Note however that delivery is expected to be electronic from the instrument team to PDS. PDS will assume the responsibility for actually writing the volumes onto hard media such as CDs or DVDs, if at all. 3.2. File Formats This section describes file formats for the kinds of files contained on MET/LIDAR Archive Volumes. 3.2.1. Document File Format Document files with the .TXT suffix exist in the Root, Index, Catalog, and Document directories. They are ASCII files which may have embedded PDS labels. Lines in a .TXT file end with a carriage return character (ASCII 13) and a line feed character (ASCII 10). This allows the files to be readable under various operating systems. Documents in the Document directory may contain formatting and figures that cannot be rendered as ASCII text. Therefore each document is given in two formats, either hypertext or plain ASCII, and PDF. The hypertext file contains ASCII text plus hypertext markup language (HTML) commands that enable it to be viewed in a Web browser such as Netscape Navigator or Microsoft Internet Explorer. The hypertext file may be accompanied by ancillary files such as images and style sheets that are incorporated into the document by the Web browser. The second format, PDF (Portable Document Format) is a proprietary format of Adobe Systems Incorporated that is frequently used for distributing documents. Adobe offers free software, Acrobat Reader, for viewing PDF files. 3.2.2. Tabular File Format Tabular files (.TAB suffix) exist in the Index and Data directories. Tabular files are ASCII files formatted for direct reading into many database management systems on various computers. All fields are separated by commas, and character fields are enclosed in double quotation marks ("). (Character fields are padded with spaces to keep quotation marks in the same columns of successive records.) Character fields are left justified, and numeric fields are right justified. The "start byte" and "bytes" values listed in the labels do not include the commas between fields or the quotation marks surrounding character fields. The records are of fixed length, and the last two bytes of each record contain the ASCII carriage return and line feed characters. This allows a table to be treated as a fixed length record file on computers that support this file type and as a text file with embedded line delimiters on those that don't. All tabular files are described by PDS labels, either embedded at the beginning of the file or detached. If detached, the PDS label file has the same name as the data file it describes, with the extension .LBL; for example, the file INDEX.TAB is accompanied by the detached label file INDEX.LBL in the same directory. 3.2.3. PDS Label Format All data files in the archive have PDS labels, either embedded at the beginning of the file or detached in a separate file. For examples of PDS labels for each type of data product, see the Data Product SIS [Applicable Document 3]. A PDS label, whether embedded or detached from its associated file, provides descriptive information about the associated file. The PDS label is an object-oriented structure consisting of sets of 'keyword=value' declarations. The object to which the label refers (e.g. IMAGE, TABLE, etc.) is denoted by a statement of the form: ^object = location in which the carat character (^, also called a pointer in this context) indicates where to find the object. In an embedded label, the location is an integer representing the starting record number of the object (the first record in the file is record 1). In a detached label, the location denotes the name of the file containing the object, along with the starting record or byte number, if there is more than one object in the file. For example: ^HEADER = ("F01.IMG",1) ^IMAGE = ("F01.IMG",1025 ) indicates that the IMAGE object begins at byte 1025 of the file F01.IMG, in the same directory as the detached label file. Below is a list of the possible formats for the ^object definition. ^object = n ^object = n ^object = "filename.ext" ^object = ("filename.ext",n) ^object = ("[dirlist]filename.ext",n) ^object = ("filename.ext",n) ^object = ("[dirlist]filename.ext",n) where n is the starting record or byte number of the object, counting from the beginning of the file (record 1, byte 1), indicates that the number given is in units of bytes, filename is the up to 8 character, alphanumeric upper-case file name, ext is the 3 character upper-case file extension, dirlist is a period-delimited path-list of parent directories, in upper case, that specifies the object file directory (used only when the object is not in the same directory as the label file). The list begins at the directory level below the root directory of the CD-ROM. '[dirlist]' may be omitted when the object being described is located either in the same directory as the detached label, or in a subdirectory named LABEL that is located in a higher level of the directory tree, typically the CD-ROM root itself. Lines of text in detached labels end with a carriage return character (ASCII 13) and a line feed character (ASCII 10). This allows the files to be readable under various operating systems. 3.2.4. Catalog File Format Catalog files (suffix .CAT) exist in the Root and Catalog directories. They are text files formatted in an object-oriented structure consisting of sets of 'keyword=value' declarations. 3.2.5. Science Data File Formats The MET/LIDAR data files are ASCII tables with comma-separated values. See the above section on Tabular File Format for a more complete description. For more information about the format and content of the data products, see the Data Product SISs [Applicable Document 3]. 4. Archive Volume Generation 4.1. Data Transfer and Validation Methods MET/LIDAR products are generated by the instrument team at York University and/or at Dalhousie University. The products will be delivered electronically (FTP or other means) to the PDS Atmospheres Node at New Mexico State University according to the Phoenix Project delivery schedule. Assembly of the volumes will be done by the instrument team. The MET/LIDAR team validates the data products for science content as part of the product generation process. The Atmospheres Node validates the data products and ancillary files for compliance with PDS standards and with the SISs. The volumes will then be made available via Internet and FTP access at the Atmospheres Node. The Atmospheres Node is responsible for delivering copies of the volumes to the NSSDC and to the PDS Engineering Node (as the backup site for the Atmospheres Node). 4.2. Data Product Sizes and Delivery Rates Table 1 summarizes expected sizes and production rates for the MET/LIDAR Standard Products. Table 1 - Standard Product Sizes and Delivery Rates Product Product Size Production Rate Expected Number of Products for Primary Mission (TBD days) Expected Total Data Volume for Primary Mission TBD Schedule for release of data products as described in the Project Archive Plan should be included here. TBD. 4.3. Interface Media Characteristics All volumes in the MET/LIDAR Standard Product Archive conform to ISO 9660 standards [Applicable Document 5]. 4.4. Backup and Duplicates Backup copies of MET/LIDAR products are the responsibility of the instrument team until the volumes have been validated by the Atmospheres Node. Duplicate copies of MET/LIDAR archive volumes will be housed at the PDS Engineering Node and at NSSDC. 4.5. Labeling and Identification Each MET/LIDAR archive volume will be identified by a unique volume ID formed according to the scheme PH##_xxxx (TBD), where: PH represents the Phoenix mission, ## represents the MET (MT) or LIDAR (LD) instrument, and Xxxx is a volume sequence number starting with 0001. 5. Support Staff and Cognizant Persons Jim Whiteway, MET/LIDAR Principal Investigator, York University Cameron Dickinson, MET/LIDAR Collaborator, York University Allan Carswell, MET/LIDAR Collaborator, York University Peter Taylor, MET Collaborator, York University Carlos Lange, MET Collaborator, University of Alberta Tom Duck, MET?LIDAR Collaborator, Dalhousie University David Fisher, MET Collaborator, Lyle Huber, PDS Atmospheres Node, New Mexico State University Jim Murphy, PDS Atmospheres Node, New Mexico State University Questions and comments regarding the MET/LIDAR Archive Volume may be directed to the PDS Atmospheres Node, lhuber nmsu.edu, 505-646-1862. ?? MET/LIDAR Archive Volume SIS 4/17/2008 Archive Volume SIS Version 0.0 i iv