PERL VALIDATION TOOLKIT USERS GUIDE

 

 

CONTENTS

·        INTRODUCTION

·        PERL TOOLS

o        TKVALID.PL

o        VALIDATE.PL

o        LISTERR.PL

o        CHECKISO.PL

o        CHECKREQ.PL

o        LISTNEW.PL

o        TESTLINE.PL

o        CHECKREF.PL

o        LISTKWV.PL

 

 

INTRODUCTION

 

The PDS Validation Tools (PVT) are a set of scripts and programs that perform the following validation tests:

 

·        Check ISO9660 compliance (checkiso.pl, using isocheck program)

·        Check for required files and directories (checkreq.pl)

·        Find label validation errors (listerr.pl, using lvtool)

·        Find new keywords and standard values (listnew.pl, using lvtool)

·        Check that the index file points to the data (checkidx.pl)

·        Test for errors in ASCII files (testline.pl)

·        Check references against reference database (checkref.pl)

·        List keyword/value pairs for inspection (listkwv.pl, using kwvtool)

 

A single controlling script (validate.pl) allows the user to select which of the tests to run via command line. Running validate.pl –b <volume root> executes all the tests automatically.

 

An even easier way to use PVT is through its graphical user interface (tkvalid.pl). Simply run tkvalid.pl, then use your mouse to select the volume root and choose the tests to run.

 

PERL TOOLS

 

 

TKVALID.PL

 

·        Script: tkvalid.pl – perl/tk front end to the PDS Validation Tools

·        Usage: tkvalid.pl [--lvt]

 

DESCRIPTION

 

This script is a perl/tk front end of the PDS Validation Tools. It gets user input through dialog windows and runs the appropriate validation tests automatically.

 

NOTE: tkvalid.pl requires that the user’s perl installation include the Tk module.

 

 

Use the “Browse” button to set the root directory of the volume to validate or click once on the white box located next to the Browse button and then enter the desired directory manually.

 

Next, select the tests that you want to run on the volume by clicking on the appropriate box next to the test type. Details on these types of tests are

described in the sections found below.

 

When the Label Validation test is checked, another window will automatically appear:

 

 

This window contains a variety of options that modify the output behavior of this test. By default, aliasing and only validating

Level 3 labels found in the volume are set. Located to the right of this screen is a list of warning and error messages that

may be found during this test. By selecting one or more of these messages, it will prevent those selected messages from

appearing in the output report.

 

Description of options found on this screen:

 

 

°         Ignore All Warning Messages

 

Do not display any warning messages in the output report

 

 

°         Ignore Info Messages

 

Do not display any informational messages in the output report

 

 

°         Validate Only Files that Contain Level 3 Labels

 

By default, this is selected. If this is not selected, the test will validate all files found in the specified volume.

 

 

°         Enable aliasing

 

Alias keywords in a label whenever applicable. By default, this is selected.

 

 

°         Save Expanded Files

 

Do not delete any files that were expanded. When a label contains a STRUCTURE pointer, it creates a temporary file, also known as an expanded file. The messages in the output report will refer to this expanded file and so this temporary file may need to be retained if a user would want to physically look at the label to see what the message is referring to.

 

 

°         Clear All Selected Messages

 

Clears any messages selected in the list of possible warning and error messages

 

 

OPTIONS

 

o        --lvt

This option brings up a different window than the default main window. This option will bring up a “Messages to Omit” window that displays all possible warning and error messages that can appear in an label validation test report.

 

This window will allow a user to select messages that they do not want to see in the final label validation test report and save it onto a file.

 

 

Within this window, there exists a menubar at the top where you can perform the following functions:

 

·        Save selections onto a file

·        Load a previously saved file of messages chosen

·        Clear all Selected Messages on the screen

·        Select All Warning Messages

·        Select All Error Messages

 

 

VALIDATE.PL

 

·        Script: validate.pl – Runs validation tests on a volume

·        Usage:

o       For Windows users: validate.pl [-b] [--rw] [-12345678] <volume root> [report directory]

o       For Linux and Solaris users: validate.pl [-b]  [--rw]  [-12345678] <volume root> [report directory]  [-c] <cd-rom device>

 

If –3 (or –b) is specified, these options are also valid:

·        [-s] [--em] <filename> [--ninfo] [--nwarn] [--nol3d] [--max] <integer value> [-nalias] [--se]

 

 

DESCRIPTION

 

This script can run any or all of the following validation tests. (Note that any of these tests can be run individually, using the programs shown in parentheses, below:)

  1. check ISO9660 compliance (isocheck, checkiso.pl)
  2. check for required files and directories (checkreq.pl)
  3. check whether every file has a label (checklab.pl)
  4. find label validation errors (lvtool, listerr.pl)
  5. find new keywords and standard values (lvtool, listnew.pl)
  6. check that the index file points to the data (checkidx.pl)
  7. test for errors in ASCII files (testline.pl)
  8. check references against reference database (checkref.pl)
  9. list keyword/value pairs for inspection (kwvtool, listkwv.pl)

 

The volume root (the path to the root directory of the volume) is required. The test report will have the name VOLUME_ID.rpt and will be placed in the same directory as validate.pl, unless the [report directory] is specified.

 

There are two ways to run validate.pl. The first is interactively, where the only input is the root directory of the volume to be validated. The script prompts the user, asking which tests to perform.

 

 

OPTIONS

 

o        -b

 

The script can also be run in “batch mode”, by setting the –b option. If –b is set, all tests are run automatically, no questions asked. If – is set (and –b is not), then the tests corresponding to the numbers given are run. For example, -1357 runs the tests numbered 1, 3, 5, and 7, as defined above.

 

o        --rw

 

This option will enable the output report display window. With this option set, a window will appear that displays the results of the tests run by the user after all tests are executed.

 

o        [report directory]

 

Specify the directory path that the test report will go into. By default, the report is placed in the same directory as validate.pl.

 

o        -n

 

An additional option was added to account for the “pseudo PDS” atached labels used by NAIF. If processing NAIF volumes, use this –n flag.

 

 

If test #3 (-3) or batch mode (-b) is enabled, the following options are also valid:

 

o        -s

 

This option will enable a window containing a summary of the messages found during the label validation test.

A window will appear containing a brief summary of the types of warning, error, and info messages found after the validation test is finished.

User has the option of choosing to see a detailed or brief message regarding each type of message found.

 

A detailed message would look something like the following:

 

(#1) HARDWARE_MODEL_ID:  Value is not a standard value: SUN SPARC STATION

            pds_test_dvd/collected_test_products/tables//ERIN_LABEL.x00: Line 5

            pds_test_dvd/collected_test_products/tables//SL9PPRG.LBL: Line 5

            pds_test_dvd/collected_test_products/tables//SL9PPRG_MOD.LBL: Line 4

            pds_test_dvd/collected_test_products/tables//SL9PPRH.LBL: Line 5

            pds_test_dvd/collected_test_products/tables//SL9PPRL.LBL: Line 5

 

where as a summarized message would be something like

 

(#5) TABLE:  Using PDS GENERIC object definition.

            12 occurrences in 12 files

 

o        --em <filename>

 

This option allows a user to specify a text file containing messages that they do not wish to see in the label validation test report.

 

This file is generated by running “tkvalid.pl --lvt” and selecting the save option after selecting the messages that the label validation test will ignore.

 

o        --ninfo

 

Remove all informational messages from the label validation test report.

 

o        --nwarn

 

Remove all warning messages from the label validation test report.

 

o        --nol3d

 

Option to validate all input files, regardless if it is a level 3 label file. Default is to only validate level 3 label files.

 

o        --max <integer value>

 

Option to specify the maximum number of error messages to report before quitting the label validation test, where <integer value> is a valid integer value greater than 0. By default, this is set to 300.

 

o       --nalias

 

Option to disable the aliasing feature. Default is set to ON.

 

o       --se

 

Option to save all expanded label files created during the label validation test. By default, these files are deleted after execution of the validation test.

 

 

For Linux and Solaris Users,

 

o        –c <cdrom device path>

 

This option is valid when a user wants to run the isochecker (test #1). For Solaris, your cd-rom device might be located along the lines of “/vol/dev/dsk/<disc name>”. For Linux, your cd-rom device might be located along the lines of “/dev/cdrom”. Check with your System Administrator for the exact path if you are unsure.

 

 

LISTERR.PL

 

·        Script: listerr.pl – List ODL validation errors on a volume

·        Usage: listerr.pl <lvtool report> --em <filename> -s

 

DESCRIPTION

 

This script scans the output of an lvtool report and lists all the errors, warnings, informational, and missing pointers. By default, it is sorted by error type.

 

An lvtool report file is required to run this script.

 

NOTE: Prior to running listerr.pl, lvtool must be run with the –nw option set.

 

After all the information is gathered, a window will appear showing a summary of the messages found in the report if the –s option is set.

 

 

 

Selecting messages from this screen will allow the final report to contain associated filenames for each selection that was made:

 

 

(#31) MAXIMUM_LONGITUDE: Not an allowed keyword.  (It was not found on the optional or required keywords list)

            images/FF01.LBL:  Line 42

            images/MDIM.LBL: Line 52

            images/i480009a.img: Line 176

            images/i497909a.img: Line 176

 

 

Otherwise, a summary will be displayed for non-selected messages:

 

 

(#31) MAXIMUM_LONGITUDE: Not an allowed keyword. (It was not found on the optional or required keywords list)

            4 occurrences in 4 files

 

 OPTIONS

 

o        --em <filename>

 

Specify a text file containing messages that you would like to remove from the final report.

 

This file is generated by running “tkvalid.pl --lvt” and selecting the save options after you have finished selecting your messages.

 

o        -s

 

Display the window containing a summary of the messages found in the lvtool report.

 

By default, all associated filenames pertaining to each message type stated in the report will be shown.

 

 

CHECKISO.PL

 

·        Script: checkiso.pl – Check ISO compliance

·        Usage: checkiso.pl <CD Reader>

 

DESCRIPTION

 

This script runs the program ISOCHECK on a CD reader and checks whether the CD it holds is ISO9660 compliant. For Win32 systems, the CD reader is a volume, for example, e:\. For Unix systems, the CD reader is a device file, for example, \dev\cdrom. In any case, checkiso.pl ONLY checks CD media and should NOT be used with anything else. If the CD is ISO9660 compliant, a simple confirmation message is printed. If the CD fails the test, an error message and a report (iso.rpt) is generated.

 

 

 

CHECKREQ.PL

 

·        Script: checkreq.pl – Check for required files and directories

·        Usage: checkreq.pl <volume root>

 

DESCRIPTION

 

This script checks that required directories and files are present on a volume. It prints a report to the standard output device. The only option it takes is the path to the root directory of the volume. NOTE: Checkreq.pl does not check for ISO9660 compliance and treats filenames the way perl treats filenames on ISO9660 CD-ROMs.

 

 

LISTNEW.PL

 

·        Script: listnew.pl – List new keywords and values on a volume

·        Usage: listnew.pl –f <lvtool report>

 

 

DESCRIPTION

 

This script scans the output of lvtool and lists all the keywords and standard values that are not currently in the data dictionary. An lvtool report file is required as input.

 

Note: prior to running listnew.pl, lvtool must be run with the "-nw" flag.

 

 

OPTIONS

 

o        –f

 

If the "-f" option is set, the listing includes the files in which the new keywords and standard values are found.

 

 

TESTLINE.PL

 

·        Script: testline.pl – Test ASCII files for PDS standards compliance

·        Usage: testline.pl [-v] [-s] [-l] <filename or volume root>

 

 

DESCRIPTION

 

This script performs the following tests on ASCII files:

 

  1. Whether each line is terminated with a CR-LF pair.
  2. Whether any line exceeds 80 characters in length, including the CR-LF pair.
  3. Whether any line contains an embedded TAB character.
  4. Whether any line conains another character outside the range 20-7E (hexidecimal).

 

There is one required argument. It can be either the name of a file to test or the absolute path to the volume root, from which all appropriate files are tested recursively. For single file testing, the -l option tells the script to interpret the file as an attached label. For recursive testing, files with TXT, ASC, CAT, LBL, and FMT extensions are automatically checked. All other files are checked to see if an attached label is present. If it is, the attached label portion of the file is tested, unless the -s flag is set.

 

 

OPTIONS

 

o        –v

 

By default, testline.pl only reports files that have errors. If this flag option is set, all files will be listed.

 

 

o        –l

 

Tells the script to interpret the input file as an attached label.

 

o        –s

 

This flag causes testline.pl to skip attached labels.

 

 

CHECKREF.PL

 

·        Script: checkref.pl – Check REF.CAT references against all AllRefs.txt

·        Usage: checkref.pl [-p] [-v] [-w] <volume root>

 

DESCRIPTION

 

This script checks references in REF.CAT against the AllRefs.txt database. It finds and lists the following:

 

  1. Duplicate references in REF.CAT
  2. References in *.CAT that are missing from REF.CAT
  3. References in REF.CAT that are not in the database
  4. Citations in REF.CAT that don't match those in the database

 

When a citation in REF.CAT doesn't match the same reference in the database, both are printed out for comparison.

 

Note: The AllRefs.txt database must be in the current working directory.

 

Checkref.pl takes one required parameter, the root directory of the volume.

 

 

OPTIONS

 

o        –p

 

With this flag, a private copy of REF.CAT is used. It must be named “REF.CAT” in the current working directory. This is convenient for editing REF.CAT.

 

o        –v

 

With this option, the list of REFKEYIDs in REF.CAT and those cited are also printed out.

 

o        –w

 

With this option, whitespaces are ignored when matching citations.

 

 

LISTKWV.PL

 

·        Script: listkwv.pl – Lists all keyword/value pairs

·        Usage: listkwv.pl <volume root>

 

 

DESCRIPTION

 

This script runs the program KWVTOOL on all PDS labels found under the volume pointed to by <volume root>. A report is generated under the name “kwv.rpt”.