| DATA_SET_DESCRIPTION |
Data Set Overview
=================
This volume contains portions of the CRISM Calibration Data Record
(CDR) Archive, a collection of calibration files from the Compact
Reconnaissance Imaging Spectrometer for Mars on the Mars
Reconnaissance Orbiter spacecraft.
CDRs are created in two ways:
(a) One type of calibration file, for example the wavelength
calibration, is either invariant or expected to change only
infrequently. These files are stored in directories named using a
2-letter acronym for the contents of the file.
(b) The other type of calibration file is a snapshot of some
attribute of the instrument that is time-variable, for example the
thermal background measured by the IR detector. These files are
stored in directories named YYYY_DOY, each one of which contains
subdirectories containing distinct types of calibration files.
Processing
==========
EDRs containing bias measurements or measurements of background,
focal plane lamps, or the internal integrating sphere are
processed into level-4 CDRs. A level-4 CDR contains derived values
needed to convert a scene-viewing EDR into units of radiance.
These are in image format, with multiple versions corresponding to
different pixel binning states. Other level 4 CDRs are derived
from ground measurements. A level-6 CDR contains tabulated
information, of two types: (a) a file list of level 4 CDRs
generated inflight, or (b) tabulated information, for example, for
correcting for detector non-linearity, converting housekeeping to
physical units, or converting EDRs from 12 to the original
at-sensor 14 bits prior to calibration.
Data
====
There are two types of calibration files. CODMAC level 4 CDRs
(CDR4s) are in image format, and CODMAC level 6 CDRs (CDR6s) are
text files in ASCII format. Both types have a file name that
encodes the time at which they are applicable, and the
configuration of the data used to derive them (detector, frame
rate, binning, wavelength filter, exposure time parameter, etc.).
YYYY_DOY/BI directory:
----------------------
This directory contains CDR4s representing measurements of the
response of each detector to zero signal, i.e., the detector bias.
These are measured as dark frames for the VNIR detector. For the
IR detector, these are the zero exposure-time intercept of a fit
of dark measurements taken at several exposure times. For each
detector there are separate files for each frame rate, wavelength
filter, and binning state, time-stamped for their mean time of
acquisition.
YYYY_DOY/BK directory:
----------------------
This directory contains CDR4s representing thermal background
measured by the IR detector. These are measured as dark frames.
There are separate files for each frame rate, exposure time, and
binning state, time-stamped for their mean time of acquisition.
YYYY_DOY/BP directory:
----------------------
This directory contains CDR4s representing bad pixels. There are
separate files for each frame rate, exposure time, wavelength
filter, and binning state, time-stamped for their mean time of
acquisition.
YYYY_DOY/SP directory:
----------------------
This directory contains CDR4s representing sphere output in DN/ms
at the sphere's closed-loop setpoint. There are separate files for
each sphere bulb. For the VNIR all data are taken at 1 Hz, but
there are versions for each binning state. For the IR there are
separate versions for each frame rate, wavelength filter, and
binning state. Both the VNIR and IR files are time-stamped for
their mean time of acquisition.
YYYY_DOY/ST directory:
----------------------
This directory contains a table of CRISM low-rate telemetry in raw
counts, from the beginning of a UTC calendar day to its end. This
is used in preference to the telemetry attached to each image for
correction of thermal effects, because of uncorrectable artifacts
in the raw values of critical temperature measurements in the
image headers.
YYYY_DOY/UB directory:
----------------------
This directory contains CDR4s representing pixel-by-pixel
uncertainties in VNIR bias or IR background images. There are
separate files for each frame rate, exposure time, wavelength
filter, and binning state, time-stamped for their mean time of
acquisition. This product is not a part of the nominal data
processing pipeline and instead is used to monitor detector
health.
YYYY_DOY/ATF directory:
----------------------
This directory contains a table of EDRs containing scene data and
the corresponding EDRs containing time-dependent calibration
measurements needed to calibrate the scene EDRs. It is used to
process calibration EDRs to CDRs, and scene EDRs to TRDRs. If
there is a discrepancy between the actual EDRs used for
calibration and the predicted EDRs (in the BTF directory), the
TRDRs resulting from scene EDRs are quality-flagged.
YYYY_DOY/BTF directory:
----------------------
This directory contains a predicted table of EDRs containing scene
data and the corresponding EDRs containing time-dependent
calibration measurements. It is constructed from uplinked
commands.
AS directory:
-------------
This CDR6 gives the maximum expected 14-bit scene or sphere 14-bit
DN level (units from the detector, after correction for the pixel
processing table and subtraction of bias) for core observation
types to be made by CRISM. There are 5 core observation types
defined by combinations of frame rate, exposure time, and scene: 1
Hz observation, exposure parameter 27, observing the internal
integrating sphere; 3.75 Hz, exposure parameter 301, observing a
nominally non-ice region of Mars; 3.75 Hz, exposure parameter 147,
observing a potentially ice-containing region; 15 Hz, exposure
parameter 425, observing any region; and 30 Hz, exposure parameter
425, observing any region. The values given here are used with
bias images in the YYYY_DOY/BI directories to generate masks of
probably bad pixels (in the YYYY_DOY/BI directories).
AT directory:
-------------
This CDR4 gives the atmospheric transmission as measured from a
nadir-pointed hyperspectral scan across Olympus Mons, taken 27
November 2006. It is not used during calibration to radiance or
I/F; instead it is intended for subsequence correction for
atmospheric attenuation. There is a different version for each
detector, wavelength filter, and binning state.
BS directory:
-------------
This CDR6 gives the amplitude of the bias step correction for each
detector as a function of frame rate and quadrant. The bias step
is a step function in detector bias that occurs at a row number
(band) correlated with exposure time parameter.
BW directory:
-------------
This CDR6 gives the parameters to calculate the spectral profile
for each wavelength bin averaged over IR columns 270-369 or VNIR
columns 260-359. This represents the 'sweet spot' of each
detector at which spectral smile and keystone are minimum, and is
the preferred detector location for extraction of overlapping data
for analysis of emission phase functions.
CM directory:
-------------
This CDR4 gives the along-slit angle measured from slit center for
each detector. It is similar to the SPICE Instrument Kernel. There
is a different version for each detector, wavelength filter, and
binning state.
CT directory:
------------
This CDR6 gives the atmospheric transmission for each wavelength
bin averaged over IR columns 270-369 or VNIR columns 260-359. This
represents the 'sweet spot' of each detector at which spectral
smile and keystone are minimum, and is the preferred detector
location for extraction of overlapping data for analysis of
emission phase functions. It is a sample of the 'native' atmospheric
transmission images in the AT directory.
DB directory:
-------------
This CDR6 gives the correction to detector bias for changes in
detector temperature since the last bias measurement, as a
function of frame rate and quadrant.
DM directory:
-------------
This CDR4 is a mask of detector dark columns, scattered light
columns, and scene columns for each detector. There is a different
version for each wavelength filter and binning state.
EB directory:
-------------
This CDR6 gives the correction to detector bias for differences in
focal plane electronics temperature since the last bias
measurement, as a function of frame rate and quadrant.
GH directory:
-------------
This CDR6 gives the scaling factors needed for each frame rate to
calculate and subtract the inter-quadrant electronics ghost in
each detector, using bias-corrected signal.
HD directory:
-------------
This CDR6 gives the coefficients for correcting raw housekeeping
for perturbations due to changes in lamp or cooler state and frame
rate. There is one file for both detectors.
HK directory:
-------------
This CDR6 gives the coefficients to convert housekeeping
parameters from raw counts to physical units.
HV directory:
-------------
This CDR6 gives the coefficients to correct housekeeping voltages
for perturbations due to changes in current.
LC directory:
-------------
This CDR6 gives the coefficients to correct bias-subtracted DN
from each detector at each frame rate for non-linearity in
detector response.
LI directory:
-------------
This CDR6 gives 8 to 12 bit lookup tables (inverse of 12 to 8).
These are used to restore 8-bit to 12-bit DN values.
LK directory:
-------------
This CDR6 gives 12 to 8 bit lookup tables loaded in the CRISM
instrument.
LL directory:
-------------
This CDR4 contains matrices to remove estimated leaked higher
order light in the IR detector, by scaling shorter-wavelength
signal and subtracting it from longer-wavelength signal.
NU directory:
-------------
This CDR4 is a time-tagged, row-normalized measurement of detector
nonuniformity. There is one version for each binning state,
wavelength filter, and frame rate. It is only created for the VNIR
detector.
PP directory:
-------------
This CDR6 contains the gain and offset to use for each row to
convert 12-bit DNs to the native 14-bit DNs produced by each
detector, and the 12 to 8 bit lookup tables used for each detector
row (band) when lossy compression is employed.
PS directory:
-------------
This CDR4 is used for nearest neighbor spectral resampling. It
gives the number of detector rows (bands) by which to shift each
column (sample) of an image to minimize the effects of spectral
smile. The effect of the pixel shift is to restore the wavelength
at a detector element to within one-half detector element of the
wavelength in the central columns of the detector. (The
wavelength and spectral bandwidth in the central columns are given
in the SW and BW directories respectively.) There is a different
version for each detector, wavelength filter, and binning.
RA directory:
-------------
This CDR4 contains the ratio of focal plane lamp illumination to
an external flat field, normalized on a row-by-row basis. This is
a contingency and not part of the normal calibration pipeline, so
its creation is TBD. There is a different version for each
detector, wavelength filter, and binning.
RF directory:
-------------
This CDR4 contains a nearest-neighbor resampled solar flux image.
It is related to the 'native' solar flux images in the SF
directory by the pixel- shifting given in the PS directory. There
is a different version for each detector, wavelength filter, and
binning.
RT directory:
-------------
This CDR4 gives the atmospheric transmission as measured from a
nadir-pointed hyperspectral scan across Olympus Mons, taken 27
November 2006. It has been nearest-neighbor resampled in the
spectral direction. It is not used during calibration to radiance or
I/F; instead it is intended for subsequence correction for
atmospheric attenuation. It is related to the 'native' atmospheric
transmission images in the AT directory by the pixel- shifting
given in the PS directory. There is a different version for each
detector, wavelength filter, and binning. There is a different
version for each detector, wavelength filter, and binning state.
RW directory:
-------------
This CDR4 contains a nearest-neighbor resampled wavelength image.
It is related to the 'native' wavelength images in the WA
directory by the pixel- shifting given in the PS directory. There
is a different version for each detector, wavelength filter, and
binning.
SB directory:
-------------
This CDR4 contains the spectral bandpass width for each detector
element. There is a different version for each detector,
wavelength filter, and binning.
SF directory:
-------------
This CDR4 contains solar flux at 1 AU for each detector element.
There is a different version for each detector, wavelength filter,
and binning. It is used together with the solar distance given in
an EDR or TRDR label to calculate I/F.
SH directory:
-------------
This 2-layer CDR4 is used to correct flight measurements of the
onboard integrating sphere for effects of shutter position
irreproducibility. Layer 0 is a corrected sphere image from ground
radiometric calibration in units of 14-bit DN. Layer 1 is a
shutter mirror non-repeatability correction image, a
multiplicative correction that gets scaled by the ratio of layer 0
to the measured flight sphere image. There is a different version
for each detector, wavelength filter, and binning.
SL directory:
-------------
This CDR6 gives the 14-bit DN at which each detector is saturated,
as a function of quadrant and frame rate.
SS directory:
-------------
This 3-layer CDR4 gives sphere spectral radiance at the
closed-loop set point for each sphere bulb, as pixel by pixel
coefficients to a 2nd order polynomial function of sphere
temperature. There is a different version for each detector,
wavelength filter, and binning.
SW directory:
-------------
This CDR6 gives the center wavelength for each wavelength bin
averaged over IR columns 270-369 or VNIR columns 260-359. This
represents the 'sweet spot' of each detector at which spectral
smile and keystone are minimum, and is the preferred detector
location for extraction of overlapping data for analysis of
emission phase functions.
TD directory:
-------------
This CDR4 gives temperature dependence of detector responsivity,
as pixel by pixel coefficients to a 2nd order polynomial function
of detector temperature. There is a different version for each
detector, wavelength filter, and binning.
UR directory:
-------------
This CDR6 gives uncertainty in absolute and relative sphere
signal, for each bulb. This product is not a part of the nominal
data processing pipeline and its creation is TBD; it is intended
for end users of calibrated data as an indication of limits to
data interpretation.
VL directory:
-------------
This CDR6 gives the 14-bit DN at which each detector -
ON AVERAGE - is saturated. It is used together with the predicted
limiting case expected 14-bit scene DN levels (in the AS
directory) to produce bad pixel maps. It also gives additional
criteria for defining a bad pixel, thresholds for noise and
relative pixel response.
WA directory:
-------------
This CDR4 gives the center wavelength for each detector element.
There is a different version for each detector, wavelength filter,
and binning.
WV directory:
-------------
This CDR6 gives the table of detector rows (bands) loaded into
wavelength filters 0-3. There is one version for each detector.
File Naming Convention
======================
The file naming convention for level-6 CDRs is as follows.
(ProductType)(Level)_(Partition)_(Time)_(Product)_
(SensorID)_version.(Ext)
where:
Product Type = CDR
Level = 6
Partition = n, partition of the spacecraft clock.
Time = nnnnnnnnnn, spacecraft start time of applicability of data
product; units are spacecraft clock counts, in units of
whole seconds.
Product = nn, acronym describing data product listed above
Sensor ID = S or L (or J=joint)
Version = 0, 1,..., 9, a,..., z
Ext = TAB
The file naming convention for level-4 CDRs is as follows.
(ProductType)(Level)(Partition)(Time)_
(Product)(FrameRate)(Binning)
(ExposureParameter)(WavelengthFilter)
(Side)(SensorID)_ version.(Ext)
where:
Product Type = CDR
Level = 4
FrameRate = n, rate in Hz at which data are taken (0=1 Hz,
1=3.75 Hz, 3=156 Hz, 4=30 Hz, 5 = N/A)
Binning = n, number of spatial pixels binned
(0=unbinned, 1= 2x binned, 2= 5x binned,
3= 10x binned, 4=N/A)
Exposure parameter = nnn, an integer 1-480 indicating commanded
exposure time in units of (inverse frame
rate)/480; 000 if inapplicable
Wavelength filter = n, and integer 0-3 indicating which onboard
menu of rows of the detector are represented
Side = #, 1 or 2 for focal plane or sphere bulbs; or 0 if N/A
Ext = IMG
The tables in the YYYY_DOY/ATF and YYYY_DOY/BTF directories have a
different file naming convention, as follows.
(Product)_(Sensor)_(YYYY)_(DOY)_version
where:
Product Type = BTF for before-the-fact predicted or ATF for
after-the-fact actual
Sensor = VN or IR
YYYY = year
DOY = day of year
Version = nn
Media/Format
============
The CRISM archive will be made available online via Web and FTP
servers. This will be the primary means of distribution.
Therefore the archive will be organized as a set of virtual
volumes, with each data set stored online as a single volume. As
new data products are released they will be added to the volume's
data directory, and the volume's index table will be updated
accordingly. The size of the volume will not be limited by the
capacity of the physical media on which it is stored; hence the
term virtual volume. When it is necessary to transfer all or part
of a data set to other media such as DVD for distribution or for
offline storage, the virtual volume's contents will be written to
the other media according to PDS policy, possibly dividing the
contents among several physical volumes.
|
.png)
.png)
