| DATA_SET_DESCRIPTION |
Data Set Overview
=================
This data set contains products acquired by the IR2 instrument of the
Venus Climate Orbiter (VCO, also known as PLANET-C and AKATSUKI)
spacecraft.
This data set uses the Committee on Data Management and Computation
(CODMAC) data level numbering system. The IR2 EDRs are considered Level 2
or Edited Data (equivalent to NASA Level 1A). The EDRs are reconstructed
from Level 1 or Raw Data.
This data set consists entirely of FITS files. Uncalibrated images in
units of counts are included. The headers of the FITS files are included
as part of the data set, and much of the FITS header information is
reproduced in the PDS labels.
Processing
==========
Overview
--------
The data processings are done on DANS (Data Analysis Network System)
managed by C-SODA (Center for Science-satellite Operation and Data
Archive), ISAS/JAXA at Sagamihara, Kanagawa, Japan.
Note that DANS consists of SIRIUS (Scientific Information Retrieval and
Integrated Utilization System), EDISON (Engineering Database for ISAS
Spacecraft Operation Needs), ``Reformatter'', DARTS (Data ARchives and
Transmission System) and data analysis servers.
The summary of the data processing is as follows. The data downlinked as
telemetry are transferred to the data accumulation system at ISAS, and
are registered to SIRIUS and are sorted in time. From SIRIUS, level 0
data are retrieved to the Reformatter for the PLANET-C Project Team that
is provided by C-SODA to each mission. All processings (data pipeline)
after level 0 until level 3 are done on the Reformatter. These
procedures are executed automatically.
To implement these procedures, various program codes are used, written
in C, C++, Fortran 90/95, Ruby, Python, Perl, and POSIX shell.
Definitions of the processing levels
------------------------------------
The definitions of the processing levels for UVI, IR1, IR2, and LIR
instruments are same for simplicity and homogeneity. The processing
levels are defined as follows:
abbr name description
---- -------- ---------------------------------------------------
L0 Level 0 raw data files from telemetry.
L1a Level 1a counts in FITS format.
L1b Level 1b flipped and rotated L1a with summary of geometry
information in FITS format.
L2b Level 2b calibrated physical quantity in FITS format.
L2c Level 2c (more) calibrated physical quantity in FITS format.
L3b Level 3b L2b mapped onto longitude-latitude grids in NetCDF
format that is preprocessed using the limb-fitting
technique for correction of pointing of the camera.
L3c Level 3c L2c mapped onto longitude-latitude grids in NetCDF
format that is preprocessed using the limb-fitting
technique for correction of pointing of the camera.
Detailed description of the processings
---------------------------------------
This section describes detailed procedure at each processing level.
The summarized diagram of the pipeline is shown below:
(1) (2) (3) (4) (5)
SIRIUS -----> L0 -----> L1a -----> L1b -----> L2b -----> L3b
\
\(4) (5)
`-> L2c -----> L3c
(1) L0 data processing
L0 data are consists of three kinds of file, tile of image, common
ancillary data for the image and instrument-specific ancillary data
for the image. The image size of each tile file is 128 x 128 pixels.
These L0 data are retrieved from the SIRIUS. The tile files of image
are usually compressed by HIREW algorithm, so the tile files are
decompressed at the first step. The decompressed tile files are
converted from unsigned 16-bit integer into signed 16-bit integer,
and then concatenating decompressed tile files, the raw image file
is created. If one or more tile files are missing, missing value is
substituted to the corresponding pixels at this processing level.
For LIR images, only valid region of data is cropped after
concatenation of tile files.
If the compressed tile file is identified as corrupted, e.g., the
data size of the tile file is not equal to the data size recorded in
the header of the tile file, only pixels supposed to be valid are
decoded and invalid pixels are filled with missing value.
(2) L1a data processing
From the raw image and ancillary data files in L0, L1a FITS file is
created.
If there are wrong ancillary data, the corrected data are stored in
the header of the HDU of L1a FITS file using other source. This is
currently applied only to UVI images.
Some types of the ancillary data to be stored in the header of the
HDU of L1a FITS file, e.g., value of P_OBSPRG that represents
observation program ID used to acquire image, are automatically
determined by command history files or data files created from HK
(House Keeping) telemetry packets. Note that the command history
files stores the commands sent from the ground to the spacecraft,
and it can be retrieved from EDISON. The data files created from HK
telemetry packets also can be retrieved from EDISON.
For IR1 and IR2 images, command sequences (i.e., observation
program) to acquire image are changed from the planned one at the
instrument design phase to reduce the total time for taking images
as short as possible. This change caused the issue that cannot
uniquely determine both of selected filter and exposure time for
each image only from the telemetry data. This issue is solved by a
script using the executed observation programs extracted from
command history files and the relationship between executed
observation program used for taking each image and known behavior of
the instrument-specific ancillary data that depends on the executed
observation program.
(3) L1b data processing
From L1a FITS file, L1b FITS file and associated geometry
information FITS file are created using SPICE kernels and the SPICE
toolkit.
The orientation of image data become nominal (spacecraft +Y axis is
upward with the coordinate described in 'document/imagecoord.txt')
at this processing by the procedure combining vertical flipping of
the image and some of the 90-degree rotations. This procedure
differs depending on the instruments/channels. What kind of
operation has been done is recorded as value of the P_FLPROT keyword
in the header of the HDU in FITS file. This operations are nominally
done as follows:
cam channels operations P_FLPROT
--- -------- -------------------- --------
UVI all rotate CCW 270 deg 3
IR1 all flip vertically 10
IR2 1.65 um no rotation/flipping 0
IR2 others rotate 180 deg 2
LIR all flip vertically 10
For the purpose of quick look, JPEG image is created for each FITS
image file.
(4) L2b and L2c data processing
From L1b FITS file, L2b and L2c FITS file are created using FITS
files for calibration, depending on the instrument and the channels.
L2c is ``more calibrated'' version than L2b. This processing level
is available for IR1 and LIR at now.
For IR1, handling of pixels with special flag values during smear
correction are different between L2b and L2c. For L2b product, if
the pixels with special flag values appear in the column in a
quadrant, all the values of the pixels at the column in the quadrant
are set to missing value. IR1 L2b product is processed with the aim
of photometric analysis. For L2c product, if the pixels with special
flag values appear in the column in a quadrant, contribution from
the pixels to the total amount of smear is estimated using available
value near the pixel in the same column. IR1 L2c product is
processed with the aim of cloud tracking.
For LIR, in addition to L2b data calibration process, L2c data
processing applies calibration using baffle (hood) temperature. This
requires baffle (hood) temperature acquired from HK, so the product
is not available when HK lacks.
For the purpose of quick look, JPEG image is created for each FITS
image file also at this processing level.
(5) L3b and L3c data processing
From L2b or L2c FITS file, L3b or L3c NetCDF file, and L3bx or L3cx
NetCDF file are created using SPICE kernels and SPICE toolkits. The
fully-automated algorithms of limb-detection, limb-fitting and
interpolation onto equally spaced longitude-latitude grids has been
implemented for the processing.
For verification of the result of limb-detection and limb-fitting,
L3bx or L3cx NetCDF file is also created. This file also contains
correction of pointing of the camera. Using the correction of
pointing of camera, L3bx or L3cx FITS file is created. L3bx and L3cx
FITS files have a similar structure to geometry information file
(``geo'' file), so one can use L3bx or L3cx FITS file as pointing-
corrected geometry information file instead of geometry information
file created at the L1b data processing.
Assumption on the altitude for observed quantity
------------------------------------------------
For geometry calculations, what altitude that the observed quantity is
coming from and includes information on is roughly assumed as follows:
cam channels assumed altitude
--- ------------- ----------------
UVI 283 nm 70 km
365 nm 70 km
IR1 0.90 um day 60 km
0.90 um night 60 km
0.97 um 60 km
1.01 um 60 km
IR2 1.735 um 50 km
2.26 um 50 km
2.32 um 50 km
2.02 um 70 km
1.65 um N/A
LIR pic 65 km
The assumed altitude is recorded in the FITS header keyword, S_CLDALT.
Parameters
==========
The primary data is ADU (analog-digital unit) counts that are nominally
proportional to the number of photons received at the detector (CSD/CCD)
with the applied filter. The ADU counts are nominally proportional to
radiance at the target wavelength corresponds to the applied filter.
In this data set, data in ADU counts are stored without any calibration.
For the filters used in the IR2 instrument, see ``catalog/ir2inst.cat''
for details.
Data
====
Overview
--------
There is only one kind of image data FITS file. Each value in data array
represents value at the center of the pixel. For the definition of the
image coordinate and relationship with indexes of data array, please
refer the ``document/imagecoord.txt'' file. The image data FITS file has
``_l1b_'', ``_l2b_'', or ``_l2c_'' in the filename depending on the
processing level of the file.
File format and structure
-------------------------
The files with the extension ``.fit'' under ``data'' directory are FITS
format file. These FITS files are considered to be compliant with the
FITS Standard Version 4.0 published online on 22 July 2016 by the IAUFWG
except for handling of keywords that stores numeric value. If value of
the keyword is supposed to be ``Not Available'', the value is set to
'N/A' in string literal in spite of definition of the keyword that
stores numeric value. This is a work around for non-existence of numeric
literals that represent special values in FITS Standard Version 4.0.
For each image file, image data is stored in the data part of the
secondary HDU (Header Data Unit) as IMAGE extension. The list of
HDU in the geometry files is as follows:
HDU 0: Primary HDU (header only)
HDU 1: Image data
File naming conventions
-----------------------
The file naming conventions for the data files are same over the four
cameras, UVI, IR1, IR2, and LIR. In addition to the four cameras, the
data of LAC and RS have very similar file naming conventions. In this
section, the file naming conventions for the four cameras are described.
The filename has the format
{cam}_{YYYY}{MM}{DD}_{hh}{mm}{ss}_{filter}_{level}_v{ver}.{ext}
where
{cam}: abbreviated name of cameras, i.e., uvi, ir1, ir2, and lir
Note that for LAC and RS data, lac and rs are used, respectively.
{YYYY}: year in four-digits
{MM}: month in two-digits
{DD}: day in two-digits
{hh}: hour in two-digits
{mm}: minute in two-digits
{ss}: second in two-digits
Note that {YYYY}-{MM}-{DD}T{hh}:{mm}:{ss} means the representative
time of observation for the image in UTC. This time is also stored
in HDU of the FITS file as string value of DATE-OBS keyword.
{filter}: abbreviated filter name or type of image with 3 characters
used to acquire image
For each camera, values of {filter} and corresponding NAIF name of
channel are listed below.
{cam} {filter} NAIF name of channel used to acquire image
----- -------- ------------------------------------------
uvi 283 VCO_UVI_283 or VCO_UVI_283_TOPOB
dif VCO_UVI_DIF or VCO_UVI_DIF_TOPOB
365 VCO_UVI_365 or VCO_UVI_365_TOPOB
sht VCO_UVI_SHT or VCO_UVI_SHT_TOPOB
ir1 09d VCO_IR1_09D
09n VCO_IR1_09N
097 VCO_IR1_097
101 VCO_IR1_101
dif VCO_IR1_DIF
drk VCO_IR1_DRK
ir2 174 VCO_IR2_174
226 VCO_IR2_226
232 VCO_IR2_232
202 VCO_IR2_202
165 VCO_IR2_165
drk VCO_IR2_DRK
lir pic VCO_LIR_PIC
opn VCO_LIR_OPN
sht VCO_LIR_SHT
{level}: abbreviated name of product
The values of {level} is summarized in the following table. Note
that the l3{b,c} and l3{b,c}x products are not provided as part of
PDS3 data set, but will be provided at DARTS at ISAS/JAXA and PDS
``annex''.
{level} corresponding DATA_SET_ID description
-------- ------------------------- ---------------------------
l1b VCO-V-{CAM}-2-EDR-V{VER} uncalibrated count data
l2{b,c} VCO-V-{CAM}-3-CDR-V{VER} calibrated data with
physical dimension
geo VCO-V-{CAM}-3-SEDR-V{VER} geometry data at the center
of each pixel
l3{b,c} not applicable interpolated data projected
onto longitude-latitude
l3{b,c}x not applicable grid data for evaluation of
automated limb-fitting and
pointing corrected data
that can be used instead of
``geo''
{ver}: version string of the product in 2 digits
The value is larger than or equal to 10. When the version number
appeared in DATA_SET_ID is multiplied by 10, the result is equal
to the value of {ver}.
{ext}: extension of the file
The following typical extensions are included in this data set
except for ``nc'' that is only included in l3{b,c} and l3{b,c}x
products. The ``data'' directory contains only ``fit'' and ``lbl''
files, and the ``browse'' directory contains only ``jpg'',
``html'', and ``lbl'' files.
{ext} description
----- ----------------------------------------
fit FITS file based on the FITS Standard 4.0
lbl PDS3 label file
jpg JPEG file
html HTML (mostly compliant with HTML 5.2)
nc NetCDF file in NetCDF-4 Format
Structure and size of image data
--------------------------------
All IR2 images consist of a multiple of tiles. One tile represents
subimage with 128 x 128 pixels. Almost all tiles are compressed by the
lossless data compression algorithm, HIREW. Each compressed tile is
uncompressed to obtain raw tile data, and then raw image data that
consists of multiple tiles is reconstructed by concatenating raw tile
data files. Then the data is stored in FITS file with ancillary data.
Therefore the data has 128*m pixels in horizontal and 128*n pixels in
vertical, where m, n are integer and are within [1, 8].
Special pixels and its expressions in data array
------------------------------------------------
The following three types of special pixels are defined.
terminology description
--------------- -------------------------------------------------
missing pixel Data of the pixel is missing in the telemetry due
to loss of the packets or the override by other
data in the DR (Data Recorder).
dead pixel Detector element of the pixel is damaged
permanently by some reason, e.g., hit of X-rays
to the corresponding detector element.
saturated pixel Data at the pixel that are out of range from the
nominal measurement due to receiving too much
photons to the detector element.
The following special values are defined and are recorded in the header
of the HDU in FITS file.
keyword description
------- ------------------------------------------------------
P_MPIXV Missing pixel flag value.
Data at missing pixels are substituted this value.
P_DPIXV Dead pixel flag value.
Data at dead pixels are substituted this value.
If dead pixels are not known, the value of the pixel
will be 'N/A'.
P_SPIXV Saturated pixel flag value.
Data at saturated pixels are substituted this value.
If saturated pixels are not identified at this
processing level or are expressed using value of the
keyword P_SPIXO, the value of the pixel will be 'N/A'.
P_SPIXO Saturated pixel offset value.
Data at saturated pixels are added this offset value
to the original value of each pixel.
If saturated pixels are not identified at this
processing level or are expressed using value of the
keyword P_SPIXV, the value of the pixel will be 'N/A'.
This keyword is only used for IR2 L2b data.
For verbosity, the following keywords are also used to record number
of special pixels in the data in the header of the HDU.
keyword description
------- ----------------------------------------------
P_MPIXN Number of missing pixels in the stored data.
P_DPIXN Number of dead pixels in the stored data.
P_SPIXN Number of saturated pixels in the stored data.
Ancillary Data
==============
+Y-axis Inversion Events
------------------------
In order to ensure thermal stability of the spacecraft and its
instruments, the spacecraft +Y-axis must be inverted relative to the
solar system plane periodically. The dates of these inversions are as
follows:
Date Events
------------------- --------------
2015-12-09T23:36:00 south to north
2016-02-08T23:16:00 north to south
2016-11-24T05:00:00 south to north
2017-02-01T04:30:00 north to south
2017-11-16T01:00:00 south to north
2018-02-23T03:00:00 north to south
2018-04-04T05:41:00 south to north
2018-10-04T04:00:00 north to south
Geometry information
--------------------
The data on geometry in FITS header were calculated using the VCO SPICE
kernels data set, VCO-V-SPICE-6-V1.0. This data set is available at
DARTS in ISAS/JAXA and the PDS SPICE Archives in JPL/NASA.
Detailed geometry information associated with images in this data set
is provided in VCO-V-IR2-3-SEDR for purpose of reference.
Coordinate System
=================
In Space
--------
Some geometric quantities appearing in the labels and the header of FITS
files are in J2000 coordinates. In this coordinate, the +Z-axis points
northward along the Earth's J2000 rotation axis and the +X-axis points
toward the first point of aries.
On Venus
--------
For Venusian geometry, IAU VENUS that is defined as the planetocentric
coordinate system for Venus is used.
For Image
---------
For all the data stored in the IMAGE extension HDU in FITS files
provided in this data set, +Y-axis of the spacecraft is upward and
-Z-axis of the spacecraft is rightward, when we use the commonly-used
coordinate system with the origin at the lower-left corner of the image,
horizontal axis (or X-axis or 1st axis) oriented from left to right,
vertical axis (or Y-axis or 2nd axis) oriented from bottom to top. This
coordinate system is commonly used in visualization software of FITS
file, such as SAOImage DS9.
The file ``document/imagecoord.txt'' describes detailed descriptions on
the relationships between the image coordinate and indexes of the data
used in this data set.
Software
========
No software is included in this data set. However, the data products,
being FITS files, can be readily accessed by means of a variety of FITS
readers. PLANET-C Project Team members have been successful in using
SAOImage DS9 version 7.6 by Smithsonian Astrophysical Observatory, the
astropy module for Python version 3.0.3 (Python 3.7 has been used
successfully), and the IDL Astronomy Users' Library. Note that for some of
these readers, the existence of multiple HDU in the file must be included
in the issued command.
|
.png)
.png)
