Title: MODIS Ocean Filenames, Structures, and useful metadata
1MODIS OceanFilenames, Structures, and useful
metadata
- Kay Kilpatrick
- University of Miami/RSMAS
2Most FAQWhy does MODIS oceans have so many files
and why is everything so BIG!
- 1km global resolution every day
- Many more products
- Meeting the needs of a diverse user community
(pleasing all the people all the time) - Pretty pictures/quick looks Medium quality, low
resolution - Tracking fronts and features all pixels, high
resolution - High quality climate data High quality, medium
resolution - Modelers high quality, various projections not
maps - Algorithm developers underlying raw values
3Where to get data and more information
- Information locations
- MODIS Oceans home page
- http//modis-ocean.gsfc.nasa.gov
- MODIS Oceans QA Browse 36km Imagery (MQABI)
- http//jeager.gsfc.nasa.gov/browsetool/
- Select Terra collection 4
- Useful links to documentation and related web
pages - http//modis-ocean.gsfc.nasa.gov/doclinks.html
- Data Ordering locations
- NASA GES DAAC WHOM (NASA - Goddard DAAC)
- http//daac.gsfc.nasa.gov/MODIS/
- WHOM, datapool, multi parameter subsetting
- EOS DATA GATEWAY EDG
- http//modis.gsfc.nasa.gov/data/ordering.html
- JPL PODAAC 4km global SST only
- http//podaac.jpl.nasa.gov/modis
4MODIS Ocean Products
- MODIS Instruments
- Terra (1030 morning), MO
- Aqua (1330 afternoon) MY
- Resolution
- Spatial
- Level 2 - 1km swath , 2030km x 1354km
- Level 3 - 4km, 36km, 1 deg all products are
global - Temporal Resolution
- Level 2 - 5 minute granule
- Level 3 - daily, 8 day week, monthly, yearly
- Other data sets
- - SST matchups database (kkilpatrick_at_rsmas.miami.
edu) - - ocean color diagnostic data sites
5MODIS Ocean data products
- There are 86 ocean parameters available in over
100 categories of MODIS Ocean data types
archived by (and may be obtained from) the NASA
Goddard Distributed Active Archive Center. - The three basic groupings of MODIS ocean data
parameters are - ocean color
- sea surface temperature
- ocean primary production
6Parameters
- 36 Ocean Color parameters
- 3 Sea Surface Temperature parameters
- 8 Primary Productivity parameters
- (including 2 Primary Production indices)
- 38 Quality Control parameters.
7Processing levels
- Level 1 - Unprocessed top of the atmosphere
radiance/reflectance - At 1-km spatial resolution
- 5 minute granule time resolution 288
granules/day - Level 2 swath data
- At 1-km spatial resolution
- 5 minute granule time resolution color 144/day,
SST 288/day - Level 3 global binned or mapped data
- spatial resolutions of 4.63km, 36km, or 1 degree
- Time resolutions of one day, 8 days, a month or a
year. - The binned ISEAG grid
- The SMI mapped cylindrical Equidistant
Projection - Centered at 0,0.
8Level 4 Productivity
- Ocean primary production data is available only
as binned or mapped Level 4 (i.e. L4) data. - Ocean Productivity outputs are averaged weekly or
yearly. Like the L3 data, the L4 data is
organized spatially as either 4km ISEAG gridded
bins or as maps using a Cylindrical Equidistant
Projection. The mapped data products are
available in a choice of 4km , 36km, or 1 degree
spatial resolutions. More than one model is used
for deriving these data products and some quality
statistics are available.
9L2 filenames2 night and 6 day ocean files for
each 5minutes
- MDCL2 - water leaving radiance products
- MDCL2A - derived products group 1
- MDCL2B- derived products group 2
- MDOCQC - ocean color QC parameters
- MD28L2 -SST products
- MD28QC- SST QC parameters
- File size 80-100MB/file
- Y for AQUA or O for Terra
10Water leaving radiance products
- MDOCL2 water leaving radiance (parameters 1-12)
- 7 bands nLw412,nLw443,nLw488,nLw531,nLw551,nLw667
,nLw678 - Tau 865 Aerosol Optical thickness 865
- Epsilon 765/865
- Aerosol model 1
- Aerosol model 2
- Epsilon of clear water
11Derived products group 1
- MDOCL2A- 13 products (parameters 13-25)
- CZCS_pigment Chlorophyll-a pheopigment
(fluorometric,empirical) - Chlor_MODIS Chlorophyll-a concentration (HPLC,
empirical) - Pigment_c1_total Total pigment concentration
(HPLC, empirical) - Chlor_flur_ht Chlorophyll fluorescence line
height - Chlor_fl_base Chlorophyll fluorescence baseline
- Chlor_fluor_effic Chlorophyll fluorescence
efficiency - Susp-solids-conc Total suspended matter
concentration in ocean - Cocco_pigment_conc Pigment concentration in
coccolithophore blooms - Cocco_conc_detached Detached coccolithophore
concentration - Calcite_conc Calcite concentration
- K_490 Diffuse attenuation coefficient at 490 nm
- Phycoeryth_conc Phycoerythrobilin concentration
- Phycou_conc Phycourobilin concentration
-
12Derived products group 2
- MDOCL2A -11 products (parameters 26-36)
- chlor_a2 Chlorophyll-a concentration (SeaWiFS
analog) - chlor_a3 Chlorophyll-a concentration
(semianalytic) - ipar Instantaneous photosynthetically available
radiation - arp Instantaneous absorbed radiation by
phytoplankton for fluorescence - absorp_coef_gelb Gelbstoff absorption
coefficient at 400 nm - chlor_absorb Phytoplankton absorption
coefficient at 675 nm - tot_absorb_412 Total absorption coefficient at
412 nm - tot_absorp_443 Total absorption coefficient at
443 nm - tot_absorb_488 Total absorption coefficient at
488 nm - tot_absorb_531 Total absorption coefficient at
531 nm - tot_absorb_551 Total absorption coefficient at
551 nm
13SST L2 products
- M28L2 - 3 products (parameters 36 to 40)
- SST_D1 - Sea surface temperature (daytime), 11um
- SST_D2 - Do not use Sea surface temperature
(daytime), 4 um - SST_N1 - Sea surface temperature (nighttime),
11um - SST_N2 -Sea surface temperature (nighttime), 4um
14L2 Ocean color QC file
- MDOCQC -
- U_Wind m/s
- V_Wind m/s
- Pressure mBar
- Humidity kg/m2
- Ozone dobson
- Latitude degree
- Longitude degree
- SolarZenith angle
- SolarAzimuth angle
- SatelliteZenith angle
- SatelliteAzimuth angle
- nLw670 W/m2/um/sr
- Aerosol radiance 765 W/m2/um/sr
- Rayleigh radiance 443 W/m2/um/sr
- Glint radiance W/m2/um/sr
- Whitecap radiance W/m2/um/sr
15L2 SST QC file
- MD28QC
- D1,N1 Channel 20 brightness temperature degrees
C - D2.N2 Channel 22 brightness temperature degrees
C - D3,N3 Channel 23 brightness temperature degrees
C - D4,N4 Channel 31 brightness temperature degrees
C - D5,N5 Channel 32 brightness temperature degrees
C - D6,N6 Channel 20 radiance W/m2/um/sr
- D7,N7 Channel 22 radiance W/m2/um/sr
- D8,N8 Channel 23 radiance W/m2/um/sr
- D9,N9 Channel 31 radiance W/m2/um/sr
- D10,N10 Channel 32 radiance W/m2/um/sr
16L3 Binned global files
- Binned files MDOCD,W,M,Yprod.parameter
- MD28D,W,M,Yprod.parameter
- Only available at 4km resolution
- intergerized sinusoidal equal area grid (ISEAG)
- Only bins with data values are present
- land bins and bins with no data are not in the
files - Each files is self contained with sums, weights
and quality levels,and all flags - Useful if you need to do statistic or your own
averaging/compositing algorithms. - File size 640MB
17L3 mapped files
- Various time and space resolutions
- Cylindrical equidistant projection
- All bins present, including fill values (255) for
land and missing data - 8 map files for each product
- Mean, Count,Stdev, Common flags,Quality
flags,Flag byte 1, Flag byte 2, Flag byte 3 - Files sizes 4km 67MB, 36km1MB,1deg .2MB
18File formats
- All Ocean products are in EOS HDF format
- Values are stored as counts
- Scaling information to convert from counts to
real numbers is located in the file metadata for
each parameter (SDS) - Scale_type
- Slope
- Intercept
19Pixel Quality Flags
QLlt3
QL0
- Each product contains all values at all levels of
quality you must filter the data for your
application using the quality flag. - Each pixel is associated with a quality level
stored as a 2 bit value (0,1,2,3) in the
quality SDS. - The position of the quality level within the byte
is given in the metadata attributes for each
parameter (Quality_Bits). - Example MODCL2A Chlor_MODIS quality is in bit
position 11-12, while Chlor_Fluor_ht is in
position 19-20. (note 1-based numbering
convention)
20Quality level definitions
- There are four quality levels
- 0good
- 1questionable/suspect
- 2 sun glint or possible cloud contaminated
- 3bad, thick cloud or other failure
- These levels are derived by evaluating various
combinations of threshold tests common to all
products and specific to individual products.
Other sets of flags,known as common and product
specific flags, are used to store the results of
these threshold tests.
21MODIS OceanFlags, quality control, and masks
Part 2
- Kay Kilpatrick
- University of Miami/RSMAS
22Flags
- The Ocean products are designed to meet the needs
of a variety of users. These flags are key to
controlling the quality and coverage of the data
for your application. - Quality flags
- Common flags
- Product specific flags
23How are they stored in the files
All flags are stored as data type byte Quality
SDS -Quality levels are stored at 2 bits Level
3- 1 byte per file only for that
product integer value of 0,1,2,3,or 255 Level
2 - maximum of 3 bytes per files, must decode the
bits Common flags SDS - each test is 1 bit,
Level 23 1 byte per file L2_flags SDS -
product specific tests each test is 1 bit Level
23 max 3 byte per file
24Example metadata
ncdump -h MODOCL2.A2001191.1605.003.2001312071143
dimensions Number of recordsSwath
2030 Number of samples per recordSwath
1354 variables short nLw_412(Number
of recordsSwath, Number of samples per
recordSwath) nLw_412Long_name
"Normalized water-leaving radiance at 412 nm"
nLw_412Scale_type "y Slope
x Intercept" nLw_412Slope
0.001f nLw_412Intercept
-5.f nLw_412Name "Radiance"
nLw_412Units "W/m2/um/sr"
nLw_412Quality_Bits "1-2"
25Common flags
- The common flags are tests that are the same for
every product The results of these tests are
stored in the common_flag SDS as type byte. - Bit 1 Pixel not processed
- Bit 2 Atmospheric correction failed
- Bit 3 Satellite Zenith angle gt45
- Bit 4 Solar Zenith angle gt70
- Bit 5 Shallow water
- (lt5km from coast or lt50m deep,or inland lake)
- Bit 6 Sun glint threshold or cloud 678
homogeniety exceed - Bit 7 Invalid or missing ancillary data
- Bit 8 Land (include ephemeral water)
26Common flag metadata
byte common_flags (Number of recordsSwath,
. common_flagsUnits "bits"
common_flagsf01_name "UNPROC"
common_flagsf02_name "ATMCOR"
common_flagsf03_name "SATZ"
common_flagsf04_name "SOLZ"
common_flagsf05_name "SHALLOW"
common_flagsf06_name "GLINT"
common_flagsf07_name "SUPPDATA"
common_flagsf08_name "LAND"
27L2_flags radiance byte 1 of 2 (nLw_412, nLw_443,
nLw_488, nLw_531, nLw_551, nLw_667, nLw_678,
Tau_865, Eps_78, aer_model1, aer_model2)
28L2_flags radiance byte 2 of 2 (nLw_412, nLw_443,
nLw_488, nLw_531, nLw_551, nLw_667, nLw_678,
Tau_865, Eps_78, aer_model1, aer_model2)
29L2_flags derived products group 1 byte 1 of
3(eps_clr_water, CZCS_pigment, chlor_MODIS,
pigment_c1_total, chlor_fluor_ht,
chlor_fluor_base, chlor_fluor_effic,
susp_solids_conc, cocco_pigmnt_conc,
cocco_conc_detach, clacite_conc, K_490,
phycoeryth_conc, phycou_conc)
30L2_flags derived products group 1 byte 2 of 3
(eps_clr_water, CZCS_pigment, chlor_MODIS,
pigment_c1_total, chlor_fluor_ht,
chlor_fluor_base, chlor_fluor_effic,
susp_solids_conc, cocco_pigmnt_conc,
cocco_conc_detach, clacite_conc, K_490,
phycoeryth_conc, phycou_conc)
31L2_flags derived products group 1 byte 3 of 3
(eps_clr_water, CZCS_pigment, chlor_MODIS,
pigment_c1_total, chlor_fluor_ht,
chlor_fluor_base, chlor_fluor_effic,
susp_solids_conc, cocco_pigmnt_conc,
cocco_conc_detach, clacite_conc, K_490,
phycoeryth_conc, phycou_conc)
32L2_flags derived group 2 byte 1 of 2 (chlor_a_2,
chlor_a_3, ipar, arp, absopr_coef_gelb,
chlor_absorb, tot_absorb_412, tot_absorb_443,
tot_absorb_488, tot_absorb_531, tot_absorb_551)
33L2_flags derived group 2 byte 2 of 2 (nLw_412,
nLw_443, nLw_488, nLw_531, nLw_551, nLw_667,
nLw_678, Tau_865, Eps_78, aer_model1, aer_model2)
34L2_flags SST byte 1 of 2 (SST_D1, SST_N1,
SST4_N2)
35L2_flags SST products byte 2 of 2 (SST_D1,
SST_N1, SST4_N2)
36Product specific flag metadata
L2_flagsName "one bit per flag, flag 1 is in
the least significant bit" L2_flagsUnits
"bits" L2_flagsf01_name
"blend" L2_flagsf02_name
"package" L2_flagsf03_name
"IparArp_In" L2_flagsf04_name
"hi_wind" L2_flagsf05_name
"Cloudy" L2_flagsf06_name
"chl_hipackage"
L2_flagsf07_name "NDT_package1"
L2_flagsf08_name "NDT_package2"
..
37Other useful metadata
- Processing control
- INPUTPOINTER
- CorVer
38Metadata Processing control
Provides information on various externally
controlled thresholds and settings
Processing Control "REFTYP
1AER_MODEL Quadratic/ Linear 3d MET1
.. SUNGLINT1 0.005SOLZEN1
70.0SATZEN1 63.0HIGHTAU1 10.00CLDICE1
2.50COCCOLITH1 11.00, 8.10, 1.00,
2.00, 1.00, 1.60, 0.95, 1.50EPSILON1
0.67, 1.50TURBIDW1 3.420MSKFLG
LAND1NOAERO 0RHO_WN_MAX 0.010GlintSC
2.280 2.280 2.280 2.280 2.280 2.260
2.260 2.270 2.270 2.250 2.250LgCalc
0LgSSTc 0.000 0.000 0.000 0.000HighAOI
0USEPOL 1USEAVG 1DOCORR
1DOAVGCH1 2LAC SGLINTTH
0.017POLENAB 1,1,1,1,1,1,1,1,1RHO_T_MAX
5.000USEF0VAR
39Metadata INPUTPOINTER
List all files input to the processing for a
given L2 granule Level 3 lists only the first l2
granule that when into the global file
OBJECT INPUTPOINTER\n", "
NUM_VAL 75\n", " VALUE
(\"modis_radcor_v12_56.col.hdf\",
\"MOD28L2.A2001073.2000.004.2002183150615.hdf\",
\"MOD28QC.A2001073.2000.004.2002183150615.h df\",
\"MOD021KM.A2001073.2000.003.2001183210423.hdf\",
\"MOD03.A2001073.2000.00 3.2001183190535.hdf\",
\"S200107312_NCEP.MET\", \"S200107318_NCEP.MET\",
\n", \"S200107400_NCEP.MET\",
\"S200107212_EPTOMS.OZONE\", \"S200107rayleigh_mod
is_667_iqu3.hdf\", \"rayleigh_modis_678_iqu3.hdf\"
, \ \"new_modis_pol_corr5a.hdf\",
\"new_modis_pol_ corr5a.hdf\")\n", ..
40Metadata CORVER Radcor version notes
CorVer "v12_56 Inter-detector corrs from few
grans in each calib Epochs Bands 8-12 calibrated
wrt time trends using modal analysis of Hawaii
time series, adjusted absolutely using MOBY data
Cross-scan and mirror-side corrs using the
flat-field rough cal of 6xx bands via a filtered
modes (flh consistent) LIMITATIONS need
cross-scan per detector mirrors-side corrs not
stable per Epoch inter-detector trends in Band
15,16 still reflected in Bands 8-14 6xx
detectors not stable enough"
41Where to get data and more information
- Information locations
- MODIS Oceans home page
- http//modis-ocean.gsfc.nasa.gov
- MODIS Oceans QA Browse 36km Imagery (MQABI)
- http//jeager.gsfc.nasa.gov/browsetool/
- Select Terra collection 4
- Useful links to documentation and related web
pages - http//modis-ocean.gsfc.nasa.gov/doclinks.html
- Data Ordering locations
- NASA GES DAAC WHOM (NASA - Goddard DAAC)
- http//daac.gsfc.nasa.gov/MODIS/
- WHOM, datapool, multi parameter subsetting
- EOS DATA GATEWAY EDG
- http//modis.gsfc.nasa.gov/data/ordering.html
- JPL PODAAC 4km global SST only
- http//podaac.jpl.nasa.gov/modis
42nLw quality
Mandatory Quality for all of Gordon's nLw
parameters 0 good, if common flags are clear,
and product-specific L2_flags 1-4 and 9-16 are
clear. 1 questionable, if large satellite or
solar zenith angles 2 cloud or sun glint
contaminated 3 bad other than cloud, if any
input radiances are negative or saturated , or
Atmospheric Correction failed, aerosol
model16,or Land
43eps_clr water Quality
Mandatory Quality Carder's eps_clr_water
parameter 0 good, if common flags, input Lw
flags and product specific L2_flag 15 is clear.
1 questionable, if any of shallow, large
zenith angles, bad ancillary data, or
product-specific L2_flag 15 is set. 2 cloud or
sun glint contaminated 3 bad, if any input
radiances are negative or saturated, or
Atmospheric Correction failed, or Land
44Chlor_Modis Quality
Mandatory Quality for Clark Chlor_MODIS 0
good, if pigc between 0.01 and 100 mg/m3 1
questionable, if high satellite zenith angle ,
shallow water, bad ancillary data, data values
lt0.01 or gt100 mg/m3 2 Sun glint
contaminated 3 input radiances are negative or
saturated, cloudy, aersol model16, land or
atmospheric correction problem, pig_c lt0.0
mg/m3
45chlor_fluor_ht Quality
Mandatory Quality for Abbott's chlor_fluor_ht
parameter ( quality level demoted by 1 if high
sat or solar zenith angle) 0 good, if common
flags except shallow are clear, input Lw flags
are clear, product specific flags clear 1
questionable, L2_flags 7 (FLH/CHL too large) or
L2_flag 9 (FLH too large) 2 if FLH/Chl way
too large, FLH way too large, or modis_chl-1
3 bad, if input radiances are negative or
saturated , cloudy/albedo, atmospheric
correction problem, glint, bad ancillary data,
or land
46Coccolith quality
Mandatory Quality for Gordon/Balch
cocco_pigmnt_conc, cocco_conc_detach,
calcite_conc 0 good, if common flags are
clear, input Lw flags are clear, and
product-specific L2_flags clear. 1
questionable, if any of shallow, large zenith
angles, bad ancillary or product flags 15-17 2
cloud, if any input radiances are negative and
saturated, cloud, or glint 3 bad, if any
input radiances are negative or saturated, or
Atmospheric Correction failed, aersol
model16, Land, or L2_flags 22-24 are set
47CZCS Quality
Mandatory Quality for Clark's CZCS_pigment
parameter 0 good, if pigc between 0.01 and 100
mg/m3 1 if any high satellite zenith angle ,
shallow water, bad ancillary data, or data
vales lt0.01 or gt100 mg/m3 2 Sun glint
contaminated 3 bad, if input radiances are
negative or saturated , glint, cloudy, aer0sol
model16, land or atmospheric correction
problem , pig_c lt0.0 mg/m3
48Pigment_c1_total Quality
Mandatory Quality for Clark's pigment_c1_total 0
good, if pigc between 0.01 and 200 mg/m3 1
questionable if any high Sateillite zenith angle,
shallow water, bad ancillary data, data vales
lt0.01 or gt200 mg/m3 2 Sun glint
contaminated 3 bad, if any input radiances are
negative or saturated, glint, cloudy, aersol
model16, land or atmospheric correction
problem , pig_c lt0.0 mg/m3
49Susp_solids_conc Quality
Mandatory Quality for Clark's susp_solids_conc
parameter 0 good, if values between 0.03 and
100 g/m3 1 questionable, susp_solid lt0.03 or
gt 100 g/m3, high satellite or solar zenith
angle, shallow water, bad ancillary data 2 sun
glint contaminated 3 bad, if any input
radiances are negative or saturated, or
AtmosphericCorrection failed, aersol
model16, or susp_solid less than 0.03 or
greater than 100 g/m3, Land, cloudy
50K_490 Quality
Mandatory Quality for Clark's K_490 parameter 0
good, K490 between 0.016 and 20.0 1
questionable, if any values lt0.016 or gt 20.0 ,
high satellite or solar zenith angle, shallow
water, or bad ancillary data 2 glint
contaminated 3 bad, if any input radiances are
negative and or saturated, or Atmospheric
Correction failed, aerosol model16, k490 lt0.0
, Land, or cloudy
51chlor_a_3, absorp_coeff_gelb, chlor_absorb,tot_abs
orb_ Quality
Mandatory Quality for Carder's chlor_a_3,
absorp_coeff_gelb, chlor_absorb, tot_absorb_ 0
good, if common flags are clear, input Lw flags
are clear, and product-specific L2_flags 1,9-16
are clear 1 questionable, if any of shallow,
large satellite or solar zenith angles, bad
ancillary, aphi675 too small, or low 412 or 555
flag set 2 glint contaminated 3 bad, if any
input radiances are negative or saturated, or
Atmospheric Correction failed, or Land, aerosol
model16, RRS lt0
52IPAR Quality
Mandatory Quality Carder's IPAR 0 good, if
common flags are clear, input Lw flags are
clear, and product-specific L2_flags clear 1
questionable, if any of shallow, large zenith
angles, bad ancillary 2 cloud or glint
contaminated 3 bad, if any input radiances are
negative or saturated, or Atmospheric Correction
failed, or Land, or aerosol model16
53ARP Quality
Mandatory Quality Carder's ARP parameter 0
good, if common flags are clear, input Lw flags
are clear, and product-specific L2_flags is
clear 1 questionable, if any of shallow,
large satellite or solar zenith angles, bad
ancillary, or high windspeed flag set 2 cloud
or glint contaminated 3 bad, if any input
radiances are negative or saturated, or
Atmospheric Correction failed, or Land, or
aersol model16, or RRSlt 0
54Chlor_a2 Quality
Mandatory Quality for SeaWiFs analog OC3M
Chlorophyll chlor_a2 0 good, if pigc between
0.01 and 100 mg/m3 1 Sat Zenth angle gt70,
shallow water, bad ancillary data, data values
lt0.01 or gt100 mg/m3 2 Sun glint
contaminated 3 input radiances are negative or
saturated, glint, cloudy, aersol model16, land
or atmospheric correction problem, pig
5511-12um SST Quality
Mandatory Quality for SST products 0 good, if
common flags are clear, input radiance flags are
clear, and product-specific L2_flags 9,11-16 are
clear 1 questionable, large zenith angle,
bad ancillary or flags 3,10-11 2 sun glint
or channel uniformity bad contaminated 3 if
any input radiances are negative and not
saturated, or Atmospheric Correction failed, or
Land
56SST4 Quality
Mandatory Quality for SST4 products 0 good,
if common flags are ok, inputs, both sat zenith
tests, both sst4-sst, BT22-BT23-ref, both
uniformity, brights, and reynolds(optional) 1
questionable, if any of if sst4-sst test2 , sat
zenith test 2, or unif test2 2 if sst4-sst
test 1, sat zenith angle test 1, or uniformity
test1 3 if any input radiances are negative
and not saturated, BT22-BT23-ref out of range,
brights outside (-4..33)
57Science Quality flag and Explanation
ScienceQuality_ltproduct short namegt
(updates in .met only)
Valids Passed Failed Inferred Passed Inferred
Failed Being Investigated (default in
.hdf) Suspect Hold