Hyperspectral Thermal Emission Spectrometer
Personal tools
You are here: Home / Documents / File Description
« April 2024 »
April
MoTuWeThFrSaSu
1234567
891011121314
15161718192021
22232425262728
2930
 

File Description

Level 1 (L1) Data File

Example File Name: 20210718t140829_CambridgeshireGB_L1_B100_V01.hdf5
This file contains calibrated HyTES data in radiance units of W/m^2/µm/sr. The data is recorded with a band interleaved by pixel (BIP) format and contains 32-bit floats with the dimensions:

  • 495 samples for data in 2013, 512 samples for data from 2014 and later.
  • A variable number of lines
  • 256 bands from 7.5-12 µm
  • This file also contains locational metadata from the instruments NGDCS

Lastly the file contains per-pixel geolocation information, namely latitude, longitude, height, and number of steps taken during ray-casting:

  • The line dimension of the geolocation is typically slightly less than that of the radiance data. This is due to the slight variation from nadir (as is represented in the camera model), there is not enough geolocation (GPS) information for the last few lines of the scene. Therefore, there is a mismatch of a few lines between the raw scene data and the pixel geolocation data. If one is using the pixel geolocation information with L1A data, the mismatch should be addressed by clipping data from the end of the scene. This will be the case unless the pointing/camera model significantly changes.The min and max lines that can be processed are searched for, prior to data generation, and they are printed out during processing, so it can always be verified in that way.
  • All the specific dimensions for geolocation information for each file are described within the HDF file as well as its associated header (.hdr) file.

L1 Header Files

Example File Name: 20210718t140829_CambridgeshireGB_L1_B100_V01.hdf5.geo.hdr
These text header files specify the parameters of their corresponding .hdf5 file in an human-readable and ENVI-compatible format. Also used in KMZ generation.

L1 Geolocation Dat File

Example File Name: 20210718t140829_CambridgeshireGB_L1_B100_V01.geo.dat
This file contains only the per-pixel geolocation information, namely latitude, longitude, height, and number of steps taken during ray-casting:

  • The line dimension of the geolocation is typically slightly less than that of the radiance data. This is due to the slight variation from nadir (as is represented in the camera model), there is not enough geolocation (GPS) information for the last few lines of the scene. Therefore, there is a mismatch of a few lines between the raw scene data and the pixel geolocation data. If one is using the pixel geolocation information with L1A data, the mismatch should be addressed by clipping data from the end of the scene. This will be the case unless the pointing/camera model significantly changes.
  • The min and max lines that can be processed are searched for, prior to data generation, and they are printed out during processing, so it can always be verified in that way.
  • All the specific dimensions for geolocation information for each file are described within the HDF file as well as its associated header (.hdr) file.

L1 Geolocation Header Files

Example File Name: 20210718t140829_CambridgeshireGB_L1_B100_V01.geo.hdr
These text header files specify the parameters of their corresponding .dat file in an human-readable and ENVI-compatible format.

Level 2 (L2) Products HDF5 File for "Twin Otter" flights only

Example File Name: 2014-07-06.194610.SaltonSea.Line2-Run1-Segment01.L2.hdf5

  • Level 2 Products are not available for all lines, although most lines from 2014 and later are available.
  • This file has several attributes of useful data including 'product_version' and 'acquisition_time'
  • The data are arrays of 32-Bit Float values corresponding to each HyTES pixel and the arrays are shuffled and compressed with GZIP to reduce file size. The HDFs also include Fletcher32 checksums to ensure data integrity. These features are seamlessly supported by standard HDF5 readers.
  • This file contains several arrays of data representing the Level 2 products as described below:

L2 Emissivity Data

  • This array contains HyTES emissivity spectral data from 8-11.5 µm with the same dimensions as the Level 1A data except we only retrieve on 202 TES 'window bands' instead of 256.
  • This is the standard emissivity product derived using the Temperature Emissivity Separation (TES) algorithm (Gillespie et al., 1998) with an In-Scene Atmospheric Correction (ISAC) approach for HyTES channels from 8-11.5 µm (only clear window channels and well-calibrated data). Spectral emissivity from 7.4-8 µm cannot be accurately derived due to strong water vapor absorption features in this spectral domain. TES is not applied to channels above 11.5 µm due to issues with calibration.

L2 PC Regression (PCemis) Emissivity Data

  • This array contains HyTES emissivity data from 7.4-12 µm with same dimensions as the Level 1A data. In order to produce emissivity for all HyTES bands (e.g. to be used as first guess for retrieving methane at 7.6 µm) we generated a separate PCemis product called L2.PCemis that uses a Principal Component (PC) eigenvector regression approach to produce emissivity for all HyTES channels. The eigenvectors were calculated using a set of ~150 lab spectra similar to the approach used by U. Wisconsin to produce the MODBF product (Seemann et al., 2008).
  • The L2.PCemis produces a smoother spectrum that will fit all the original L2.emiss data, and extend this data below 8 micron and above 11.5 micron, but caution should be used when using this emissivity to identify geologic features because some features may be smoothed out, or absent.

L2 Land Surface Temperature Data File

  • This file contains HyTES Land Surface Temperature (LST) data in units of Kelvin with same dimensions as the Level 1A data except only one band (temperature in Kelvin).
  • Level 2 LST Products are derived from atmospherically corrected level-1 radiance data using the TES algorithm.

Level 2 Products HDF5 File for "ER2" flights only

  • This file has several attributes of useful data including 'product_version' and 'acquisition_time'
  • The data are arrays of 32-Bit Float values corresponding to each HyTES pixel and the arrays are shuffled and compressed with GZIP to reduce file size. The HDFs also include Fletcher32 checksums to ensure data integrity. These features are seamlessly supported by standard HDF5 readers.
  • This file contains several arrays of data representing the Level 2 products as described below:

Level 2 Emissivity Data (L2_Emissivity)

  • This array contains HyTES emissivity spectral data from 8.2-11.5 µm with the same dimensions as the Level 1A data except we only retrieve on 186 TES 'window bands' instead of 256.
  • This is the standard emissivity product derived using the Temperature Emissivity Separation (TES) algorithm (Gillespie et al., 1998) with an In-Scene Atmospheric Correction (ISAC) approach that is elevation-dependent based on a clustering approach for HyTES channels from 8.2-11.5 µm (only clear window channels and well-calibrated data). Spectral emissivity from 7.4-8 µm cannot be accurately derived due to strong water vapor absorption features in this spectral domain. TES is not applied to channels above 11.5 µm due to issues with calibration.

Level 2 PC Regression Emissivity Data (L2_Emissivity_PC)

  • This array contains HyTES emissivity data from 7.4-12 µm with same dimensions as the Level 1A data. In order to produce emissivity for all HyTES bands (e.g. to be used as first guess for retrieving methane at 7.6 µm) we generated a separate PCemis product called L2_Emissivity_PC that uses a Principal Component (PC) eigenvector regression approach to produce emissivity for all HyTES channels. The eigenvectors were calculated using a set of ~150 lab spectra similar to the approach used by U. Wisconsin to produce the MODBF product (Seemann et al., 2008).
  • The L2_Emissivity_PC produces a smoother spectrum that will fit all the original L2_Emissivity data, and extend this data below 8 micron and above 11.5 micron, but caution should be used when using this emissivity to identify geologic features because some features may be smoothed out, or absent.

Level 2 Land Surface Temperature Data File

  • This file contains HyTES Land Surface Temperature (LST) data in units of Kelvin with same dimensions as the Level 1A data except only one band (temperature in Kelvin).
  • Level 2 LST Products are derived from atmospherically corrected level-1 radiance data using the TES algorithm.

Level 3 CMF Products HDF5 File

Example File Name: 2014-07-06.194610.SaltonSea.Line2-Run1-Segment01.L3-CMF.hdf5

  • Level 3 CMF Products are not available for all lines. Availability depends on the detection of different chemical gas species: CH4, H2S, NH3, NO2, and SO2
  • For each of the gas types there is a group that has a binary flag indicating if that gas was detected for the specific line. For example: "CH4/CH4_Found" would be set to True if CH4 was detected or False if it was not (the first "CH4" indicates the group). If found there will also be two arrays named "CH4/CH4_CMF1" and "CH4/CH4_CMF2".
  • CMF 1 is the raw CMF data computed directly from the radiances, and CMF 2 is the thresholded CMF which uses interquartile range statistics to identify plume pixels, in addition to a plume dilation and contiguity algorithm to enhance any detected plume pixels. This procedure is described in detail in Hulley et al. 2016.
  • This file has several attributes of useful data including 'product_version' and 'acquisition_time'
  • The data are arrays of 32-Bit Float values corresponding to each HyTES pixel and the arrays are shuffled and compressed with GZIP to reduce file size. The HDFs also include Fletcher32 checksums to ensure data integrity. These features are seamlessly supported by standard HDF5 readers.
  • Additional details here: http://hytes.jpl.nasa.gov/downloads/HyTES_L3_Data_Guide_gch_nv_kl-1-gch-sjhv1.pdf

Level 3 QR Products HDF5 File

This data is described here: https://hytes.jpl.nasa.gov/downloads/HyTES_L3_Data_Guide_gch_nv_kl-1-gch-sjhv1.pdf

Image Files

L1A Image Preview

This is a PNG image preview of the data as an RGB composite of bands 150 (10.1 µm), 100 (9.2 µm), and 58 (8.5 µm) using the aforementioned Level 1A radiance data.
Example File Name: 2014-07-06.194610.SaltonSea.Line2-Run1-Segment01-110000-level_1a.rgb150100058.png

L2 Emissivity Preview

  • This is a PNG image preview of the data as an RGB composite of bands 150 (10.1 µm), 100 (9.2 µm), and 58 (8.5 µm) using the aforementioned Level 2 emissivity data.
  • Level 2 Products are not available for all lines, though most lines from 2014 onward have them.
    Example File Name: 2014-07-06.194610.SaltonSea.Line2-Run1-Segment01.L2-emis.png

L2 Land Surface Temperature Preview

  • This is a PNG image preview of the data as an RGB composite of bands 150 (10.1 µm), 100 (9.2 µm), and 58 (8.5 µm) using the aforementioned Level 2 temperature data.
  • Level 2 Products are not available for all lines, though most lines from 2014 onward have them.
    Example File Name: 2014-07-06.194610.SaltonSea.Line2-Run1-Segment01.L2-LST.colorbar.png

L3 CMF2 Preview

  • This is a PNG image preview of the data CMF2 data for a given gas type (CH4, H2S, NH3, NO2, and SO2).
  • The detected plumes are represented in different colors (e.g. CH4 = green, NH3 = red) with intensity of the color corresponding to the strength of the CMF detection, while the background is a black and white presentation of Level 1 radiance data from band 58 (8.5 µm).
  • Level 3 Products are not available for all lines.
    Example File Name: 2014-07-06.194610.SaltonSea.Line2-Run1-Segment01.L3-CMF2-CH4.png

Google Earth (GE)

GE Flight Path

This Google Earth file shows the flight path for the data acquisition. Notably, the altitude it also recorded in this file.
Example File Name: 2014-07-06.194610.SaltonSea.Line2-Run1-Segment01-110000.kml

GE Geo-Corrected Preview

This file, also for Google Earth, is only available for acquisitions from June 24th, 2013 and later. It shows a geo-corrected image overlay of the segment, again using the RGB radiance composite of bands 150 (10.1 µm), 100 (9.2 µm), and 58 (8.5 µm).
Example File Name: 2014-07-06.194610.SaltonSea.Line2-Run1-Segment01-GoogleEarth.kmz

GE Geo-Corrected L2 Land Surface Temperature Preview

  • This file, also for Google Earth, shows a geo-corrected image overlay of the segment, using the LST temperature color map.
  • Clicking on the filename in the left column of Google Earth will show the colorbar key.
  • Level 2 Products are not available for all lines, though most lines from 2014 onward have them.
    Example File Name: 2014-07-06.194610.SaltonSea.Line2-Run1-Segment01-GoogleEarth.L2-LST.kmz

Google Earth Geo-Corrected L3 CMF 2 Preview

  • This file, also for Google Earth, shows a geo-corrected image overlay of the segment, using the CMF2 image preview.
  • Level 3 Products are not available for all lines.
  • Available for individual gasses or as a multi-layer KMZ with toggle-able Gas Layers and L1 Background.
    Example File Name: 2014-07-06.194610.SaltonSea.Line2-Run1-Segment01-GoogleEarth-L3-CMF2-All.kmz

Geological Location CSV files

These CSV files give the Latitude, Longitude, and Altitude coordinates of each pixel recorded in the associated image.
Example File Name: 2014-07-06.194610.SaltonSea.Line2-Run1-Segment01-110000-geoLat.csv

Geological Location Data File

This .dat file contains the Latitude, Longitude, and Altitude coordinates of each pixel recorded in the associated image. This can be used with ENVI for orthorectification and georegistration.
Example File Name: 2014-07-06.194610.SaltonSea.Line2-Run1-Segment01-110000-geo.dat

References:

Gillespie, A., Rokugawa, S., Matsunaga, T., Cothern, J. S., Hook, S., and Kahle, A. B. (1998).
"A temperature and emissivity separation algorithm for Advanced Spaceborne Thermal Emission and Reflection Radiometer (ASTER) images."
IEEE Transactions on Geoscience and Remote Sensing, 36, 1113-1126.

Seemann, S. W., Borbas, E. E., Knuteson, R. O., Stephenson, G. R., and Huang, H. L. (2008).
"Development of a global infrared land surface emissivity database for application to clear sky sounding retrievals from multispectral satellite radiance measurements."
Journal of Applied Meteorology and Climatology, 47, 108-123.

Hulley, G. C., Duren, R. M., Hopkins, F. M., Hook, S. J., Vance, N., Guillevic, P., Johnson, W. R., Eng, B. T., Mihaly, J. M., Jovanovic, V. M., Chazanoff, S. L., Staniszewski, Z. K., Kuai, L., Worden, J., Frankenberg, C., Rivera, G., Aubrey, A. D., Miller, C. E., Malakar, N. K., Sánchez Tomás, J. M., and Holmes, K. T.
"High spatial resolution imaging of methane and other trace gases with the airborne Hyperspectral Thermal Emission Spectrometer (HyTES)"
Atmos. Meas. Tech. Discuss., doi:10.5194/amt-2016-8, in press, 2016.

General steps for ENVI for orthorectification and georegistration for sample 2015-02-01.190838.Cuprite.Line3-Run1-Segment01

In ENVI open the Lat/Long data: 2015-02-01.190838.Cuprite.Line3-Run1-Segment01-110000-geo.dat

Note you must have the 2015-02-01.190838.Cuprite.Line3-Run1-Segment01-110000-geo.hdr file in the same folder, this tells ENVI the format of the file

In ENVI open the data you want to register: 2015-02-01.190838.Cuprite.Line3-Run1-Segment01-110000-level_1a.dat

Again you need the associated .hdr file if this is a .dat file. Alternatively you could use the LST PNG, emissivity data, or similar visual here as long as it's the same width and height as the original image.

Create the GLT in ENVI following these general instructions:

https://www.l3harrisgeospatial.com/docs/GeoreferenceFromInputGeometry.html#Building

Generally you can accept the default values in this step however you need to make sure that you select WGS-84 as the datum. When selecting the bands for the X and Y coordinates, use the notes from the geo metadata to guide you. One says "use for y" and one says "use for x".

In ENVI, use the GLT created in the previous step to create a georeferenced image following these general instruction:

https://www.l3harrisgeospatial.com/docs/GeoreferenceFromInputGeometry.html#Georefer

Again you can accept most of the default values. You'll just need to select what bands you want to use. For our L1 visuals I use band 150 (10.1 µm) for Red, band 100 (9.2 µm) for Green, and band 58 (8.5 µm) for blue. This step can take a while especially with a large image.

Once the processing on the last step is complete you'll have a georeferenced image in ENVI. ENVI also has a tool called something like "GLT and Georeference" that combines the GTL and georeference steps which you might find useful (less clicking).