############################################################################## # # # ATOVS and AVHRR Preprocessing Package (AAPP) # # Version 7.7 Release Notes # # 27 Aug 2014 # # # ############################################################################## 1. Package contents ------------------- The following files are contained in this update release (with file sizes in bytes): RELEASE_NOTE_AAPP_7_7.txt 14758 This release note AAPP_update_7_7.tgz 1692431 AAPP source code This is an incremental update; users need to have downloaded and unpacked version 7.1 (the initial release of AAPP v7) and updates 7.2, 7.3, 7.4, 7.5 and 7.6. The following documents are also updated: NWPSAF-MF-UD-002 (Software) NWPSAF-MF-UD-003 (Data Formats) These documents are available from the AAPP web page (see www.nwpsaf.org). 2. Main Changes --------------- - Tools to process FY-3C sounder data, including decoding of the CMA hdf5 files, spatial averaging for the microwave sounders, re-mapping and BUFR encoding. See section 6 for details. - Add the new IASI 500 channel selection, which will be used by EUMETSAT (in EARS-IASI and GTS global feed) from September 2014. See section 7 for details. - NOAA-15 AMSU-A calibration update, allowing for antenna position sensor degradation. Sections 6 and 7 give details on the new functionality. Section 8 gives a full list of changes from v7.6. 3. Installation --------------- To update an existing AAPP installation, the procedure is: a. Copy the tgz files to your AAPP top directory (i.e. the directory containing AAPP, metop-tools and iasi-tools) b. Unpack using the command tar -xvmzf AAPP_update_7_7.tgz or if your system does not support the "-z" option in tar, use gunzip -c AAPP_update_7_7.tgz | tar -xmf - c. Go to the AAPP_7 top directory and type make clean make The reason for the "make clean" is that that some array sizes have been increased in order to accommodate the extra IASI channels. If you are not starting from an existing installation, the sequence is (i) unpack AAPP v7.1 and cd to the AAPP top directory, (ii) unpack the update releases 7.2, 7.3, 7.4, 7.5, 7.6, 7.7 (as in steps a-b above), (iii) run "configure" (see chapter 3.4 of the AAPP Installation Guide), (iv) run "make" and optionally "make install", (v) install MAIA data files if required. 4. License ---------- To use this software, users need to have registered for AAPP v7 with the NWP SAF (www.nwpsaf.org), and to have agreed to the terms of the license agreement. 5. Version control information ------------------------------ The source package (AAPP_update_7_7.tgz) corresponds to Subversion Revision 410 in the AAPP repository held at the Met Office. 6. Tools for working with FY-3C data ------------------------------------ For more details on these tools, please see the Software Manual. As usual, before calling any of these scripts it is recommended to first source the $AAPP_PREFIX/ATOVS_CONF file as described in Section 3.7 of the AAPP Installation Guide. For the BUFR tools you need to define the environment variable BUFR_TABLES, i.e. export BUFR_TABLES= #with trailing slash The following scripts are provided: i) mwts2_sdr, mwhs2_sdr, iras_sdr Convert sensor data records (SDR) in hdf5 to AAPP internal binary format (.l1c). See the Data Formats document (or the relevant "include" file) for format details. The hdf5 files are output from the CMA direct broadcast processing package, and are also available on the CMA web portal. Usage: mwts2_sdr [-o mwtsfile.l1c] mwtsfile.hdf ii) aapp_encodebufr_1c Encode the FY-3 l1c files into BUFR. This is an extension of existing capability (e.g. AMSU, IASI, ATMS, CrIS). You may wish to define the MASTER_TABLE and BUFR_EDITION environment variables (default 13 and 4 respectively). Two BUFR sequences are available for FY-3. The first is designed for international data exchange, e.g. for dissemination of the data by EUMETSAT. The instruments are encoded separately. aapp_encodebufr_1c -i mwtsfile.l1c MWTS2 aapp_encodebufr_1c -i mwhsfile.l1c MWHS2 aapp_encodebufr_1c -i irasfile.l1c IRAS At present, a "Table D" sequence number has not yet been specified; therefore the descriptors are explicitly written into the Data Description Section of the message. The second BUFR sequence allows for mapped instruments, i.e. "level 1d" (e.g. MWTS2 and MWHS2 mapped to IRAS), and was primarily devised for use at the Met Office: aapp_encodebufr_1c -i mwtsfile.l1c MWTS21D aapp_encodebufr_1c -i mwhsfile.l1c MWHS21D aapp_encodebufr_1c -i irasfile.l1c IRAS1D Note that the ".l1c" formats for FY-3 sounders include slots for mapped instruments, i.e. there is no specific "l1d" format. The BUFR sequence is defined in AAPP/src/tools/libaappbufr/define_fy3_generic_1d.F. The encoding program contains options for thinning the data, via environment variables MWTS2_THIN, MWHS2_THIN. These default to 1 (every spot), but may be set to n (1 spot in n, 1 line in n) or -n (1 spot in n, every line). Typically n=3. The reason for different cross-track and along-track thinning is that MWTS2 raw samples are closer together in the cross-track direction (16km at nadir) than the along-track direction (35km). iii) aapp_decodebufr_1c Decode a BUFR file, back to l1c format. Currently available only for the first BUFR sequence. aapp_decodebufr_1c -i mwtsfile.bufr MWTS2 aapp_decodebufr_1c -i mwhsfile.bufr MWHS2 aapp_decodebufr_1c -i irasfile.bufr IRAS iv) mwts2_beamwidth, mwhs2_beamwidth These tools allow you to spatially average the data, usually to reduce noise. mwts2_beamwidth mwtsfile.l1c [outfile] mwhs2_beamwidth mwhsfile.l1c [outfile] The output file is optional; if not specified then the input file is over-written. The beamidth specification is provided in files mwts2_beamwidth.dat and mwhs2_beamwidth.dat, in $DIR_PREPROC. You can use the environment variables MWTS2_BEAMWIDTH_FILE and MWHS2_BEAMWIDTH_FILE to point to your own files. v) mwhs2_to_mwts2, mwts2_to_mwhs2, mwts2_to_iras, mwhs2_to_iras Each l1c format provides slots for mapped instruments (e.g. MWTS2 l1c has slots for 15 MWHS2 channels, and IRAS l1c has slots for 28 MWTS2+MWHS2 channels). The general usage is program target_file file_to_map e.g. to add MWHS2 information to an existing MWTS2 file: mwhs2_to_mwts2 mwtsfile.l1c mwhsfile.l1c 7. IASI channel selection and PC processing ------------------------------------------- The file IASI.fdf (normally installed in $DIR_PREPROC) has been updated with the new channel selection, with 500 channels. New channels have been added - none have been removed. The previous selection is still available, as file IASI_366.fdf, as is the earlier IASI_314.fdf. You can use an environment variable IASI_FDF to point to the file that you want to use, and an environment variable DIR_IASI_PREPROC if you want to put the files in a different directory. It is sometimes useful to keep Metop-A and Metop-B IASI processing files in separate directories. The channel selection version number is set to 5 for the new selection. The IASI l1d format has been expanded to accommodate the extra channels. If you are using EUMETSAT's Principal Component BUFR product for Metop-B (*pcs_l1.bin), note that the BUFR files for Metop-B have the version number set to 101 in the BUFR. If you want to use them in AAPP, you need to copy the Metop-A eigenvector files (supplied with AAPP v7.1) and rename to IASI_EV[1,2,3]_xx_M01_20110125000000Z_xxxxxxxxxxxxxxZ_20110119000101Z_xxxx_xxxxxxxxxx i.e. with the version 101 in the file name. Then modify iasi_eigenvectors_spec.dat to point to these new eigenvector files. 8. Detailed list of changes from v7.6 (r387) to v7.7 (r410) ------------------------------------------------------------- Key: M=Modified, A=Added, D=Deleted M AAPP/include/iasi1d.h M AAPP/include/iasipc.h M AAPP/src/preproc/libatovpp/IASI.fdf A AAPP/src/preproc/libatovpp/IASI_366.fdf IASI channel selection - additional channels taking the total to 500. M config/g95 Add g95 compiler flag "-i4". On some 64-bit systems, the default integer size is 8 bytes with the g95 compiler, but 4 bytes with gfortran. AAPP requires 4 bytes. (Also affects OPS-LRS). M AAPP/src/navigation/libnavtool/stations.txt M AAPP/src/navigation/libnavtool/Makefile A AAPP/src/navigation/libnavtool/orbit_angle.F Update to stations.txt from EUMETSAT. Tool to calculate angle relative to ecliptic plane. M AAPP/src/navigation/libtle/tle_sdc.F M AAPP/src/navigation/libtle/tle_dc.F Make TLE routines compatibile with older historic TLE bulletins M AAPP/src/navigation/libtle/Makefile D AAPP/src/navigation/libtle/tmp.F Remove unwanted tmp.F M AAPP/include/iras1c.h A AAPP/include/mwhs21c.h A AAPP/include/mwhs2_sdr.h A AAPP/include/mwhs2chandata.h A AAPP/include/mwts21c.h A AAPP/include/mwts2_sdr.h A AAPP/include/mwts2chandata.h M AAPP/src/tools/libaapphdf5/iras_sdr_read.c M AAPP/src/tools/libaapphdf5/iras_sdr_out.F M AAPP/src/tools/libaapphdf5/mwhs_channels.dat M AAPP/src/tools/libaapphdf5/mwts_channels.dat A AAPP/src/tools/libaapphdf5/mwhs2_sdr.intf.c-F A AAPP/src/tools/libaapphdf5/mwhs2_sdr_out.F A AAPP/src/tools/libaapphdf5/mwhs2_sdr_read.c A AAPP/src/tools/libaapphdf5/mwts2_sdr.intf.c-F A AAPP/src/tools/libaapphdf5/mwts2_sdr_out.F A AAPP/src/tools/libaapphdf5/mwts2_sdr_read.c M AAPP/src/tools/libaapphdf5/Makefile A AAPP/src/tools/bin/mwhs2_sdr.c A AAPP/src/tools/bin/mwhs2_sdr.ksh A AAPP/src/tools/bin/mwts2_sdr.c A AAPP/src/tools/bin/mwts2_sdr.ksh M AAPP/src/tools/bin/Makefile M AAPP/src/preproc/libatov/Makefile A AAPP/src/preproc/libatov/iomwhs21c.F A AAPP/src/preproc/libatov/iomwts21c.F Add FY-3C sounders HDF5 decode (MWTS2, MWHS2, IRAS) M AAPP/src/tools/libaappbufr/aapp_put_1c.F M AAPP/src/tools/libaappbufr/aapp_get_1c.F A AAPP/src/tools/libaappbufr/aapp_get_1c_mwhs2.F A AAPP/src/tools/libaappbufr/aapp_get_1c_mwts2.F A AAPP/src/tools/libaappbufr/aapp_put_1c_mwhs2.F A AAPP/src/tools/libaappbufr/aapp_put_1c_mwts2.F A AAPP/src/tools/libaappbufr/aapp_put_1d_iras.F A AAPP/src/tools/libaappbufr/aapp_put_1d_mwhs2.F A AAPP/src/tools/libaappbufr/aapp_put_1d_mwts2.F A AAPP/src/tools/libaappbufr/define_fy3_generic_1d.F M AAPP/src/tools/libaappbufr/Makefile M AAPP/src/tools/bin/aapp_encodebufr_1c.F M AAPP/src/tools/bin/aapp_encodebufr_1c.ksh Add FY-3C sounders BUFR encode/decode, 1c and 1d. M AAPP/src/tools/libaappbufr/aapp_put_1d_iasi.F M AAPP/src/tools/libaappbufr/aapp_put_1d_hirs.F M AAPP/src/tools/libaappbufr/aapp_put_1d_amsub.F M AAPP/src/tools/libaappbufr/aapp_put_1c_iras.F M AAPP/src/tools/libaappbufr/aapp_put_1c_mwts.F M AAPP/src/tools/libaappbufr/aapp_put_1c_mwhs.F Round time to nearest second before encoding, to avoid seconds values of 60.0. M AAPP/src/tools/libaappbufr/aapp_get_1c_amsua.F M AAPP/src/tools/libaappbufr/aapp_get_1c_amsub.F M AAPP/src/tools/libaappbufr/aapp_get_1c_hirs.F Allow output when scan number is zero (applies to EUMETSAT Metop data at equator) M AAPP/src/tools/bin/avhrr_lac_class_to_aapp_a-j.F M AAPP/src/tools/bin/avhrr_gac_class_to_aapp_a-j.F Allow for cases when time code year is negative; GAC files have 2 header records; set avh_h_startdatajd. M AAPP/src/tools/libsatid/satid.txt Add FY3C. Correct international designation for NOAA-7 and NOAA-B M AAPP/src/calibration/libmhscl/mhs_iwttmp.F M AAPP/src/calibration/libmhscl/mhs_smpmn.F M AAPP/src/calibration/libmhscl/mhs_cal.F M AAPP/src/calibration/libmhscl/mhssetu.F M AAPP/src/calibration/libmhscl/mhs_status.F M AAPP/src/preproc/libatovpp/surfelev.F M AAPP/src/preproc/libatovin/inmhsget.F M metop-tools/src/libeps_avhrrl1b_6.5/convert_avh1b.F Portability fixes: size of ICHAR is machine dependent and is not necessarily the size of the default integer. Allow re-calibration of Metop-B NOAA files with MHS instrument ID >100. M AAPP/src/calibration/libamsuacl/amasetu.F M AAPP/src/calibration/libamsuacl/ama_updt.F M AAPP/src/calibration/libamsuacl/amsua_clparams.dat Further degradation of NOAA-15 AMSU-A1-1 antenna position sensor. Errors of 180 deg now occur, so relax the position checking. Also, fix re-calibration of NOAA-15 (pre-existing quality flags were still being tested for channels 7/15). Assume navigation is OK if you are just running a re-calibration. M AAPP/src/preproc/libatovpp/Makefile A AAPP/src/preproc/libatovpp/map_mwhs2_to_iras.F A AAPP/src/preproc/libatovpp/map_mwhs2_to_mwts2.F A AAPP/src/preproc/libatovpp/map_mwts2_to_iras.F A AAPP/src/preproc/libatovpp/map_mwts2_to_mwhs2.F A AAPP/src/preproc/libatovpp/mwhs2_beamwidth.dat A AAPP/src/preproc/libatovpp/mwts2_beamwidth.dat M AAPP/src/preproc/bin/Makefile A AAPP/src/preproc/bin/mwhs2_beamwidth.F A AAPP/src/preproc/bin/mwhs2_beamwidth.ksh A AAPP/src/preproc/bin/mwhs2_to_iras.F A AAPP/src/preproc/bin/mwhs2_to_iras.ksh A AAPP/src/preproc/bin/mwhs2_to_mwts2.F A AAPP/src/preproc/bin/mwhs2_to_mwts2.ksh A AAPP/src/preproc/bin/mwts2_beamwidth.F A AAPP/src/preproc/bin/mwts2_beamwidth.ksh A AAPP/src/preproc/bin/mwts2_to_iras.F A AAPP/src/preproc/bin/mwts2_to_iras.ksh A AAPP/src/preproc/bin/mwts2_to_mwhs2.F A AAPP/src/preproc/bin/mwts2_to_mwhs2.ksh Mapping routines for FY-3C sounders M AAPP/src/preproc/bin/atovpp.ksh M AAPP/src/preproc/bin/iasi_ev_readhdf5.c Correct eigenvector handling for Metop-B. M AAPP/gmt/avhrr_l1b_calib_plot.com M AAPP/gmt/l1d_plot.com M AAPP/gmt/avhrr_l1b_calib_plot.F M AAPP/gmt/l1d_plot.F M AAPP/gmt/l1d_plot_tovs.F M AAPP/gmt/l1d_plot_atovs.F M AAPP/gmt/l1d_plot.ps M AAPP/gmt/avhrr_l1b_calib_plot.ps Update the GMT example scripts to work with Metop data, and use the Metop-A test case as input. Also some portability fixes.