AAPP Updates and Bugfixes

Updates and Bugfixes

This page documents known bugs (and fixes) for AAPP v8, OPS-LRS v8, AAPP v7 and OPS-LRS v7. For a list of bugs and fixes for older versions of AAPP, please see the archived page.

Questions for other users and developers can be posted on the AAPP Forum.

The reported bugs and problems do not always cause harm. Especially the portability problems depend on the hardware platform, compiler version, option settings used. It is generally not necessary to implement all such modifications into your system. It is recommended to follow just the instructions of the Installation Guide and to consult this list only in case of problems. Nevertheless, programming bugs and algorithm improvements should be implemented.

List of Known Bugs and Problems of AAPP v8 (applicable to the current version)

  1. On some 64-bit systems the size of the default integer in Fortran is 8 bytes; AAPP requires 4 bytes. The behaviour can be different for different compilers. For example, a system has been reported in which g95 gives 8 byte integers but gfortran and gcc give 4-byte integers. gfortran is preferred in this case, but if you want to use g95 then we recommend the use of the “-i4” compiler flag – set this in Makefile.ARCH, and also use the flag if you are building the GMT example executables (which don’t use Makefile.ARCH).
  2. sdh2orbnum.exe is sensitive to spaces in the input date/time and gives wrong answers if you get it wrong. The solution is always to use the wrapper script sdh2orbnum.
  3. The AAPP installation script generally uses the latest version of ecCodes. From ecCodes v2.25.0 onwards, the ecCodes build by default uses the AEC library (for Adaptive Entropy Coding in GRIB processing), and the build fails if the library is not present. Since this functionality is not needed by AAPP, the October 2022 version of install_aapp8.sh tests whether the library is available and if it is not then it adds the flag -DENABLE_AEC=OFF when configuring ecCodes.
  4. Some modifications are needed when running MAIA with NOAA-21 VIIRS data from CSPP v4. Affected routines are jpss_meta_root_io.F90 and maia_read_Viirs.F90. Corrected versions have been posted in miscellaneous and will be included in the next minor release.

List of Known Bugs and Problems of OPS-LRS v8 (applicable to the current version)

  1. On some 64-bit systems the size of the default integer in Fortran is 8 bytes; OPS-LRS requires 4 bytes. This issue affects the C to Fortran calls in OPS-LRS and can cause run-time errors: the solution is to set the appropriate flag in ARCH_FFLAGS_BASE in Makefile.ARCH. If the build fails, try optimisation level 0 (work-around for a g95 compiler problem).
  2. Bad data at the start of a pass can cause OPS-LRS to fail. One solution is to configure the reception system to start acquiring data at a higher elevation. Another is to try the scripts iasi_l0_qc.ksh and skip_mdr_advice.pl which make use of EUMETSAT’s tool kai to interrogate and modify the IASI L0 file. From AAPP v8.9 you can invoke this test by including the “-k” option in AAPP_RUN_METOP. If you use this tool (whether successfully or not), please report your findings to the NWP SAF Helpdesk.
  3. Successive runs of OPS-LRS, with the same input data, do not give identical results in some fields (e.g. geolocation and AVHRR clusters). Likely related to the multi-threading.
  4. In section 6 of the OPS-LRS User Manual it is explained that when running two instances of OPS-LRS simultaneously they need to use different ports. It has been found that if there is a port conflict then the second process aborts (with a correct diagnostic message) but the TES__ServeurTemps task is not always automatically killed and can consume large amounts of computer memory. In OPS-LRS v8.4 a check on port availability has been included in the perl wrapper.
  5. When building with gfortran, OPS-LRS attempts to link the library libgfortranbegin.a. This library is obsolescent and is not included in some newer GCC installations. The install_aapp8.sh script has been modified to automatically adjust the Linux-gfortran config file.
  6. OPS-LRS v8.4 uses netstat (part of net-tools) to check port availability. If netstat is not installed on your system, you will see a warning message when running OPS-LRS. This is not in itself a problem, but if a port becomes unavailable then OPS-LRS will fail (see item 4 above). If you have administrator privilege you should be able to install net-tools in the usual way, e.g. “yum install net-tools -y”.
  7. OPS-LRS v8.4 gives a compilation error when built with the default mode of gcc v11, which uses the c++17 standard. However, it should be possible to suppress this error by using the compiler flag “-std=c++11”. Modify your config/Linux-gfortran file and add this flag the line that defines ARCH_CPPFLAGS_NORMAL. Alternatively, if you have already run “configure” then edit your Makefile.ARCH in the same way. You should then be able to run “make” as normal.

List of Bugs and Problems found in earlier versions of AAPP v8

  1. For decoding NOAA-20 CrIS Full-Spectral-Resolution BUFR files in AAPP, you need to use the ecCodes interfaces. However, this fails because some “allocate” statements are missing. Please go to directory src/tools/eccodes/libecbufr and edit the files eccodes_get_1c_cris.F and eccodes_get_1c_cris.F. In each file, find the first “allocate” statement and add 2 lines containing allocate(fovs(nsubsets)) and allocate(fors(nsubsets)). Then re-build in the usual way using “make lib; cd ../bin; make bin”. Fixed in AAPP v8.4.
  2. Sometimes eccodes_encodebufr_1c fails because it is trying to encode values that are outside the ranges permitted by the BUFR definition, e.g. because of bad data at the start of a pass. We recommend that you set the following environment variable so that the program is more tolerant. You can put this in ATOVS_ENV8 or in your calling script:
    export ECCODES_BUFR_SET_TO_MISSING_IF_OUT_OF_RANGE=1

    3. Update (July 18): due to a bug in ecCodes this doesn’t always work: if a variable that is constant within a message (e.g. some IASI status flags) is out of range, the encoding will still fail. ECMWF are working on it. Further update (Oct 18): this is fixed in ecCodes release 2.9.0. Also, changes have been made in AAPP v8.4 to reduce the risk of this problem happening.

  3. In AAPP v8.1, there are two spurious lines in the NOAA-20 entry in file atms_channels.dat (in AAPP/src/tools/libaapphdf5). Please delete the two lines that contain the phrase “Dummy line”. Then re-install into AAPP/data/preproc using the command “make dat”. (Note that the install_aapp8.sh script dated 23/02/2018 will do this correction for you). Fixed in v8.2
  4. In AAPP v8.1, both BUFR decoding programs (aapp_decodebufr_1c and eccodes_decodebufr_1c) fail for NOAA-20 ATMS BUFR files, with an error reading atms_channels.dat. To fix it, please edit eccodes_get_1c_atms.F and aapp_get_1c_atms.F. In each case, look for the line that says “read(30,*) id” and insert the following immediately afterwards:
    line="     "
    Then re-compile with “make lib; cd ../bin; make bin”. Fixed in v8.2.
  5. If you are processing historical NOAA-16 data with atovin, the wrong satellite ID is assigned. Applies to AAPP v8.1 to v8.4. To fix it, edit your fdf.dat file and either remove the final NOAA-21 entry or change its “NESDIS satid” (the first number) to a number that is greater than 14. This number is not actually used in AAPP, but the original value (2) clashes with NOAA-16. A similar problem occurs when processing HIRS/2 and MSU from pre-NOAA11 satellites; to be fixed in next release (or contact the NWPSAF Helpdesk if you need it sooner). Fixed in v8.5.
  6. For HIRS calibration the default (HCALIB_TT_FLAG=1) works correctly but one of the alternatives (HCALIB_TT_FLAG=2) does not, because of a missing channel loop in h_intercept_algoV4.F. Fixed in v8.5.
  7. For AAPP v8.3 and 8.4 there is a build failure on Ubuntu in iasi-tools, when hdf5 libraries are included in the build. Tests suggest that it can be fixed by editing file iasi-tools/Makefile.proj and changing line 2 to the following: 
    CLIBS := -L$(TOPDIR)/../AAPP/lib -latov -lf7nl1b -lauxaapp -L$(TOPDIR)/../metop-tools/lib -leps_metopl0 -leps_common $(CLIBS)

    Then run make as normal. Fixed in v8.5.

  8. Due to ageing of the S-NPP ATMS scan mechanism, it has been observed that some of the quality flags are being set intermittently. In particular, QF9_GRAN_HEALTHSTATUS has been frequently set after 25th Feb 2020. This is reflected in the AAPP granule quality flag – “granuleLevelQualityFlags” in the l1c BUFR products. The BUFR decoders in AAPP reject the flagged data. If you uses these BUFR decoders it is recommended that you modify eccodes_get_1c_atms.F at lines 419 and 437 and change “.le. 2” to “.le. 6”. Similarly, if you use the old BUFRDC decoder, modify aapp_get_1c_atms.F at line 187, changing “.LE.2” to “.LE.6”. Then re-compile. Fixed in AAPP v8.6
  9. If you build a fresh installation of AAPP and immediately (within 24 hours) run the test cases, the avh_get_vis_coefs script will not run automatically because it thinks the files are up to date. A solution is provided in the 12/08/2020 update of install_aapp8.sh.
  10. AAPP versions up to 8.7 do not compile under GCC 10. There were two issues fixed in v8.8: (i) Several routines in AAPP/src/tools/bin/ attempt to perform division by hexadecimal constants (“BOZ literal constants”), which is not permitted. The constants need to be wrapped with INT(). (ii) In metop-tools/include/common-aapp.h, variables aapp_avh1bhd_addr and aapp_avh1bdt_addr need to be declared as extern. Some more GCC 10 issues were addressed in AAPP v8.10.

List of Bugs and Problems found in AAPP v7.x

  1. Orbit numbers derived from TLEs from EUMETSAT are not always consistent with orbit numbers from NOAA TLEs. Fixed in update 7.2.
  2. Bad ATMS BTs (>330K) can be included in the spatial filtering process, corrupting the good BTs. Fixed in update 7.2.
  3. In v7.2, the scripts atovin and atovin_antcorr can fail if there is a file in the working directory whose name is a single character long, e.g. “a”, “b”, etc. To fix the problem, insert single quotes round the [a-z] and [A-Z] at line 48 of atovin.ksh (and line 52 of atovin_antcorr.ksh), i.e. ‘[a-z]’ ‘[A-Z]’. Fixed in update 7.3.
  4. In v7.3, the mwts_sdr and mwhs_sdr utilities fail for FY-3B. Fixed in update 7.4.
  5. For level 1c CrIS BUFR encode/decode, the scaling factors for surface height and satellite height are incorrect. Fixed in update 7.4. But note that the surface height is currently the ellipsoid-geoid separation, not the land surface height. This will be addressed at a later date.
  6. in v7.1-7.3, compilation of lec_previ_grib_api.F fails for g77, as the “trim” function is not supported. Fixed in 7.4.
  7. MAIA2.1 threshold files are not correctly extracted under Cygwin, since Windows does not permit use of the “:” character in a file name. Fixed in update 7.5.
  8. Metop-B AVHRR channel 4 and 5 radiance calibration is wrong, due to formatting error in avhcal.txt. Fixed in update 7.5.
  9. Metop-B AMSU-A channel 3, 4, 5 and 8 calibration differs from EUMETSAT calibration by ~0.1K. Fixed in update 7.5.
  10. When using tool noaa_class_to_aapp to ingest AVHRR data from satellites that were operating before the year 2000, the year is not decoded correctly. Fixed in 7.7.
  11. When decoding EUMETSAT’s global stream of BUFR data for AMSU-A, MHS and HIRS, the first scan after crossing the equator (northbound) does not get included in the output. See Forum posting for more details. Fixed in 7.7
  12. In v7.7, the iras_sdr tool for decoding FY-3 IRAS data works for FY-3C but not for FY-3B. The problem was related to string comparisons. Fixed in v7.8.
  13. The mapping of MWTS2 to IRAS (introduced in v7.7) needs optimising, e.g. the angle tolerances. Fixed in v7.8.
  14. atovin can report an array overflow error in AAPP v7.11 and 7.12, following the introduction of NEDT information. Please see Forum posting for details on how to correct inhhdr.F, inahdf.F, inbhdr.F, inmhshdr.F and inmhsget.F. Fixed in AAPP v7.13.

List of Bugs and Problems found in earlier versions of OPS-LRS

  1. The OPS-LRS script ops_process fails with perl_5.18 and later versions. The solution is either (i) use 5.16 or earlier (available from CPAN) or (ii) modify file run/OPS/perl/lib/IASI/OPS.pm: at line 113, add an extra set of brackets around the qw(), i.e. (qw( DIR_NAVIGATION DIR_IASICONFIG ))
  2. OPS-LRS v7.0+p12 fails on Ubuntu (12.04) with a buffer overflow in SD_FRW__ServeurDonnees. A workaround is to edit file src/CMN/src/CMN__OPSFile.cc, and insert the following after line 936 (i.e. before the sprintf statement):
    // bug fix P. Roquet P. Brunel fails on ubuntu 12.04 en gcc -O2
    m_edSoft = 0;
    m_revSoft = 0;
     Then re-build OPS-LRS.
  3. If you are using perl v5.26 or later, you will need to modify the configure script: change “config” to “./config” at lines 151 and 174. Update (Sept 2021): the install_aapp8.sh script has been modified to automatically make this change for you. Fixed in OPS-LRS v8.4.