Error running RTTOV14 tests

Viewing 6 posts - 1 through 6 (of 6 total)
  • Author
    Posts
  • #50714
    Matt PagetMatt Paget
    Participant

    Hi,
    I’m building a docker image in our system, for use by @atprata, and I have an error running the RTTOV14 tests.

    # Command
    ./test_rttov14.sh ARCH=gfortran-openmp

    # Error

     TEST_ID        C     P  S   L EBRAHG         GASES         PORTSCAPI BRPRLTQLFSNMIS    I    O T    M    N CPTAFIDTAKBTAT  REAL TIME   USER TIME STATUS
     
    amsre/401        72    6 1  70 X.X..2 o3                    20.31X..4 XX.XXXX.X..321    1    1 0    1    1 .....XXXXX....                         FAIL
    
    		    | STOP 1
    		    |  2025/05/28  02:36:29  fatal error in module ../../src/test/rttov_test.F90:1035
    		    |	   Cannot read
    		    | /tmp/build/rttov_test/test_rttov14.1.gfortran-openmp/amsre/401/out/rttov_test.txt:
    		    | Cannot match namelist object name %f_._skin
    		    |  2025/05/28  02:36:29  fatal error in module ../../src/test/rttov_test.F90:0280

    I temporarily patched rttov_test.F90 (line 1034 in original) to add READ(..., IOMSG=msg) so that the above prints the extra “Cannot match namelist object name %f_._skin” message. This error is repeated for all the tests (just showing the first one here).

    Any suggestions, or any checks/tests I can do to help chase this one down please?
    Thank you,
    Matt

    ===========================
     RTTOV COMPILATION SUMMARY
    ===========================
    
    Compiling with flags           : gfortran-openmp
    Compiling in directory         : ./
    
    RTTOV features available:
    NetCDF coefficient I/O         : y
    Emissivity/BRDF atlases        : y
    C/C++ wrapper                  : y
    Python wrapper and RTTOV GUI   : y
    
    Compiling with user LAPACK lib : n
    
    Regenerating Makefiles using:
    $ ../build/Makefile.PL RTTOV_NETCDF=1 RTTOV_F2PY=1 RTTOV_USER_LAPACK=0

    Environment
    – Ubuntu 24.04.2 LTS
    – GNU Fortran 13.3.0
    – GNU Make 4.3
    – binutils 2.42
    – perl v5.38.2
    – netCDF 4.9.2
    – netcdf-fortran 4.6.2
    – python 3.12.7 / numpy 2.2.6

    #50721
    James HockingJames Hocking
    Keymaster

    Hi Matt,

    This is a new one: there have been no such problems with the test suite noted with all the various compilers/systems that we have tested RTTOV on over the years.

    I think the problem is either in the way the namelist (rttov_test.txt) is being generated by the rttov_test.pl Perl script (so maybe something to do with the version of perl), or it’s an issue with the Fortran-compiled rttov_test.exe executable reading the namelist (so something to do with the version of gfortran).

    It works on my system with gfortran v13.2.0 and perl v5.32.1. I suspect it is a perl-related issue.

    Can you copy/paste the contents of /tmp/build/rttov_test/test_rttov14.1.gfortran-openmp/amsre/401/out/rttov_test.txt so that I can compare it to the same file on my system?

    It’s worth noting that this error is specific to the test suite and is not indicating a problem with RTTOV itself. You can do a basic check that RTTOV is working by running one of the example programs (e.g. the script run_example_fwd.sh – run it in the same way as the test_rttov14.sh script) as these examples are independent of the test suite.

    Best wishes,
    James

    #50726
    Matt PagetMatt Paget
    Participant

    Hi James,
    Thank you for your attention and response to this.

    The run_example_fwd.sh script works (TEST SUCCESSFUL). I tried downgrading gfortran to 13.2 but 13.3 is a security patch so hard to untangle it with apt (I suspect I would have to start from a different base and disable noble-security apt source). If the rttov_test.txt file below is different to yours then I’ll try downgrading perl.

    Cheers,
    Matt

    # test_rttov14.1.gfortran-openmp/amsre/401/out/rttov_test.txt
    &rttov_test_nml
    defn%opts%config%verbose = .TRUE.
    defn%opts%config%apply_reg_limits = .FALSE.
    defn%opts%config%check_profiles = .TRUE.
    defn%opts%config%transmittances_only = .FALSE.
    defn%opts%config%bt_overcast_calc = .FALSE.
    defn%opts%config%gas_opdep_calc = .TRUE.
    defn%opts%config%adk_bt = .TRUE.
    defn%opts%config%adk_refl = .TRUE.
    defn%opts%config%opdep13_gas_clip = .TRUE.
    defn%opts%interpolation%enable_interp = .TRUE.
    defn%opts%interpolation%pressure_gradients = .FALSE.
    defn%opts%interpolation%interp_mode = 4
    defn%opts%rt_all%solar = .FALSE.
    defn%opts%rt_all%rayleigh_single_scatt = .TRUE.
    defn%opts%rt_all%nlte_correction = .FALSE.
    defn%opts%rt_all%refraction = .TRUE.
    defn%opts%rt_all%plane_parallel = .FALSE.
    defn%opts%rt_all%use_t2m = .TRUE.
    defn%opts%rt_all%use_q2m = .TRUE.
    defn%opts%rt_all%rad_down_lin_tau = .TRUE.
    defn%opts%rt_all%rayleigh_max_wavelength = 2
    defn%opts%rt_all%rayleigh_min_pressure = 0
    defn%opts%surface%solar_sea_refl_model = 1
    defn%opts%surface%ir_sea_emis_model = 2
    defn%opts%surface%mw_sea_emis_model = 3
    defn%opts%surface%use_foam_fraction = .FALSE.
    defn%opts%surface%lambertian = .FALSE.
    defn%opts%surface%lambertian_fixed_angle = .TRUE.
    defn%opts%surface%use_tskin_eff = .FALSE.
    defn%opts%clw_absorption%permittivity_param = 2
    defn%opts%clw_absorption%clw_cloud_top = 322
    defn%opts%scatt%radar = .FALSE.
    defn%opts%scatt%rayleigh_multi_scatt = .FALSE.
    defn%opts%scatt%chou_tang_mod = .FALSE.
    defn%opts%scatt%zero_hydro_tlad = .FALSE.
    defn%opts%scatt%thermal_solver = 3
    defn%opts%scatt%solar_solver = 1
    defn%opts%scatt%baran_ice_version = 2
    defn%opts%scatt%dom_nstreams = 8
    defn%opts%scatt%dom_accuracy = 0
    defn%opts%scatt%dom_opdep_threshold = 0
    defn%opts%scatt%chou_tang_factor = 0.05
    defn%opts%scatt%mw_pol_mode = 1
    defn%opts%scatt%ice_polarisation = 1.4
    defn%opts%cloud_overlap%per_hydro_frac = .FALSE.
    defn%opts%cloud_overlap%hydro_frac_tlad = .TRUE.
    defn%opts%cloud_overlap%overlap_param = 0
    defn%opts%cloud_overlap%col_threshold = 0
    defn%opts%cloud_overlap%two_col_max_frac_max_p = 750
    defn%opts%dev%no_opt_param_tladk = .FALSE.
    defn%do_direct = .TRUE.
    defn%do_tl = .TRUE.
    defn%do_ad = .TRUE.
    defn%do_k = .TRUE.
    defn%do_k_bf = .FALSE.
    defn%do_k_tl = .FALSE.
    defn%do_k_ad = .FALSE.
    defn%do_taylor = .FALSE.
    defn%taylor_by_chan = .FALSE.
    defn%taylor_on_btrefl = .FALSE.
    defn%calc_diag_output = .FALSE.
    defn%calc_rad2 = .FALSE.
    defn%calc_emis_terms = .FALSE.
    defn%user_check_opts = .FALSE.
    defn%user_check_prof = .FALSE.
    defn%input_gas_units = -1
    defn%run_gas_units = 2
    defn%do_print = .TRUE.
    defn%print_opts = .FALSE.
    defn%print_profiles = .FALSE.
    defn%print_quality = .FALSE.
    defn%lallchans = .FALSE.
    defn%temp_alloc = .FALSE.
    defn%over_alloc = .FALSE.
    defn%ntimes = 1
    defn%nthreads = 0
    defn%mult = 1
    defn%scale_inc = 1
    defn%scale_out = 1
    defn%f_lprofiles = ‘../in/lprofiles.txt’
    defn%f_coef = ‘../in/coef.txt’
    defn%f_channels = ‘../in/channels.txt’
    defn%f_angles = ‘/angles.txt’
    defn%f_prof_id = ‘/prof_id.txt’
    defn%f_be = ‘/be.txt’
    defn%f_datetime = ‘/datetime.txt’
    defn%f_gas_units = ‘/gas_units.txt’
    defn%f_simple_cloud = ‘/atm/simple_cloud.txt’
    defn%f_o3 = ‘/atm/o3.txt’
    defn%f_p_half = ‘/atm/p_half.txt’
    defn%f_q = ‘/atm/q.txt’
    defn%f_t = ‘/atm/t.txt’
    defn%f_._skin = ‘._skin.txt’
    defn%f_near_surface = ‘near_surface.txt’
    defn%f_._near_surface = ‘._near_surface.txt’
    defn%f_skin = ‘skin.txt’
    defn%prof_ndigits = 3
    defn%realprec = 6
    defn%nlevels = 70
    defn%nsurfaces = 1
    defn%nprofiles = 6
    defn%nchannels = 72
    defn%coef_prefix = ‘/opt/rttov-14.0’
    /

    #50730
    James HockingJames Hocking
    Keymaster

    Hi Matt,

    Thanks for this, the offending lines in the namelist are:

    defn%f_._skin = ‘._skin.txt’

    and also

    defn%f_._near_surface = ‘._near_surface.txt’

    The correct lines are also present:
    defn%f_skin = ‘skin.txt’
    defn%f_near_surface = ‘near_surface.txt’

    So this is an issue with the rttov_test.pl Perl script and not related to the gfortran version. If you could test after downgrading your Perl to v5.32.1 (or earlier) that would be very helpful. I will investigate this from my end, but I won’t be able to look at it for a few days.

    It would also be helpful if you can copy paste the result of doing this:

    $ ls -R test_rttov14.1.gfortran-openmp/amsre/401/in/profiles

    Since the example_fwd test ran successfully you can go ahead with using RTTOV in your application. The test suite is for testing the code, not for running it for user applications, so this should not hold up your work.

    Best wishes,
    James

    #50792
    Matt PagetMatt Paget
    Participant

    Hi James,

    Problem resolved. It was a mistake at my end. Your guidance above and @atprata’s conda build helped me find the issue. Somewhat embarrassingly,
    – I had repackaged the rttov140.tar.xz to *.tar.gz for convenience with our other build patterns but, as I did the repackaging on my mac, it introduced a bunch of macos extra attributes files (“._*”) that were then getting collected in a glob-like call in the perl test script.
    – So my fix was to use the *.tar.xz and adapt our build pattern accordingly.

    Thank you for your help, James. Much appeciated.
    Cheers,
    Matt

    #50794
    James HockingJames Hocking
    Keymaster

    Hi Matt,

    Thanks very much for letting me know. Glad you got it working!

    Cheers,
    James

Viewing 6 posts - 1 through 6 (of 6 total)
  • You must be logged in to reply to this topic.