question about RTTOV v13 Test

[yuhan jiang]

hello！
I’m having trouble with testing, when I finished compiling and then tested, all tests fail like this:
STOP 1
| 2022/11/30 23:57:05 fatal error in module ../../src/hdf/rttov_hdf_mod.F90:0106
|
| 2022/11/30 23:57:05 fatal error in module
| ../../src/coef_io/rttov_read_coefs.F90:0229
|
| 2022/11/30 23:57:05 fatal error in module ../../src/test/rttov_test.F90:1805
|
| 2022/11/30 23:57:05 fatal error in module ../../src/test/rttov_test.F90:0317
How can I solve this poblem?Wating for your reply.
Thanks
jiang

[James Hocking]

Dear Jiang,

This is occurring because the code is failing to open the HDF5 library. I haven’t seen this error before so I’m not sure what would cause it.

Which version of the HDF5 library are you using? It must be v1.8.8 or later.

Have you exported the HDF5 library to your LD_LIBRARY_PATH (or the equivalent variable on your system) before running the test?

$ export LD_LIBRARY_PATH=/full/path/to/your/hdf5_library/lib/:$LD_LIBRARY_PATH

Best wishes,
James

[Andrew Prata]

Dear James

Today I tried to install and compile RTTOV v13.2 and I ran into exactly the same error.

I should note that I’m trying to run RTTOV on my Mac which has the M2 chip (arm64 architecture).

I was able to get RTTOV to compile successfully in a conda environment (using mamba); however, as Jiang found, all of the tests fail with the HDF5 library issue.

Note that I did try:
$ export LD_LIBRARY_PATH=/Users/apra0062/mambaforge/envs/rttov/lib

And ran the following test:
$ ./test_rttov13.sh ARCH=gfortran

Any advice would be much appreciated. Has anyone you know tried to compile and run on the new Apple Silicon machines?

Thanks
Andrew Prata | andrew.prata@monash.edu

[James Hocking]

Dear Andrew,

Could you confirm which version of the HDF5 library you are using?

When you update the LD_LIBRARY_PATH, are you pointing to the HDF5 lib/ directory, rather than your RTTOV lib/ directory:

$ export LD_LIBRARY_PATH=/full/path/to/your/hdf5_library/lib/:$LD_LIBRARY_PATH

I’m not sure if RTTOV has been successfully compiled on the M2 architecture, but I wouldn’t necessarily expect problems.

Best wishes,
James

[Andrew Prata]

Hi James

Thank you for the quick reply. I can see how my above comment might be confusing. I have created a conda environment called “rttov”. So all of the library files inside the environment get installed in my $CONDA_PREFIX, which in this case is: /Users/apra0062/mambaforge/envs/rttov and so I believe the lib directory for hdf5 is here: /Users/apra0062/mambaforge/envs/rttov/lib

If I had called the environment ‘my-env’, for example, then my library path would be: /Users/apra0062/mambaforge/envs/my-env/lib

For what it’s worth, my RTTOV lib directory is here: /Users/apra0062/packages/rttov132/lib

The version of my hdf5 library is 1.14.0, which I found by running:
(rttov)$ h5dump –version
h5dump: Version 1.14.0

Presumably I can install a different version of hdf5 with conda – just let me know which version should work. I can also try and install with ‘brew’ but I would like to have rttov and its dependancies installed in a self-contained environment.

Thanks
Andrew

[James Hocking]

Hi Andrew,

I haven’t yet tested RTTOV with HDF5 v1.14.0, but it has been tested with HDF5 v1.12.0, v1.10.4, and v1.8.8+, so if you could try v1.12.0 that would be good to confirm or rule out a compatibility issue with the latest HDF5 release.

Thanks,
James

[Andrew Prata]

Hi James

Thanks for the suggestion. I ran with a conda environment using hdf5 v1.12.2, netcdf-fortran 4.6.0 and python 3.9.7. After a successful compile, all 11 tests passed. So it looks like it’s an hdf5 version issue (at least when trying to build with a conda environment).

Thanks for the help.

Cheers
Andy

[James Hocking]

Hi Andy,

Thanks for letting me know about this: I will look into the issue with RTTOV and HDF5 v1.14 for the next release.

Cheers,
James

[Marco Wurth]

Hi James,

I am running into the same issue as Jiang and Andy and only hdf5 v1.12.3 is provided on my cluster (intel x86_64) and no other version unfortunately.
I find it suprising that Andy did not experience the issue with v1.12.2 but I do with the difference of only a minor revision of hdf5.

Using RTTOV 13.2 I get slightly different error lines compared to what Jiang posted at the top but they are essentially the same:
2023/12/15 14:29:21 fatal error in module rttov_hdf_mod.F90:0106
2023/12/15 14:29:21 fatal error in module rttov_read_coefs.F90:0246
2023/12/15 14:29:21 fatal error in module rttov_test.F90:1814
2023/12/15 14:29:21 fatal error in module rttov_test.F90:0319

Did you figure out a workaround for this issue already?

Cheers,
Marco

[James Hocking]

Hi Marco,

This seems to be caused when RTTOV attempts to open the HDF5 library when it is already open. It doesn’t cause a problem with earlier versions of the HDF5 library.

In many user applications this “re-opening” would not occur, and indeed when running the example*.sh scripts (which run the src/test/example_*.F90 programs) there is no problem.

But the problem occurs with the RTTOV test suite (run using rttov_test.pl) due to the way this is configured.

I’ve made updates which resolve the issue with the test suite and should also prevent potential issues more generally in user code. The fix for RTTOV v13.2 is here:
https://nwp-saf.eumetsat.int/downloads/rttov13_files/rttov_read_coefs.F90

Download this file to src/coef_io/ and recompile RTTOV. If you need a fix for RTTOV v13.1 or v13.0 let me know.

Best wishes,
James

[Marco Wurth]

Hi James,

thank you so much for the bugfix, now the tests all pass!

All the best,
Marco

[James Hocking]

Hi Marco,

Great, thanks for letting me know.

Best,
James

[Author]

Posts