RTTOV v13 instllation

[Atsushi Matsuoka]

Hi,

I’m new to RTTOV v13. Now trying to install it, following the guide, on my new laptop (Ubuntu 22.04.4 LTS using 13th Gen Intel® Core™ i9-13900HX × 32, 64 GB memory). Compilation has been successfully done with gfortran. However, the following error appears:
1
$ ./rttovgui
ERROR : the environment variable RTTOV_GUI_PREFIX is not set.

I’ve added absolute path in the rttov_gui.env. However, the same error still stands. I don’t know what to do from here.

Any clues? Your comments would be greatly appreciated.

Thanks,
Atsushi

[Atsushi Matsuoka]

Hi,

I made some progress but still encounter an error. Below shows what I did:

1. Compilation successful with HDF5_PREFIX = …/hdf5-1.12.1-h2b7332f_3, uncommented for FFLAGS_HDF5 and LDFLAGS_HDF5.
2. ./install_python_for_gui.sh
3. Set RTTOV_GUI_PREFIX=/home/atsushimatsuoka/rttov132/gui in rttov_gui.env
4. source ./rttov_gui.env
5. (rttovgui) atsushimatsuoka@system76-pc:~/rttov132/gui$ ./rttovgui
Traceback (most recent call last):
File “/home/atsushimatsuoka/rttov132/gui/rcontroller/controller.py”, line 15, in <module>
import rmodel
File “/home/atsushimatsuoka/rttov132/gui/rmodel/__init__.py”, line 1, in <module>
from . import project
File “/home/atsushimatsuoka/rttov132/gui/rmodel/project.py”, line 5, in <module>
from rttov.mfasis import MfasisLut
File “/home/atsushimatsuoka/rttov132/gui/rttov/__init__.py”, line 22, in <module>
from . import getcoefval
File “/home/atsushimatsuoka/rttov132/gui/rttov/getcoefval.py”, line 7, in <module>
from rttov_gui_f2py import rttov_gui_get_coef_val_i0, \
ModuleNotFoundError: No module named ‘rttov_gui_f2py’

Trying to install f2py but with no success. Can anyone help for solving this issue?

Best,
Atsushi

[James Hocking]

Hi Atsushi,

The error message indicates that the rttov_gui.f2py.so library is not found. This library is built when you compile RTTOV with GUI support. This requires f2py to be installed which is part of the numpy package. When you compiled RTTOV did the GUI-related code compile? The rttov_compile.sh script should have printed out some information about this. If it built successfully, then the rttov_gui_f2py.so library should be in the output lib/ folder once compilation is complete.

You can then either copy this file to the gui/ folder or (better) make a symbolic link to it from the gui/ folder.

Best wishes,
James

[Atsushi Matsuoka]

Hi James,

Yes, after running rttov_compile.sh, it says compiled successfully:
…
…
make[1]: Leaving directory ‘/home/atsushimatsuoka/rttov132/src/gui’
cd gui && make bin
make[1]: Entering directory ‘/home/atsushimatsuoka/rttov132/src/gui’
cd ../../tmp-gfortran/gui && gfortran -I../.././/include -D_RTTOV_HDF -I/home/atsushimatsuoka/anaconda3/pkgs/hdf5-1.12.1-h2b7332f_3/include -fPIC -O3 -ffree-line-length-none -I../.././/mod -c ../../src/gui/rttov_gui_test_run.F90 -o ../.././/obj/rttov_gui_test_run.o
cd ../../tmp-gfortran/gui && gfortran -o ../.././/bin/rttov_gui_test_run.exe ../.././/obj/rttov_gui_test_run.o -L../.././/lib -lrttov13_gui -lrttov13_brdf_atlas -lrttov13_other -lrttov13_emis_atlas -lrttov13_coef_io -lrttov13_parallel -lrttov13_hdf -lrttov13_main -L/home/atsushimatsuoka/anaconda3/pkgs/hdf5-1.12.1-h2b7332f_3/lib -lhdf5hl_fortran -lhdf5_hl -lhdf5_fortran -lhdf5
make[1]: Leaving directory ‘/home/atsushimatsuoka/rttov132/src/gui’

RTTOV compiled successfully

(base) atsushimatsuoka@system76-pc:~/rttov132/build$

By launching another terminal, which also activates rttovgui written in .bashrc, I can see that rttov_gui_f2py.so is now located under ~/rttov132/gui as follows:

(rttovgui) atsushimatsuoka@system76-pc:~/rttov132/gui$ ls
doc requirements.txt rttovguienv.yml
icons rmodel rttov_gui_f2py.so
install_python_for_gui.sh rttov run
Miniconda3-latest-Linux-x86_64.sh rttovgui rview
r1Dvar rttov_gui.env test
rcontroller rttovguienv_python3.7.yml

However, I still have the same error complaining about rttov_gui_f2py by running:

(rttovgui) atsushimatsuoka@system76-pc:~/rttov132/gui$ ./rttovgui
“Same error message as posted before”

Any clues?

Atsushi

[James Hocking]

Hi Atsushi,

Before the rttov_compile.sh script does the compilation, it prints out a summary of what is going to be compiled. Does it say that the Python wrapper is enabled at that point?

I can see from your output that RTTOV compiled successfully, but what you have quoted does not show whether or not the rttov_gui_f2py.so library was compiled or not.

There is a symbolic link gui/rttov_gui_f2py.so which points to the default location of this library after building (so long as you don’t specify a different installation directory): you are seeing this symbolic link, not the file itself. If you do “$ ls -l” within the gui/ directory you will see whether or not the symbolic link is broken or not, i.e., does the file that it points to actually exist?

Best wishes,
James

[Atsushi Matsuoka]

Hi James,

As you pointed out, the summary during the compilation actually says that f2py does not appear to be installed…:

…
…
Checking ../build/Makefile.local for user-specified HDF5 library…
…found FFLAGS_HDF5 and LDFLAGS_HDF5: compiling with HDF5 library

Checking ../build/Makefile.local for user-specified NetCDF library…
…did not find FFLAGS_NETCDF or LDFLAGS_NETCDF: compiling without NETCDF library
If you want to compile with NETCDF you must specify this in ../build/Makefile.local first

Checking ../build/Makefile.local for user-specified LAPACK library…
…did not find FFLAGS_LAPACK or LDFLAGS_LAPACK: compiling with lapack.f included in RTTOV

Testing for f2py using ‘f2py –fcompiler=gnu95’…
…f2py does not appear to be installed, Python interface and RTTOV GUI cannot be compiled

Previous build detected in ./: perform a clean compilation? (y/n)
Choose y if you have changed the compilation options since the previous build and re-compilation fails.
> y

Specify any additional flags to pass to make (e.g. -j); leave blank if unsure
>

===========================
RTTOV COMPILATION SUMMARY
===========================

Compiling with flags : gfortran
Compiling in directory : ./

RTTOV features available:
HDF5 coefficient I/O : y
Emissivity/BRDF atlases : y
C/C++ wrapper : y
Python wrapper : n
RTTOV GUI : n
HTFRTC netCDF files : n

Compiling with user LAPACK lib : n

Regenerating Makefiles using:
$ ../build/Makefile.PL RTTOV_HDF=1 RTTOV_F2PY=0 RTTOV_USER_LAPACK=0

I’m a bit lost. What should I do?

Best,
Atsushi

[James Hocking]

Hi Atsushi,

Section 1.1.1 of the GUI user guide (docs/rttov_gui_v13.pdf) gives some guidance on how to set up your Python environment for the RTTOV GUI. I would recommend using conda. You must then activate the conda environment before compiling RTTOV and f2py should then be found (it is part of the numpy Python package).

Best wishes,
James

[Atsushi Matsuoka]

Hi James,

I have conda installed with the required python modules. After activating rttovgui, I re-compiled as follows (near the end of this message) but now it complains the libhdf5hl_fortran.so.200:

(rttovgui) atsushimatsuoka@system76-pc:~/rttov132/gui$ ./rttovgui
Traceback (most recent call last):
File “/home/atsushimatsuoka/rttov132/gui/rcontroller/controller.py”, line 15, in <module>
import rmodel
File “/home/atsushimatsuoka/rttov132/gui/rmodel/__init__.py”, line 1, in <module>
from . import project
File “/home/atsushimatsuoka/rttov132/gui/rmodel/project.py”, line 5, in <module>
from rttov.mfasis import MfasisLut
File “/home/atsushimatsuoka/rttov132/gui/rttov/__init__.py”, line 22, in <module>
from . import getcoefval
File “/home/atsushimatsuoka/rttov132/gui/rttov/getcoefval.py”, line 7, in <module>
from rttov_gui_f2py import rttov_gui_get_coef_val_i0, \
ImportError: libhdf5hl_fortran.so.200: cannot open shared object file: No such file or directory

I see it’s there but likely not linked well:
/home/atsushimatsuoka/anaconda3/pkgs/hdf5-1.12.1-h2b7332f_3/lib/libhdf5hl_fortran.so.200
/home/atsushimatsuoka/anaconda3/lib/libhdf5hl_fortran.so.200

Where should I set that correctly? Sounds like my setting for the hdf library when compiling was wrong but wanted to double check.

Best,
Atsushi

————————————-
Re-compiling with the installed conda:

(rttovgui) atsushimatsuoka@system76-pc:~/rttov132/build$ ./rttov_compile.sh
…
…
Checking ../build/Makefile.local for user-specified HDF5 library…
…found FFLAGS_HDF5 and LDFLAGS_HDF5: compiling with HDF5 library

Checking ../build/Makefile.local for user-specified NetCDF library…
…did not find FFLAGS_NETCDF or LDFLAGS_NETCDF: compiling without NETCDF library
If you want to compile with NETCDF you must specify this in ../build/Makefile.local first

Checking ../build/Makefile.local for user-specified LAPACK library…
…did not find FFLAGS_LAPACK or LDFLAGS_LAPACK: compiling with lapack.f included in RTTOV

Testing for f2py using ‘f2py –fcompiler=gnu95’…
…f2py detected: do you want to compile the Python wrapper and RTTOV GUI? (y/n)
> y

Previous build detected in ./: perform a clean compilation? (y/n)
Choose y if you have changed the compilation options since the previous build and re-compilation fails.
> y

Specify any additional flags to pass to make (e.g. -j); leave blank if unsure
>

===========================
RTTOV COMPILATION SUMMARY
===========================

Compiling with flags : gfortran
Compiling in directory : ./

RTTOV features available:
HDF5 coefficient I/O : y
Emissivity/BRDF atlases : y
C/C++ wrapper : y
Python wrapper : y
RTTOV GUI : y
HTFRTC netCDF files : n

Compiling with user LAPACK lib : n

Regenerating Makefiles using:
$ ../build/Makefile.PL RTTOV_HDF=1 RTTOV_F2PY=1 RTTOV_USER_LAPACK=0

[James Hocking]

Hi Atsushi,

You usually need to ensure that the HDF5 library is available for dynamic linking. On most Linux systems you do this by adding the location of it to the $LD_LIBRARY_PATH environment variable:

$ export LD_LIBRARY_PATH=$LD_LIBRARY_PATH:/home/atsushimatsuoka/anaconda3/lib/

Try doing this and then launching the GUI: hopefully it will work.

Best wishes,
James

[Atsushi Matsuoka]

Hi James,

After export the library, I got:

(rttovgui) atsushimatsuoka@system76-pc:~/rttov132/gui$ ./rttovgui
xkbcommon: ERROR: failed to add default include path /home/atsushimatsuoka/miniconda3/envs/rttovgui/share/X11/xkb
xkbcommon: ERROR: failed to add default include path /home/atsushimatsuoka/miniconda3/envs/rttovgui/share/X11/xkb
Segmentation fault (core dumped)

It’s confused with anaconda3/miniconda3. Now feeling that I should start all the installation process again…

Or any advise?

Best,
Atsushi

[James Hocking]

Hi Atsushi,

Googling for that error message throws up quite a variety of results, but I don’t really know what is causing the problem.

If you have both anaconda and miniconda installed and the system is confusing them, then yes, probably the sensible option is to try rebuilding.

Note that when you compile RTTOV, you should activate your conda environment (anaconda or miniconda) before running the build script.

To save you some time, instead of doing a full recompilation of RTTOV, you can try the following in the RTTOV src/ directory:

$ make ARCH=yourARCH wrapper/clean

[here “yourARCH” is gfortran or gfortran-openmp, whatever you specified previously in the build script]
This just removes the build files for the wrapper code.

$ conda activate env-name

[here env-name is your environment name]

$ make ARCH=yourARCH

This will recompile the wrapper code using your conda environment.
Then you can try running the GUI:

$ cd ../gui/
$ . rttov_gui.env
$ ./rttovgui

If there is still confusion about anaconda vs miniconda it might be worth removing one of them. I’m not sure there is any reason to have both installed.

Best wishes,
James

[Author]

Posts