{"id":10014,"date":"2017-02-27T15:57:25","date_gmt":"2017-02-27T15:57:25","guid":{"rendered":"https:\/\/www.nwpsaf.eu\/site\/?page_id=10014"},"modified":"2025-11-27T14:02:21","modified_gmt":"2025-11-27T14:02:21","slug":"1dvar-user-guide","status":"publish","type":"page","link":"https:\/\/nwp-saf.eumetsat.int\/site\/software\/1d-var\/documentation\/1dvar-user-guide\/","title":{"rendered":"1DVAR User Guide"},"content":{"rendered":"

<\/a><\/center><\/p>\n

<\/a>NWPSAF 1D-Var User Manual<\/h1>\n

<\/a>Software Version 2.0<\/small><\/h1>\n

<\/a>NWPSAF-MO-UD-032<\/h3>\n

<\/a><\/p>\n\n\n\n\n\n\n\n\n\n
Revision
\nHistory<\/span><\/td>\n<\/tr>\n
Document revision<\/span><\/td>\nDate<\/span><\/td>\nAuthor<\/span><\/td>\nNotes<\/span><\/td>\n<\/tr>\n
Version 1.1<\/td>\n06\/12\/16<\/td>\nF. Smith<\/td>\nDraft Version valid for NWPSAF 1D-Var v1.1<\/td>\n<\/tr>\n
Version 1.1.1<\/td>\n20\/06\/18<\/td>\nS. Havemann<\/td>\nv1.1.1: new release after LWP bugfix<\/td>\n<\/tr>\n
Version 1.2<\/td>\n24\/02\/20<\/td>\nS. Havemann<\/td>\nv1.2: Version valid for NWPSAF 1D-Var v.1.2<\/td>\n<\/tr>\n
Version 2.0<\/td>\n21\/05\/21<\/td>\nE. Pavelin, J. Hocking<\/td>\nv2.0: Version valid for NWPSAF 1D-Var v2.0<\/td>\n<\/tr>\n
Version 2.1<\/td>\n01\/09\/21<\/td>\nM. Cooke, O. Young<\/td>\nNew release for RTTOV14<\/td>\n<\/tr>\n<\/tbody>\n<\/table>\n

<\/a><\/p>\n

<\/a> The documentation was developed within the context
\nof the EUMETSAT Satellite Application Facility on
\nNumerical Weather Prediction (NWP SAF),
\nunder the Cooperation Agreement dated 7 December 2016,
\nbetween EUMETSAT and the Met Office, UK,
\nby one or more partners within the NWP SAF.
\nThe partners in the NWP SAF are the Met Office,
\nECMWF, DWD and Meteo France.<\/i><\/p>\n

<\/a>\u00a9 2021, EUMETSAT, All Rights Reserved. <\/i><\/p>\n

<\/a>Contents<\/h4>\n

1. Introduction<\/a>
\n2. Compiling and Running the 1D-Var Code<\/a>
\n3. Input Files<\/a>
\n4. Output Files<\/a>
\n5. Setting Up a New Observation Type<\/a>
\n6. Simulating observations<\/a>
\n7. Use of PC scores<\/a>
\n8. Surface emissivity retrieval<\/a>
\n9. Notes, bugs and features<\/a>
\n10. Getting Help<\/a>
\n11. References<\/a>
\n12. Acknowledgements<\/a>
\n(Appendices are stored as seperate files to reduce the size of
\nthe printout for this document)<\/i>
\nAppendix A. Top Level Design<\/a>
\nNote that this refers to a separate PDF document and
\nsupersedes the old Appendix A.
\nAppendix B. Minimization<\/a>
\nAppendix C. Infrared cloudy Retrievals<\/a>
\nAppendix D. Microwave cloud liquid water retrieval<\/a>
\nAppendix E. Instructions for using the observation simulation code<\/a><\/p>\n

<\/a><\/p>\n

<\/a>1. Introduction<\/h4>\n

<\/a>This document describes a stand-alone 1DVar code for
\nnadir-viewing passive sounding satellite instruments that has been
\ndeveloped at the Met Office as part of the NWP SAF. The code contains
\nthe merged capabilities of the obsolete Met Office, ECMWF and SSMIS
\n1D-Var schemes. It was originally developed for the IASI instrument
\nbut it may be used with many different sounding instruments with
\nminimal changes.<\/p>\n

<\/a>The philosophy behind the development of this
\ncode was to produce a flexible, stand-alone 1DVar retrieval
\nsystem that may be used for a wide variety of satellite
\ninstruments. However, it is not exhaustive. Extra subroutines could be
\nadded by the user to meet their own requirements, such as code to
\nadd features such as cloud detection, or code may be modified to include
\nextra retrieval variables.<\/p>\n

<\/a>The program functions first by reading in user
\nsupplied background and observation data (together with
\nappropriate error covariance matrices). The routines that are
\nprovided to do this assume data is supplied in a predefined
\nASCII format, but these routines can be replaced with ones more
\nspecific to the format of data that is to be used.<\/p>\n

<\/a>Routines for channel selection and cloud
\ndetection are included but bias correction needs to be performed
\nby the user, or a bias correction subroutine added at the appropriate
\nplaceholder in the code.<\/p>\n

<\/a>Retrievals are made through the optimal
\nestimation methods of Rodgers (1976). Three
\nminimisation routines<\/a> are provided. The one that is to be used
\ndepends on the size and linearity of the problem.<\/p>\n

A radiative transfer program is required that will supply both
\nradiances and Jacobians and this is accessed through a general
\npurpose interface routine. Currently RTTOV Versions 12 and 13
\nare implemented.<\/p>\n

The code is written in European Standard Fortran 90 and advanced
\nF90 features are used as much as possible. The code was originally
\ndesigned to closely follow the structures in the Met Office’s
\nObservation Processing System (OPS).<\/p>\n

[ Return to Top ]<\/a><\/p>\n

<\/a>2. Compiling and Running the Code.<\/h4>\n

Instructions for compiling and testing the code
\ncan be found in the readme.txt<\/tt> file in the top
\n1DVar directory. Further information can be found in the Release Note.<\/p>\n

For this release, the code should be run from a subdirectory.
\nThe directory
\nWorkDir<\/tt> is provided for your convenience with some test and
\nexample scripts. This directory can be copied and\/or renamed.
\nThe
\nNWPSAF_1DVar<\/tt> executable should be linked into this directory.
\nNamelist and data files will also need to be linked in.
\nThe example scripts show
\nhow this can be done.<\/p>\n

To run the code successfully, relevant radiative transfer
\ncoefficients files must also be located in the working directory
\n(WorkDir or a copy thereof).
\nThe coefficients files must be compatible with the
\nradiative transfer model chosen and the instrument whose data is
\nbeing used.
\nCoefficients files can be downloaded for free from
\nthe NWPSAF
\nRTTOV
\nwebsite<\/a>.
\nPlease read through the readme.txt file for instructions on
\nensuring your coefficient files are named correctly.<\/p>\n

[ Return to Top ]<\/a><\/p>\n

<\/a>3. Input Files<\/h4>\n

3a. Control Files: ControlData.NL<\/a>
\n3b. Control Files: Retrievals.NL<\/a>
\n3c. Data Files: ObsFile.dat<\/a>
\n3d. Data Files: Background.dat<\/a>
\n3e. Auxiliary Files: Rmatrix.dat<\/a>
\n3f. Auxiliary Files: Bmatrix.dat<\/a>
\n3g. Auxiliary Files: ChannelChoice.dat<\/a>
\n3h. Auxiliary File: EmisEigenVec<\/a>
\n3i. Auxiliary File: EmisPCAtlas <\/a><\/p>\n

There are two main control files<\/a>, ControlData.NL<\/tt>
\nand Retrieval.NL<\/tt>, and two main data
\nfiles<\/a>, ObsFile.dat<\/tt> and Background.dat<\/tt>.<\/p>\n

Auxiliary data<\/a> such as error covariance matrices
\nand channel selection choices are stored for each data type in a
\ndirectory called xxx_COEFFS_DIR<\/tt> where xxx<\/tt>
\nrefers to the data type being processed (e.g., IASI_COEFFS_DIR). The location
\nof this directory is specified using the environment variable COEFFS_DIR<\/tt>;
\nfor example, the test script Run_1DVar_test.sh<\/tt> contains the statement:<\/p>\n

export COEFFS_DIR=${topdir}\/${INSTRUMENT2}_COEFFS_DIR<\/tt><\/p>\n

All input files are ASCII and all values are in free-format.<\/p>\n

<\/a> 3a. Control Files: ControlData.NL<\/b><\/p>\n

ControlData.NL<\/tt> is a Fortran90
\nnamelist file containing the input parameters required for the
\nrunning of the code. The table below lists the parameters that
\nmay be specified through this file and which ones would normally
\nbe required. As one can see most of these parameters have a
\ndefault value. This file and the
\nNWPSAF_Read_ControlData subroutine that reads it may be easily expanded
\nin future if one wants to add further options. Note that there are
\ncomments included in this file as supplied, but some Fortran90
\ncompilers may not allow this, in which case they would need to be
\nremoved.<\/p>\n

 <\/p>\n

Namelist Control for IASI_1DVar<\/h4>\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n
Variable<\/b><\/td>\nRequired?<\/b><\/td>\nDefault Value<\/b><\/td>\nType<\/b><\/td>\nNotes<\/b><\/td>\n<\/tr>\n
FirstOb, LastOb<\/td>\nNo<\/td>\n1, Number of obs.<\/td>\nINTEGER<\/td>\n<\/td>\n<\/tr>\n
GeneralMode<\/td>\nNo<\/td>\n10=’ProductionMode’<\/td>\nINTEGER<\/td>\nAllowed values are 0=’Operational’, 10=’Production’,
\n20=’Diagnostic’, 30=’Debug’ and 40=’Verbose’. Note that Operational
\nand Production modes are the same in practice.<\/td>\n<\/tr>\n
DetectCloud<\/td>\nNo<\/td>\nTrue<\/td>\nLOGICAL<\/td>\nNote this is a cloud detection test designed for use with IR sounders.<\/td>\n<\/tr>\n
CostThresh_Land
\nCostThresh_Sea<\/td>\n		Yes<\/td>\n-9999.<\/td>\nREAL<\/td>\nIf DetectCloud is TRUE<\/td>\n<\/tr>\n
CostThresh_IRWindow_Land
\nCostThresh_IRWindow_Sea<\/td>\n		Yes<\/td>\n-9999.<\/td>\nREAL<\/td>\nIf DetectCloud is TRUE<\/td>\n<\/tr>\n
CloudAbsThresh_IRWindow<\/td>\nYes<\/td>\n-9999.<\/td>\nREAL<\/td>\nIf DetectCloud is TRUE<\/td>\n<\/tr>\n
HighCloudAbsThresh_IRWindow<\/td>\nYes<\/td>\n-9999.<\/td>\nREAL<\/td>\nIf DetectCloud is TRUE<\/td>\n<\/tr>\n
Perform1DVar<\/td>\nNo<\/td>\nTrue<\/td>\nLOGICAL<\/td>\nPurpose of running with false might be to test
\nthe setup or IO before running with true<\/td>\n<\/tr>\n
Minimisation_Method<\/td>\nNo<\/td>\n1=’Newtonian’<\/td>\nINTEGER<\/td>\n0=None (sets Perform1DVar to false), 1=Newtonian, 2=Marquardt-Levenberg<\/td>\n<\/tr>\n
MaxIterations<\/td>\nNo<\/td>\n10<\/td>\nINTEGER<\/td>\nMaximum number of iterations in the minimisation.<\/td>\n<\/tr>\n
DoTExtrapolation<\/td>\nNo<\/td>\nTRUE<\/td>\nLOGICAL<\/td>\nIf using a background profile where the top level is
\nlower than 0.01hPa then setting this option to true means
\nthe profile is extrapolated to 0.01hPa<\/td>\n<\/tr>\n
DeltaJ<\/td>\nNo<\/td>\n0.01<\/td>\nREAL<\/td>\nMaximum fractional change in cost function for
\nconvergence.<\/td>\n<\/tr>\n
SmallJCost_Gradient<\/td>\nNo<\/td>\n1.<\/td>\nREAL<\/td>\nMaximum value of cost function gradient (in terms of
\ntotal cost) for convergence. Marquardt-Levenberg
\nminimisation only.<\/td>\n<\/tr>\n
Additional_Cost_Function<\/td>\nNo<\/td>\n0=None<\/td>\nINTEGER<\/td>\n0=None, 1=CloudBoundaries
\nFor retrieval of cloud<\/td>\n<\/tr>\n
Max_ML_Iterations<\/td>\nNo<\/td>\n10<\/td>\nINTEGER<\/td>\nMax. number of iterations for the Marquardt-Levenberg inner loop (M-L minimisation only).<\/td>\n<\/tr>\n
Gamma_Factor<\/td>\nNo<\/td>\n10<\/td>\nREAL<\/td>\nFactor by which Gamma is changed between iterations.
\nMarquardt-Levenberg minimisation only<\/td>\n<\/tr>\n
GammaMax<\/td>\nNo<\/td>\n1025<\/sup><\/td>\nREAL<\/td>\nMarquardt-Levenberg minimisation only<\/td>\n<\/tr>\n
Allow_Eqn_101<\/td>\nNo<\/td>\n.TRUE.<\/td>\nLOGICAL<\/td>\nSet to false if the Eqn 101 minimisation is never to
\nbe used.<\/td>\n<\/tr>\n
Force_Eqn_101<\/td>\nNo<\/td>\n.FALSE.<\/td>\nLOGICAL<\/td>\nSet to true to use Eqn 101 in all situations
\n(Marquardt-Levenberg and additional cost function terms
\ncannot be used in this case).<\/td>\n<\/tr>\n
MaxChanUsed<\/td>\nNo<\/td>\n10000<\/td>\nINTEGER<\/td>\n<\/td>\n<\/tr>\n
Cloud_Min_Pressure<\/td>\nNo<\/td>\n100hPa<\/td>\nREAL<\/td>\nMinimum allowed pressure for retrieved cloud top in
\nhPa. Retrieval of cloud only. Decrease this value if non-convergence encountered in the case of very high cloud.<\/td>\n<\/tr>\n
Use_EmisAtlas<\/td>\nNo<\/td>\n.FALSE.<\/td>\nLOGICAL<\/td>\nSwitch to control whether an RTTOV emissivity atlas is used
\nor not.<\/td>\n<\/tr>\n
Atlas_Dir<\/td>\nNo<\/td>\n‘EmisAtlas’<\/td>\nCHARACTER<\/td>\nDirectory containing the emissivity atlas<\/td>\n<\/tr>\n
Atlas_Id_Ir<\/td>\nNo<\/td>\n1<\/td>\nINTEGER<\/td>\nAllows choice between IR atlases (1=UWIRemis, 2=CAMEL 2007, 3=CAMEL clim)<\/td>\n<\/tr>\n
Atlas_Id_Mw<\/td>\nNo<\/td>\n1<\/td>\nINTEGER<\/td>\nAllows choice between MW atlases (1=TELSEM2, 2=CNRM)<\/td>\n<\/tr>\n
Read_CLW_Background<\/td>\nNo<\/td>\nFALSE<\/td>\nLOGICAL<\/td>\nFalse = No clw profile in the Background file and
\nTrue=clw profile in the Background file<\/td>\n<\/tr>\n
Retrieve_qtotal<\/td>\nNo<\/td>\n.FALSE. = LWP Retrieval<\/td>\nLOGICAL<\/td>\nFalse=LWP retrieval, True=Qtotal retrieval<\/td>\n<\/tr>\n
NPCScoresSurfEmiss<\/td>\nNo<\/td>\n12<\/td>\nINTEGER<\/td>\nNumber of PC scores to use for IR surface emissivity retrievals<\/td>\n<\/tr>\n
MwScattering<\/td>\nNo<\/td>\n.FALSE.<\/td>\nLOGICAL<\/td>\nTrue=Use RTTOV-SCATT for microwave clw retrievals<\/td>\n<\/tr>\n
EnahncedDiagnostics<\/td>\nNo<\/td>\n.FALSE.<\/td>\nLOGICAL<\/td>\nProduces extra output files (see Output
\nFiles<\/a>)<\/td>\n<\/tr>\n
Gas_Units<\/td>\nNo<\/td>\n2<\/td>\nINTEGER<\/td>\nUnits for humidity profile input to RTTOV. 0 = ppmv w.r.t. dry
\nair (compatibility with previous NWPSAF-1DVar releases). 2 =
\nppmv w.r.t. moist air.<\/td>\n<\/tr>\n
Legacy_Settings<\/td>\nNo<\/td>\n.FALSE.<\/td>\nLOGICAL<\/td>\nSets RTTOV options interp_mode=1 and use_q2m=.false. for
\ntesting using old datasets.<\/td>\n<\/tr>\n
LwpSD<\/td>\nNo<\/td>\n0.2<\/td>\nREAL<\/td>\nSets the background error for liquid water path retrievals (standard deviation).<\/td>\n<\/tr>\n
WindspeedSD<\/td>\nNo<\/td>\n1.4<\/td>\nREAL<\/td>\nSets the background error for wind speed retrievals (standard deviation).<\/td>\n<\/tr>\n<\/tbody>\n<\/table>\n

 <\/p>\n

Two directory locations, which in previous releases were set in the
\n ControlData.NL file<\/a>,
\nare now set via environment variables.
\nThese are:<\/a><\/p>\n