## # # ##### ##### # # # # # # # # # # # # # # # # ###### # ## # # # ##### # # ## ## # # # # # # # ##### # HISTORY OF CHANGES 2.3.00 ASCAT Wind Data Processor (AWDP) Version 2.3.00 5 February 2014 This documentation was developed within the context of the EUMETSAT Satellite Application Facility on Numerical Weather Prediction (NWP SAF), under the Cooperation Agreement dated 29 June, 2011, between EUMETSAT and the Met Office, UK, by one or more partners within the NWP SAF. The partners in the NWP SAF are the Met Office, ECMWF, KNMI and Meteo France. Copyright 2014, EUMETSAT, All Rights Reserved. CHANGE RECORD Version Date Author/changed by Remarks --------------------------------------------------------------- 1.0.14 12-11-2008 Anton Verhoef First draft, for AWDP 1.0 DRI 1.0.16 18-12-2008 Anton Verhoef Modified according to DRI comments 1.1.00 12-10-2010 Anton Verhoef Modified for AWDP version 1.1 Note that version 1.1 is identical to internal version 1.0.19 2.0.00 17-06-2010 Anton Verhoef Modified for AWDP version 2.0 2.0.01 24-11-2010 Anton Verhoef Modified according to DRI comments 2.1.00 28-07-2011 Anton Verhoef Modified for AWDP version 2.1 2.1.01 20-08-2012 Anton Verhoef Modified for AWDP version 2.1.01 (internal) 2.2.00 07-05-2013 Anton Verhoef Modified for AWDP version 2.2.00 (internal) 2.2.01 18-06-2013 Anton Verhoef Modified for AWDP version 2.2.01 (internal) 2.2.02 03-07-2013 Anton Verhoef Modified for AWDP version 2.2.02 (NWP SAF release) 2.3.00 05-02-2014 Anton Verhoef Modified for AWDP version 2.3.00 (NWP SAF release) Changes in AWDP v2.3.00 5 February 2014 - Made check on Not a Number (NaN) values in the ice map more robust. On some platforms this was not handled correctly. - Remove 3rd and 4th rank wind solutions when MLE1 < 0 or MLE2 < 0 or |MLE3/MLE1| > 40. See paper of Lin et al. in Atmos. Meas. Tech., 2013. - When processing full resolution data, initialise beam azimuth when it is missing for a correct check on + and - 360 degrees boundary condition. Otherwise we may get erroneous values for azimuths close to zero. - When reading a restart file in the ice screening, stop if its size is not as expected (i.e., wrong resolution) to increase robustness. - When reading BUFR data, add a small amount to the time stamp to force that data of the same time and from different files are always sorted in the same way (file 1 first, then file 2, then file 3). This is to prevent potential problems in the row merging. - Implemented GRIB API library version 1.11.0. - Implemented Metop-A version 9.0.2. NOC calibration tables. Changes in AWDP v2.2.02 3 July 2013 - Work around for bug in GRIB API resulting in wrong number of GRIB messages counted in NCEP GRIB2 files. - Compute the Kp's (variances) in the coastal product slightly differently. Changes in AWDP v2.2.01 18 June 2013 - Implemented version 000400 of the ECMWF BUFR software library. Note that versions 000388 and 000389 are not usable with AWDP. - Added option to process data on 6.25 km grid using the swath grid present in the new SZF files. - Added calibration tables for Metop-B level 1 data version 901. - Some changes to handle NCEP GRIB2 format data and data compressed using the JPEG algorithm. Changes in AWDP v2.2.00 7 May 2013 - In the ice screening, skip North Pole or South Pole if there is no (relevant) data there. - Solved some numerical rounding issues in the ice screening. - Implemented backscatter correction tables for Metop-B data version 8.0.1. - Implemented GRIB API library version 1.9.18. - Reduction of memory usage of inversion module. - Make BUFR file attributes and BUFR data global variables to reduce memory usage. - Convert sigma0s to Zspace also when they are negative. - Changes to allow reading of PFS format version 12 and use of new SZF files, both the sigma0s and the swath grid data. Also changes to read SZO and SZR format version 12. - Relaxed check on row sorting slightly: equal time stamps in subsequent rows are allowed now. Changes in AWDP v2.1.01 20 August 2012 - Bug fix in WVC_orientation (support/convert): limit argument of acos() between -1 and 1; this problem caused core dumps on some platforms. - Determine the time interval between subsequent NWP forecasts in the GRIB input automatically. The interval was fixed to 3 hours but is now dynamically assigned. - Get GRIB data only for WVCs that are level 1 on input, unless handleall switch is set. This gives computing time reduction in the NRT processing. - Implemented some performance gain when reading full resolution data. - More sophisticated check on duplicate rows for ASCAT. - Implemented some code specific for Metop-B (e.g. in the sigma0 calibration routine). - Read soil moisture information when available in PFS input file. - Moved the setting or unsetting of the KNMI QC flag to a separate generic routine for consistency. - Increased maxnr_of_grib_messages from 350 to 400 to keep compatibility with NCEP model. - Implement ice model version which is to a great extent based on the code of Maria Belmonte. - Adapt code where necessary for compilation using 64 bits platforms/compilers. Changes in AWDP v2.1.00 28 July 2011 - Provide sigma0s over land in the coastal product. Displace land WVCs away from the coast in the same way as the WVCs over sea. - Adapted MLE normalisation and QC threshold tables to the 'NWP Ocean Calibration'. - Check on level 1 beam usability flag, do not compute winds when it is set in one of the three beams. - Implemented version 000387 of the BUFR software and version 1.9.9 of the GRIB API software. - Implemented sigma0 correction tables for L1 product version 7.4.0. - Enlarge maximum Kp level in Kp test after inversion by a factor of 2 for 12.5 km products. - Fixed a number of minor bugs. Changes in AWDP v2.0.00 24 November 2010 - Some improvements to the ice model. - Low winds (< 4m/s) for ERS/ASCAT get equal probability, also in case of more than 2 solutions. - When no soil moisture or wind information is present, the output BUFR filename gets extension 'l1_bufr' instead of 'l2_bufr'. This is e.g. the case when a level 1 file in MetOp native format is converted to BUFR format by AWDP without performing wind inversion. - Introduced option for generation of coastal product; full resolution (SZF) files can be read in parallel to level 1b files and the full resolution backscatter values are box-averaged into the Wind Vector Cells. - The ECMWF GRIB API library is used now instead of GRIBEX. AWDP is capable to handle both GRIB edition 1 and GRIB edition 2 files now (edition 2 has not been tested with operational data yet). - Replaced backscatter correction tables made with 'Visual Ocean Calibration' by correction tables made with 'NWP Ocean Calibration'. Changes in AWDP v1.1.00 12 January 2010 - No functional changes. AWDP 1.1 is the new NWP SAF release. Changes in AWDP v1.0.19 07 January 2010 - Enlarged maximum number of decoded values in one BUFR message in order to handle the larger (50000 bytes instead of 15000 bytes) BUFR messages from EUMETSAT properly. - Output data are written in BUFR edition 4 now. - Added functionality to read the L1 part of the L2 Soil Moisture PFS product v11.0. - Some changes in Quality Control, 3 or 4 wind solutions are allowed now and no MLE-based rejection is done any more for triplets outside the cone. Adapted the monitoring bit setting to these changes. - Changed the default number of ambiguities in BUFR output from 2 to 4. - Different cut-off distances between full resolution s0 position and WVC position for 25/12.5 km products in routine that reads full resolution data. Changes in AWDP v1.0.18 26 August 2009 - Version 000370 of the ECMWF GRIB software was implemented in genscat. - Version 000380 of the ECMWF BUFR software was implemented in genscat. - Solved memory leak occurring when reading PFS files. - Several improvements have been implemented in the ice model. - In the monitoring step, 0.2 m/s is added to the model winds in order to convert them to neutral winds. - Fixed some issues connected with decoding of BUFR edition 4 input files. Changes in AWDP v1.0.17 16 April 2009 - Get model wind data from GRIB only if requested or not present to save computing time. - Skip WVCs that are above ice in the ambiguity removal. - Use correct sub centre id in BUFR section 1: for the EARS ASCAT data, the sub centre id in the BUFR data was 1, whereas it was 0 in section 1. For ERS data, the sub centre id in BUFR section 1 is alway set to 0, since ERS files may contain mixed data from different ground stations. - Use ECMWF/WMO BUFR tables version 13 instead of local tables. Changes in AWDP v1.0.16 08 January 2009 - Added script InstallAWDP for easy installation. - Several computing efficiency improvements in inversion and writing of BUFR output. Changes in AWDP v1.0.15 17 November 2008 - Always use first land sea-mask in GRIB file list, neglect date of GRIB field. - Use a smaller radius around WVC to look for land mask grid points for 12.5-km product (50 km vs 80 km for 25-km product). - Use file list instead of single file name to read full resolution PFS data. Option -fullres is replaced by -szffl. - Improved the calculation of orbit number at sensing time when reading PFS data files. - Implemented sigma0 correction tables for L1 product version 6.3.0. - Reduce the number of messages resulting from the adapted Kp test. Changes in AWDP v1.0.14 04 November 2008 - Made discrimination between 25 and 12.5 km input products more robust. - Adapted ice processing implementation: use closest to background wind solution and set knmi_qc flag in case of ice. - Improved SST evaluation from GRIB close to the coast: if one of the 4 surrounding grid points contains a missing value (e.g. since it is above land), the value of the closest GRIB point will be used. - The processing does not stop any more if a BUFR message can not be decoded/expanded. The message will be skipped and processing continues with the next message. - Changes in quality control and MLE normalisation; ERS is handled in the same way as ASCAT now. - Some changes to speed up computing in the GRIB collocation and inversion. - CMOD5n (neutral winds) is now used as the default GMF (was CMOD5.5 for ASCAT and CMOD5 for ERS). - Version 000360 of the ECMWF GRIB software was implemented in genscat. - Version 000360 of the ECMWF BUFR software was implemented in genscat. - Implemented product monitoring flag for ASCAT. Changes in AWDP v1.0.13 17 March 2008 - Version 000350 of the ECMWF GRIB software was implemented in genscat. - Version 000351 of the ECMWF BUFR software was implemented in genscat. - Removed the -nonwp command line option since it is implied by the (absence of) the -nwpfl option. - From now on, you can reprocess ASCAT L2 files and evaluate LSM and SST data from GRIB without having to use GRIB model winds. In awdp_grib, the GRIB file list is checked for the presence of wind data. If they are present, they will be used to replace the model winds in the ASCAT data. If not, the model winds in the ASCAT data will be left untouched. - Implemented a check for the L1 Kp estimate quality flag. - The software version number is written to the output BUFR file from now on. For this, the 83th descriptor "SOFTWARE IDENTIFICATION" in the wind part of the BUFR is used. Since it can only contain numeric values, the version numbering is changed into awdp__, i.e. version number, subnumber and subsubnumber are all numeric now. Hence, awdp1_0l becomes awdp1_0_12 and so on. The version number (which is in awdp_data.F90) is automatically updated with every new AWDP release. - Implemented CMOD5.n for neutral winds (to be called with command line option '-cmod 5n'). - Automatically determine correction table version using acquisition date and/or software id of L1 data. - Fixed bug concerning date/time handling in PFS files. - Adapted PFS reading module for EUMETSAT PFS format change (format version 11). - Implemented correction tables for L1 product version 5.5.0. Changes in AWDP v1.0l 04 December 2007 - Version 000340 of the ECMWF GRIB software was implemented in genscat. - Version 000340 of the ECMWF BUFR software was implemented in genscat. - Multiple rows are written into one BUFR message in order to make the output more suitable for the GTS - this used to be one row per BUFR message. The size of one BUFR message is limited to approximately 1000 WVCs or 50000 bytes. The GTS should be able to handle up to 500000 bytes. - The 2DVAR Ambiguity Removal routines use module singletonfft now instead of multifft. This gives some performance improvement and makes the software more flexible. - A performance improvement was accomplished in the GRIB handling module. Changes in AWDP v1.0k 02 October 2007 - Some ERS test data were made available in awdp/tests. - In the collocation of NWP data: if a parameter (e.g. land-sea mask) is present in multiple GRIB files, a WARNING is now set (instead of the error message of earlier versions) and the data from the first file of the GRIB file list is used. - Added functionality to read files in native MetOp (PFS) format. The file format is detected automatically: if no BUFR messages are found in the input file, an attempt is done to read it as PFS file. - In the ambiguity removal: QC flagged winds are skipped in analysis field, but a selection closest to analysis is determined. Note: this influences the data product: there used to be no selected wind solution in QC flagged winds, but now there may be a selection. Hence the user needs to be careful and use only those WVCs with the KNMI QC flag not set. - Sigma0 calibration can also be done for 12.5-km product by interpolating 25-km correction table values. - Implemented new correction tables for L1 data version 5.2.1. - Reduce memory usage by dynamic allocation of memory for cells in each row, i.e. for ERS data only 19 cells are allocated; for ASCAT 25-km, 42 cells are allocated and for ASCAT 12.5-km, 82 cells are allocated. The number of allocated cells per row was fixed at 82 in earlier versions. Changes in AWDP v1.0j 28 June 2007 - Version 000320 of the ECMWF GRIB software was implemented in genscat. - Version 000320 of the ECMWF BUFR software was implemented in genscat. - Some command line options for research activities were added. - A "distance to GMF too large" flag was implemented in WVC quality (wvc_quality%gmf_distance, Fortran bit 6). This flag is set in those cases where the cone distance is too large (above a certain threshold value). - Some ASCAT and ECMWF test data were made available in awdp/tests. - AWDP was adapted in such a way that it takes by default cmod5 for ERS data and cmod5.5 for ASCAT data. This can be overruled by the -cmod command line option. The inversion code where 0.5 m/s was added to ASCAT winds in combination with the cmod5 GMF was removed. Be careful on the setting of environment variable $LUT_FILENAME_C_VV if you process ERS and ASCAT in the same setup of AWDP. If you do not set it, the inversion code will write the LUT to a file called c-vv2.dat and it will not be possible to determine which cmod version it contains if you re-use the LUT file. So be safe and set $LUT_FILENAME_C_VV to either path/cmod5.dat or path/cmod5_5.dat! - Core dump problem occurring on some servers due to large Kp values (Kp > 100%) was solved. - Improvement in 2DVAR algorithm: first step in minimization was enlarged; with some compilers/platforms the minimization got stuck in a local minimum. Changes in AWDP v1.0i 28 March 2007 - Several bugs have been removed. - Several issues related to the collocation of GRIB data have been solved. - The 2DVAR Ambiguity Removal is improved. - A command line option "-calval" is added. If this option is invoked, the check on the Sigma 0 Usability Flag of the beams is skipped and the sigma 0 calibration are applied. Changes in AWDP v1.0h 22 February 2007 - The check on the Sigma 0 Usability Flag of the beams is removed (it is always set to "unusable" in the L1b ASCAT product during the cal/val). - A sigma 0 calibration is applied. Changes in AWDP v1.0g 24 January 2007 - Several bugs have been removed. - Take land fraction in level 1 sigma 0 into account to determine quality of beam information. If the land fraction is too high, the sigma 0 information is flagged as unusable. Changes in AWDP v1.0f 13 November 2006 - EARS System Test action AT.15 (sending of data_mon files after 6 hrs) implemented. - EARS System Test anomaly AR#15 (robust handling and killing of FTP sessions) fixed. - Several issues considering flagging of the data are solved. - Implemented Ambiguity Removal method as command line option (1strank, bgclosest, prescat, 2dvar). - Adaptations for new BUFR file name convention (see EUMETSAT document EUM/MET/TEN/06/0377). - Wind inversion at low wind speeds is improved. Changes in AWDP v1.0e 9 June 2006 - Changed use of satellite names in output file names: sat_id = 4 = metopa = metop02 (first mission) sat_id = 3 = metopb = metop01 (second mission) - EARS System Test anomalies AR#2, AR#3, AR#6, AR#7, AR#8, AR#9, AR#11, AR#12, AR#13 and AR#14 are fixed.