Metop-B IASI L1 processing failures starting on Dec 4

[Liam Gumley]

Starting on Dec 4 around 10 UTC, all of our Metop-B IASI L1 processing has failed. We are running AAPP v8.10 and OPS-LRS v8.4. Nothing has changed in our software version recently. Multiple instances of AAPP 8.10 installed at separate antenna sites are failing when trying to process Metop-B IASI. See below for typical output from running the AAPP IASI L1 processor when this failure occurs.

> 2025/12/07 16:16:32 lightning2.ssec.wisc.edu 1504721 ops_process INFO : COPY IASI OPS SOFTWARE
> 2025/12/07 16:16:32 lightning2.ssec.wisc.edu 1504721 ops_process INFO : IASI OPS WARMING UP…
MP__MainProcess is a tracked alias for /opt/install/opslrs/OPS-LRS-run/OPS/perl/../bin/MP__MainProcess
1504811
> 2025/12/07 16:16:33 lightning2.ssec.wisc.edu 1504721 ops_process INFO : WAIT4ACK: (?^:ACK START) TIMEOUT: 60
> 2025/12/07 16:16:49 lightning2.ssec.wisc.edu 1504721 ops_process INFO : WAIT4ACK TROUVE: (?^:ACK START)
> 2025/12/07 16:16:49 lightning2.ssec.wisc.edu 1504721 ops_process INFO : OPS IASI START
> 2025/12/07 16:16:54 lightning2.ssec.wisc.edu 1504721 ops_process INFO : CREATE xxxx_OBT_xx_M01_xxxxxxxxxxxxxxZ_xxxxxxxxxxxxxxZ_20251207161654Z_xxxx_METOOBTUTC
> 2025/12/07 16:16:54 lightning2.ssec.wisc.edu 1504721 ops_process INFO : eps_metopl0-obt-xml /mnt/input/IASI_HRP_00_M01_20251205011913Z_20251205013046Z_N_O_20251207161621Z input/obtu/xxxx_OBT_xx_M01_xxxxxxxxxxxxxxZ_xxxxxxxxxxxxxxZ_20251207161654Z_xxxx_METOOBTUTC > execute-eps_metopl0-obt-xml.log 2>&1
> 2025/12/07 16:16:54 lightning2.ssec.wisc.edu 1504721 ops_process INFO : CREATE xxxx_SVM_xx_M01_20010101000000Z_20500101000000Z_20251207161654Z_FDFx_FDGEOEVENT
> 2025/12/07 16:16:54 lightning2.ssec.wisc.edu 1504721 ops_process INFO : satpos-svm /opt/install/AAPP_8.10/AAPP/orbelems/satpos/satpos_M01_20251205.txt input/svmu/xxxx_SVM_xx_M01_20010101000000Z_20500101000000Z_20251207161654Z_FDFx_FDGEOEVENT.x > execute-satpos-svm.log 2>&1
> 2025/12/07 16:16:55 lightning2.ssec.wisc.edu 1504721 ops_process INFO : SELECT CONFIG IASI_BRD_xx_M01_20220916000000Z_xxxxxxxxxxxxxxZ_20220915154840Z_IAST_0000000019
> 2025/12/07 16:16:55 lightning2.ssec.wisc.edu 1504721 ops_process INFO : SELECT CONFIG IASI_BRD_xx_M01_20230111000000Z_xxxxxxxxxxxxxxZ_20230110095228Z_IAST_0000000020
> 2025/12/07 16:16:55 lightning2.ssec.wisc.edu 1504721 ops_process INFO : SELECT CONFIG IASI_BRD_xx_M01_20231103000000Z_xxxxxxxxxxxxxxZ_20231102131824Z_IAST_0000000020
> 2025/12/07 16:16:55 lightning2.ssec.wisc.edu 1504721 ops_process INFO : SELECT CONFIG IASI_BRD_xx_M01_20240119000000Z_xxxxxxxxxxxxxxZ_20240118150955Z_IAST_0000000020
> 2025/12/07 16:16:55 lightning2.ssec.wisc.edu 1504721 ops_process INFO : SELECT CONFIG IASI_BRD_xx_M01_20240405000000Z_xxxxxxxxxxxxxxZ_20240404125638Z_IAST_0000000020
> 2025/12/07 16:16:55 lightning2.ssec.wisc.edu 1504721 ops_process INFO : SELECT CONFIG IASI_BRD_xx_M01_20250528000000Z_xxxxxxxxxxxxxxZ_20250527120916Z_IAST_0000000020
> 2025/12/07 16:16:55 lightning2.ssec.wisc.edu 1504721 ops_process INFO : SELECT CONFIG IASI_CTX_xx_M01_20061221045654Z_xxxxxxxxxxxxxxx_20061221171712Z_CMSx_xxxxx00000
> 2025/12/07 16:16:55 lightning2.ssec.wisc.edu 1504721 ops_process INFO : SELECT CONFIG IASI_GRD_xx_M01_20220916000000Z_xxxxxxxxxxxxxxZ_20220915154844Z_IAST_0000000028
> 2025/12/07 16:16:55 lightning2.ssec.wisc.edu 1504721 ops_process INFO : SELECT CONFIG IASI_GRD_xx_M01_20231103000000Z_xxxxxxxxxxxxxxZ_20231102131828Z_IAST_0000000029
> 2025/12/07 16:16:55 lightning2.ssec.wisc.edu 1504721 ops_process INFO : SELECT CONFIG IASI_GRD_xx_M01_20240119000000Z_xxxxxxxxxxxxxxZ_20240118150959Z_IAST_0000000030
> 2025/12/07 16:16:55 lightning2.ssec.wisc.edu 1504721 ops_process INFO : SELECT CONFIG IASI_GRD_xx_M01_20240405000000Z_xxxxxxxxxxxxxxZ_20240404125641Z_IAST_0000000031
> 2025/12/07 16:16:55 lightning2.ssec.wisc.edu 1504721 ops_process INFO : SELECT CONFIG IASI_GRD_xx_M01_20250528000000Z_xxxxxxxxxxxxxxZ_20250527120924Z_IAST_0000000032
> 2025/12/07 16:16:55 lightning2.ssec.wisc.edu 1504721 ops_process INFO : SELECT CONFIG IASI_ODB_xx_M01_20130417100000Z_xxxxxxxxxxxxxxZ_20130417082506Z_IAST_0000000012
> 2025/12/07 16:16:55 lightning2.ssec.wisc.edu 1504721 ops_process INFO : SELECT CONFIG IASI_ENG_01_M01_xxxxxxxxxxxxxxZ_xxxxxxxxxxxxxxZ_V_T_xxxxxxxxxxxxxxZ
> 2025/12/07 16:16:55 lightning2.ssec.wisc.edu 1504721 ops_process INFO : SELECT CONFIG IASI_VER_01_M01_xxxxxxxxxxxxxxZ_xxxxxxxxxxxxxxZ_V_T_xxxxxxxxxxxxxxZ
> 2025/12/07 16:16:55 lightning2.ssec.wisc.edu 1504721 ops_process INFO : SELECT CONFIG IASI_xxx_1A_M01_xxxxxxxxxxxxxxZ_xxxxxxxxxxxxxxZ_V_T_xxxxxxxxxxxxxxZ
> 2025/12/07 16:16:55 lightning2.ssec.wisc.edu 1504721 ops_process INFO : SELECT CONFIG IASI_xxx_1B_M01_xxxxxxxxxxxxxxZ_xxxxxxxxxxxxxxZ_V_T_xxxxxxxxxxxxxxZ
> 2025/12/07 16:16:55 lightning2.ssec.wisc.edu 1504721 ops_process INFO : SELECT CONFIG IASI_xxx_1C_M01_xxxxxxxxxxxxxxZ_xxxxxxxxxxxxxxZ_V_T_xxxxxxxxxxxxxxZ
> 2025/12/07 16:16:55 lightning2.ssec.wisc.edu 1504721 ops_process INFO : STORE IASI_HRP_00_M01_20251205011913Z_20251205013046Z_N_O_20251207161621Z
> 2025/12/07 16:16:55 lightning2.ssec.wisc.edu 1504721 ops_process INFO : STORE AVHR_xxx_1B_M01_20251205011914Z_20251205013045Z_x_x_20251207161632Z
> 2025/12/07 16:16:55 lightning2.ssec.wisc.edu 1504721 ops_process INFO : PROCESS IASI_HRP_00_M01_20251205011913Z_20251205013046Z_N_O_20251207161621Z
> 2025/12/07 16:16:55 lightning2.ssec.wisc.edu 1504721 ops_process INFO : STEP input/IASI_9_wo_001
> 2025/12/07 16:16:55 lightning2.ssec.wisc.edu 1504721 ops_process INFO : WAIT4ACK: (?^:ACK STEP 1 0) TIMEOUT: 60
> 2025/12/07 16:16:59 lightning2.ssec.wisc.edu 1504721 ops_process INFO : WAIT4ACK TROUVE: (?^:ACK STEP 1 0)
> 2025/12/07 16:16:59 lightning2.ssec.wisc.edu 1504721 ops_process INFO : WAIT4ACK: (?^:ACK STEP 1 ([1\d])) TIMEOUT: 1800
> 2025/12/07 16:16:59 lightning2.ssec.wisc.edu 1504721 ops_process INFO : WAIT4ACK TROUVE: (?^:ACK STEP 1 ([1\d]))
> 2025/12/07 16:16:59 lightning2.ssec.wisc.edu 1504721 ops_process ERROR : STEP COMMAND FAILED ( End of treatment: L1a – Other parameters configuration file not found )

> 2025/12/07 16:17:01 lightning2.ssec.wisc.edu 1504721 ops_process ERROR : OPS IASI ABORT
OPS IASI ABORT : !!
OPS IASI ABORT : !!
OPS IASI ABORT : !!
OPS IASI ABORT : !!
OPS IASI ABORT : !!
OPS IASI ABORT : !!
OPS IASI ABORT : !!
OPS IASI ABORT : !!
OPS IASI ABORT : !!
OPS IASI ABORT : !!

[Author]

Posts