• IR Sub-Group, Meeting

GSICS Web Meeting on Bootstrap Approach to Uncertainty Estimation 2026-08-27

Agenda

Jorge Gil (DEIMOS): Bootstrap Approach to Uncertainty Estimation

Attendees

CAS: Wei Cai, Jinpeng Miao

EUMETSAT: Mounir Lekouara, Tim Hewison, Vinia Mattioli, Sebastien Wagner, Timo Hanschmann, Viju John, Vincent Debaeker, Ali Mousivand

ESS: Thijs Krijger

Indra: Jorge Gil, Diego Lozano García

JMA: Shin Koyamatsu, Tsuguyoshi Toma

KMA: Euidong Huang, Geunhyeok Ryu, Hanbyul Lee, Sunmi Na

Muon Space: Stephen Maxwell

NOAA: Likun Wang, Fangfang Yu

NPL: Emma Wooliams, Jon Mittaz

UPC: Miriam Pablos

Summary

By way of introduction, Jorge provided an  overview of the analysis of the uncertainties in the GSICS GEO-LEO IR inter-calibration algorithm, published by Hewison, 2013: doi:10.1109/TGRS.2012.2236330. That paper did not consider correlation between multiple sources, which could be a significant shortcoming.

He also went on to introduce the Bootstrapping approach used to characterise the distribution of samples used in the inter-calibration, and the sensitivity of the resulting biases to different observational parameters. This is a new approach to uncertainty estimation, which exploits computational power to increase the sample sizes in a statistically consistent way, allowing us to revisit some of the assumptions and approximations previously made.

In the case of the OCTM method presented, it was suggested that this be extended to exploit the GEO-ring dataset, instead of only a single Meteosat’s observations. This could be beneficial for increasing the sample size and extend the dynamic range coverage. However, care would need to be taken that including GEO-ring data does not introduce additional sources of uncertainty from multiple instruments, which could completely compromise the whole analysis. Although the method is robust to radiometric calibration biases, geolocation errors, spectral mismatch and other issues have been found in the GEO-ring data.

Discussions

There followed several in-depth discussions, in which the following points were highlighted:

Correlations

  • The correlation between different sources of uncertainty has not been treated explicitly in the analysis presented. This was considered and the correlations found to be small. But in general this should be checked for application to other inter-calibration algorithms.

 

Sampling

  • The key of the bootstrap is to have an initial sample dataset that is big enough and representative of the “truth”.
  • How to check there are sufficient samples? How many is enough? e.g. tens of samples would not yield statistically robust results.
  • In the examples presented today, the opposite problem was considered – often millions of samples were available – the question then becomes how to characterise their sensitivity to multiple geophysical parameters.
  • However, dynamic range checks were included in the analysis, as one goal of the study was to achieve the required levels of uncertainty across the broadest possible range of observation space.

 

Measurement Model

  • It is important to link the model of the uncertainties to the measurement model of the instrument in question: What are the dominant sources of uncertainty in the calibration?
  • Some inter-calibration algorithm may introduce limitations due to the sampling of the data used – for example, SNOs from sun-synchronous satellites only occurring in high latitudes. This could mask underlying variations – e.g. diurnal ones. So, it is important to check the timescales over which calibration variations are possible, and that these are well-sampled.
  • The assumption of a linear model can introduce errors – including, but not limited to, any nonlinear response of the instrument’s calibration. Are higher order regressions needed? It may be possible to include these in the bootstrap itself.

 

Guidance to apply bootstrapping approach to uncertainty assessment

  • In addition to Jorge Gil, Thijs Krijget and Stephen Maxwell have some experience in the application of the bootstrapping approach, which are not limited to uncertainty analysis. e.g.  Pintar et al,. 2023, doi:10.1088/1681-7575/acb5f8):10.1088/1681-7575/acb5f8. However, none of the attendees were aware of any other publication that provides a good introduction to this topic.

 

Recommendation: R.GRWG.20260827.1: NPL to consider generating guidelines outlining the considerations to be addressed when applying the bootstrapping approach to uncertainty assessment.

  • This may be particularly relevant to the application of Fiducial Reference Measurements (FRMs), where the number of matchup samples is typically very limited.