When you measure particle sizes, indentation diameters or cell areas with OpenCV, ImageJ / Fiji or QuPath, the program always returns a number that looks very precise, often to four decimal places. Whether that number can be trusted depends on the whole measurement chain: camera and lens, calibration, illumination, segmentation algorithm, parameters, and how all of it was validated.

This article is not about a particular algorithm. It explains how to show that an image measurement method is fit for research or inspection.

Standards and references used: JCGM 100:2008 (GUM, evaluation of measurement uncertainty), ISO 5725-1:2023 (accuracy, trueness and precision), Bland & Altman (1986), and ISO 6506-2:2017 as an example of an acceptance criterion written into a test standard.

The formulas, code and numbers in this article illustrate the method. Values marked "assumed" are not measured data.

1. Separate two kinds of error

  • Systematic error (bias): the result is off in the same direction every time. If the calibration factor is 1% too large, every length is about 1% too large. Repeating the measurement does not remove it.
  • Random error (precision): the result scatters from one measurement to the next; its size can be estimated by repeating measurements.

ISO 5725 describes these as "trueness" and "precision"; together they make up "accuracy". When validating an image method, both need to be assessed.

2. Sources of error

Step Typical problem Main effect
Calibration (mm/px) Reading error on the reference scale, no recalibration after a magnification change Systematic error, scales every length
Lens Distortion, stronger towards the edges of the field Position-dependent systematic error
Focus and illumination Blur, shadows, uneven lighting Shifts the edge position, often as a bias
Segmentation Threshold, amount of smoothing, morphological parameters Systematically enlarges or shrinks the object
Pixel quantisation Edges can only be located to a pixel (or sub-pixel) Random error
The object itself Blurred boundary, irregular shape, overlaps Uncertainty in the definition

The last item is often overlooked: if even a person cannot say exactly where the boundary is, the program's result cannot be more certain than that definition. Write down the boundary rule before validating.

3. Calibration

  1. At the same magnification, focus and resolution as the measurement, photograph a reference scale of known length, such as a stage micrometer.
  2. Measure the scale at several positions and orientations, and record the mean mm/px and its spread.
  3. If the edges and the centre of the field give clearly different results, correct the lens distortion (OpenCV provides a camera calibration method) or measure only in the central area.
  4. Recalibrate whenever the lens, camera, magnification or resolution setting changes, and record the date.

4. Uncertainty from pixel quantisation

If an edge can only be located within ±0.5 pixel, with every position in that interval equally likely, the GUM rectangular distribution gives a standard uncertainty of:

u = a / √3 = 0.5 / √3 ≈ 0.29 pixel

A length is defined by two edges, so the quantisation uncertainty of a length is about √2 × 0.29 ≈ 0.41 pixel. This leads to a practical rule: the more pixels the object spans, the smaller the relative effect of quantisation. For a diameter of 100 pixels it is about 0.4%; for 1000 pixels, about 0.04%.

4a. Combining the components: an uncertainty budget

Under the GUM, independent components are combined as the square root of the sum of their squares. A calculated example for a length of 400 pixels (2.000 mm at an assumed calibration of 0.005 mm/px):

Component Type Relative standard uncertainty
Calibration factor (reference scale and its reading) B 0.20% (assumed)
Pixel quantisation, 0.41 px over 400 px B 0.10%
Repeatability of acquisition and evaluation A 0.30% (assumed)
Combined standard uncertainty 0.37%
Expanded uncertainty (k = 2) 0.75%

Two things follow from the arithmetic:

  • The largest component dominates. Halving the quantisation term here changes the combined value by less than 0.02 percentage points; improving calibration or repeatability matters far more.
  • Errors propagate through any formula the length goes into. In Brinell hardness, for example, the relative error in hardness is about twice the relative error in the diameter, so 0.75% in diameter becomes about 1.5% in hardness.

The budget does not contain a "segmentation" line because a segmentation bias is not random: it has to be found by comparison with a reference method (section 6) and then either corrected or added as a known bias.

5. Repeatability and reproducibility

  • Repeatability: the same operator and equipment capture and evaluate the same object several times (repositioning and refocusing each time), and the standard deviation of the results is calculated.
  • Reproducibility: the same is repeated with a different operator, day or instrument.

Running the program repeatedly on the same image always gives the same answer, which says nothing about the stability of the method. Repetition must include image acquisition.

6. Agreement with a reference method

Compare the image method with an accepted reference method (for example an experienced operator using a measuring microscope, or certified reference materials) on the same set of samples. The Bland–Altman method is widely used:

  1. For each sample, calculate the difference dᵢ = image method − reference.
  2. Bias = mean of the differences.
  3. Limits of agreement = bias ± 1.96 × standard deviation of the differences.
  4. Plot the difference against the mean of the two methods and check whether the difference changes with size.
import numpy as np


def bland_altman(method, reference):
    a = np.asarray(method, dtype=float)
    b = np.asarray(reference, dtype=float)
    diff = a - b
    bias = diff.mean()
    sd = diff.std(ddof=1)
    return {
        "n": diff.size,
        "bias": bias,
        "lower_loa": bias - 1.96 * sd,
        "upper_loa": bias + 1.96 * sd,
        "mean_pair": (a + b) / 2,  # x axis of the plot
        "diff": diff,
    }

Points to note:

  • Do not report only a correlation coefficient. Two methods can be highly correlated without agreeing; a method that is always 5% too high can still have a correlation close to 1.
  • Whether the limits of agreement are acceptable must be decided beforehand according to the purpose, for example by comparing them with the deviation allowed by the test standard or the product tolerance. This article gives no universal threshold. Some test standards do set one: ISO 6506-2:2017, for example, allows a Brinell indentation measuring system a maximum error of 0.5% of the diameter when it is verified against reference indentations.
  • Keep the samples used to tune parameters separate from those used for validation; otherwise the validation will look better than it is.

7. Handling failures

Instead of returning a number, the program should detect and clearly report cases such as:

  • no object found, or several candidates;
  • the object touches the image border and may be incomplete;
  • abnormal shape measures (for example an axis ratio far from what is expected);
  • a result outside the valid range of the method.

Flagged images are reviewed by a person, and the outcome of the review is recorded.

8. What the report should state

  • camera, lens, magnification, resolution and illumination;
  • calibration method, reference scale and calibration date;
  • software, version and all parameters (or the version of the code);
  • the boundary rule;
  • repeatability and the comparison with the reference method (bias and limits of agreement);
  • how many automatic results failed or were corrected by hand, and how they were handled.

This lets others reproduce the measurement, and gives reviewers and customers a basis for judging how reliable the results are. For an application, see the article "Measuring Brinell indentation diameters with OpenCV" on this site.

9. Limitations of this article

  • It describes a validation approach; it is not a substitute for the requirements of a specific test standard or accreditation scheme.
  • The uncertainty budget is a worked example with assumed values. Your components and their sizes must come from your own calibration and repeatability data.
  • Bland–Altman limits of agreement assume that the differences are roughly normally distributed and do not change with size; check the plot before relying on them.