Disease Modelling Report
Understanding the impact of variations in the reliability of visual inspection on pest detection
Methods outline
An individual based model developed by Koh et al., 2025 for Huanglongbing (citrus greening) in Seville, was used to simulate the spread of a pest across a landscape. The model provided data on the infection status of 15,941 trees within a 49km2 plot. Each tree is categorised as susceptible, asymptomatic (i.e. infected but not detectable), or symptomatic (i.e. infected and detectable) with an average asymptomatic period of 365 days after initial infection (refer to Koh et al., 2025 for further details on the model). The model was run for 1800 days.
Previous experimental data was used to estimate the mean sensitivity (probability of correctly identifying a pest when present), specificity (probability of correcting identifying a non-infected plant), and variances of different groups of surveyors (trained volunteers, untrained volunteers with experience, and untrained volunteers without experience). Then, an expert group was developed with a higher mean sensitivity/specificity and reduced variance. The experimental data uses 164 surveyors of varying training/experience to assess two symptoms for acute oak decline, an easy symptom (weeping patches/bleeds) and a harder symptom (3-5mm D shaped exit holes). These means and variances were used to generate beta distributions for each group (Figure 1).
For detection curves (Figure. 2A), the proportion of detected true positive infected trees from those sampled on each survey day was calculated for each surveyor group.
Figure 1. Estimated beta distributions representing sensitivity and specificity for different surveyor groups. Each column represents the ‘Bleeds’ and ‘Holes’ symptoms, with the rows representing the ‘Sensitivity’ and ‘Specificity’ rows. Each line represents the beta probability density function for each surveyor group. Expert groups show tighter distributions, whilst volunteers have wider distributions.
Using these distributions, 9 different scenarios were simulated, changing parameters such as the number of trees observed per survey day (20, 50, 80 trees), survey intervals (30, 90, 180 days). Each scenario was simulated 1000 times, and the site prevalence at first detection and the proportion of true positives from trees sampled through the survey for each group was recorded.
To investigate the effect of variance on the simulations, simulations were ran using only the sensitivity and specificity means without accounting for variance.
Key results
Symptom difference
‘Easier’ symptoms such as bleeds that have higher mean sensitivities are often detected earlier than ‘harder’ symptoms such as holes. Across every simulation, the median detection days for bleeds are earlier/equal to holes for each group – and often holes increase the differences between each different experience/training group with experts outperforming each group, however there is large variation between groups (Figure 2). In the scenario with a surveyor investigating 20 trees every 30 days, experts had made their first bleed and hole detection at a median site prevalence of 15.3%, whereas untrained volunteers without experience had found their first bleed detection at site prevalence of 16.9%, with first hole detection being at 20.1% (Figure 2B).
Figure 2. Graph outputs from simulation where 1 surveyor inspected 20 trees every 30 days for 1800 days. (A) Detected true positives across 1800 days per sample across runs for bleeds and holes. Grey solid line is the site true prevalence, and grey dashed line is site detectable prevalence. Each graph plots the rate of true positives for each surveyor group. The coloured lines are the mean detection rates of 1000 simulations, and the shaded bands are 95% quantile ranges. (B) Distribution of prevalence at first detection days across 1000 simulations for bleeds and holes.Each boxplot represents the distribution of first detections across different surveyor groups in relation with the site prevalence.
When the site true prevalence was 30%, the mean detection rate of bleeds in each surveyor group is always higher than that of holes (Figure 2A). For example, the mean bleed detection rate of untrained volunteers with no experience was at 7%, whereas for holes it was at 3.6% - the gap is tighter for experts where the mean bleed detection rate was 9.7%, and for holes it was at 8.8%. This general trend is observed in all simulations with all parameters. As the site true prevalence increased, the differences between the groups became larger.
Impact of surveyor variance
Figure 3. Prevalence at first detection day boxplots for a scenario with 1 surveyor checking 20 trees every 30 days for 1800 days. Panel (A) uses a beta distribution and variances (Figure 1). Panel (B) shows the simulation using only mean sensitivity values and no variance.
The variation in first day detection for holes is larger, especially for untrained volunteers with no experience (Figure 3A). The variation of untrained volunteers for no experience for prevalence at first detection days is wide (range = 68.3%), whereas for experts the variation is tighter (range = 27.9%). The variation for the trained volunteers and untrained volunteers with experience are in between the variation of an expert and the variation of an untrained volunteer with no experience (ranges = 24.3%, 25.9% respectively).
However, for bleeds the variation for first day detection for untrained volunteers with no experience is much tighter (range = 36.1%) and more like experts (range = 29.5%). When removing variance from the surveyor sensitivity distributions (i.e. only using mean sensitivity values), it can be seen the variation is much tighter for untrained volunteers with no experience for holes (range = 31.1%) whereas for other groups there are relatively minimal changes (Figure 3B).
Impact of survey design
Figure 4. Prevalence at first detection day outputs from survey scenarios with different parameters investigated. (A) Simulations that used 1 surveyor looking at 20 trees every 30 days for 1800 days. (B) Simulations that used 1 surveyor looking at 80 trees every 30 days for 1800 days. (C) Simulations that used 1 surveyor looking at 30 trees every 180 days for 1800 days.
For the scenario that encompassed 20 trees and 30 survey interval days, comparing the prevalence at first detection across different groups shows that experts had a median prevalence at first detection at 15.3% site prevalence for bleeds, whereas untrained volunteers with no experience had a median prevalence at first detection at 16.9% (Figure 4A). For holes, this was 15.3% and 20.1% respectively.
As a surveyor inspected more trees, the prevalence at first detection was lower. For example, changing the number of trees inspected from 20 to 80 meant experts had a median prevalence at first detection at 7.7% for bleeds, and untrained volunteers with no experience had 10.6% (Figure 4B). For holes, it was 9.1% and 13.8% respectively. When surveyors had larger survey intervals, median prevalence at first detection increased. For example, when changing the survey intervals from 30 to 180 days, a median prevalence at first detection was observed at 24.8% for experts with bleeds, and 24.8% for untrained volunteers with no experience (Figure 4C). For holes, it was 24.8% and 34.7% respectively. It can be observed that increasing survey intervals also increased variation across groups too.
