11 March 2019. 10.30 to 17.00

Register at B1.37, Zeeman Building

Coffee, Lunch and Tea in Mathematics Common Room (opposite B1.37)

Talks will be in Room B3.02, Mathematics Institute, Zeeman Building

Organisers: Simon French (Statistics), Leroy White (Business School)

Please note that you need to register for the meeting: see panel to the right.

By soft elicitation we mean what we do when we ask problem owners for the general knowledge, perceptions, uncertainties and values that a model needs to embody before it is populated with numbers and used in a quantitative analysis. We use soft elicitation to contrast with the hard elicitation of numbers that go into models, though we recognise that the distinction is a vague rather than a crisp dichotomy.

Many disciplines have thought about soft elicitation, though perhaps not as much as might be expected. Various terminologies have evolved, including:

• Mathematicians often refer simply to model building
• Decision analysis and more generally Operational Research refer to Soft OR and Problem Structuring methods.
• Risk Analysis use the terms: Optioneering and Hazard Analysis.
• Statistics uses Exploratory Data Analysis, Multivariate Statistics and, nowadays, Data Mining and Machine Learning.
• Knowledge Management studies refer to knowledge elicitation and, perhaps, sense-making, creativity and innovation.
• Information Systems often use the term Soft Systems.
• Management Studies has a whole host of model structures that can catalyse discussions, such as PESTEL, 7 S's and Porter's 5 Forces.

The seminar is aimed at academics and consultants who build quantitative models to solve real problems, particularly risk and decision problems. We intend to draw together perspectives from across the disciplines and share insights into the processes that are used to elicit the form and structure of models. The format of the seminar will include talks and a panel led discussion.

Programme

Time	Speaker	Title
10.30 – 11.00		Registration and Coffee
11.00 – 11.05	Leroy White Warwick Business School	Welcome and Introduction
11.05 – 11.25	Simon French University of Warwick	What is soft elicitation and what are the issues?
11.25 – 11.50	Charles Featherstone Government Office of Science	Futures
11.50 – 12.15	Sunny Modhara Network Rail	The Use of Soft Elicitation at Network Rail for Lifecycle Cost Modelling
12.15 – 12.40	Sophia Wright University of Warwick	Creating Robust Bayesian Networks
12.40 – 13.30		Lunch and networking
13.30 – 13.55	Leroy White Warwick Business School
13.55 – 14.20	Alberto Franco University of Loughborough	Beyond scripts: Making soft elicitation more ‘visible’
14.20 – 14.45	Jon Malpass BT	Soft Elicitation in practice: some experiences from BT
14.45 – 15.10	Ine Steenmans UCL	The anatomy of anticipation - eliciting intelligence requirements for policy foresight
15.10 – 15.30	Tea
15.30 – 16.30	Panel-led discussion	Colm Connaughton (Chair, University of Warwick)), Martine Barons (University of Warwick), Valentina Ferretti (LSE), John Fox (University of Warwick) Daniel Read (Warwick Business School)
16.30	Close and departure

Please note that registration, though free, will be required for the meeting to enable us to plan catering. See panel to the right or click here.

Abstracts

What is soft elicitation and what are the issues?

Simon French (University of Warwick)

In this talk, I want to reflect on the the modelling process from beginning with a mess of issues, making some sense of them, building models, perhaps a family of models, to represent the problem owners and their stakeholders perceptions. In some cases, the process produces simply structured models that capture general behaviours. In others, very large computational codes build on complex mathematical models. However, the focus will be on the process not the ultimate model(s).

The Use of Soft Elicitation at Network Rail for Lifecycle Cost Modelling

Sunny Modhara (Network Rail)

Network Rail owns and manages the national railway infrastructure in Great Britain. Our role is to provide the best and most efficient service possible to everyone who relies on the railway – passengers, the train and freight operating companies and businesses. Key to fulfilling this role are asset management decisions that determine the work undertaken on the railway infrastructure, that drive the majority of the cost of running the railway and that underpin the performance and safety of the network. Optimising these decisions and implementing them effectively will have a significant bearing on the future competitiveness of rail in a rapidly changing transport sector.

For almost twenty years, we have been developing models and information systems to support better asset management decisions, ranging from long-term decisions that direct investment in the railway to short term decisions on maintaining the railway to deliver the required level of train performance. Where possible, the models are data driven, based on a large amount of information generated from manual inspections or from automated measurements using sensors attached to trains and the infrastructure. Where data is not available, expert (usually engineering) judgement is relied on. This talk provides examples of where expert knowledge elicitation has been used to inform the modelling relationships and discusses some of the elicitation principles behind the knowledge acquisition. The general approach is multi-phased, ranging from problem set up and the collection of initial beliefs and judgements, through to the fitting of distributions and the revision of these in light of feedback. The talk will also describe how the increased data collected in recent years means that model relationships are effectively determined using a combination of elicitation and data.

Creating Robust Bayesian Networks

Sophia Wright (University of Warwick)

Now that Bayesian Networks (BNs) have become widely used, an appreciation is developing of just how critical an awareness of the sensitivity and robustness of certain target variables are to changes in the model. We have developed new methodology, underpinned by the total variation distance, to determine whether simplifications which are currently employed in the practical implementation of such systems are theoretically valid. Unlike current robustness analyses, our new technology can be applied throughout the construction of the BN model; enabling us to create tailored, bespoke models. Here we focus on two particular choices a modeller needs to make: the choice of the parents of each node and the number of levels to choose for each variable within the system, providing guidance for the soft elicitation stage. For illustrative purposes we shall be discussing the field of Food Security.

Beyond scripts: Making soft elicitation more ‘visible’

Alberto Franco (Loughborough University)

Soft elicitation as an element of model building has received attention by scholars in the Operational Research community since the 1960s, and a wide array of soft elicitation methods has been developed ever since. Operational researchers who engage in soft elicitation typically rely on ‘scripts’, namely, a sequence of designed activities that generate products such as a problem definition, a draft model structure, a value model or a subjective probability distribution. The use of scripts is by no means trivial, as they demand considerable skill in managing content and process simultaneously. Hence, scholars argue for the need to share scripts among the community to both, help disseminate knowledge of tried and tested scripts and also alleviate a long-standing concern regarding the transferability of soft elicitation methods.

Despite these contributions, we still know very little about the complexities and situated specifics of soft elicitation practice as it happens on the ground. This is mostly due to the methodological challenges involved in treating real-time soft elicitation practice as an analytical problem, which requires making this practice ‘visible’ by bringing to the fore its material and interactional features for close examination. In this talk, I will introduce ethnomethodology as one way to address this challenge. Using empirical vignettes drawn from a range of soft elicitation workshops, I will illustrate how an ethnomethodologically-informed perspective can offer a more nuanced approach to the understanding of soft elicitation in situ, and even challenge well-established scripts. I will end the talk by discussing some implications for soft facilitation training.

Soft Elicitation in practice: some experiences from BT

Jon Malpass (BT Applied Research)

Transformation and change projects tend to be designed by knowledge experts who are, generally, not part of the day-to-day operational level. This means that there is potential for making incorrect assumptions about working practices and behaviours that could cause problems when implementing the project, but also for mis-interpreting data. Consequently it is vital that designers understand as much about the environment that they are planning to change in order to ensure an appropriate and effective change. In this presentation we will discuss the “data onion” model that we use to identify where assumptions may lead to problems, briefly describe a number of research methods that we use in BT to understand operational practices and illustrate these concepts using a case study.

The anatomy of anticipation - eliciting intelligence requirements for policy foresight

Ine Steenmans (UCL)

This talk explores the elicitation of beliefs about the nature of distant future policy challenges. Policy ‘foresight’ and its related analytic activities capture a set of practices for uncovering the nature of complex issues emerging on distant time ‘horizons’ as early as is feasible. The policy challenges selected for such foresight analysis are characterised by a number of distinctive and critical features that effect the related elicitation processes. First, the issues selected for analysis are intentionally little previously considered, and therefore typically largely under-explored terrain. Second, their substance span is typically highly cross-sectoral and highly ill-structured. Finally, they are often still emergent as ‘problems’, where any attempt at describing constituent goals, issues or system behaviours is deeply uncertain. Yet, from an analytical point of view, there is a (public) imperative to seek a systematic methodological means of eliciting insight about the nature of these foresight challenges in as comprehensive a manner, as rapidly as possible. 

The questions I seek to answer are therefore: is there a generic anatomy of deeply uncertain future policy challenges? If so, can such an anatomy be used to structure soft elicitation for policy foresight? I use a critical realist methodology to synthesise relevant contributions from the strategy development, futures and decision analysis literatures to propose 10 fundamental anatomical ‘parts’ (organs?) that constitute policy foresight challenges. I will elaborate on each of these 10 anatomical constructs and demonstrate how these come together into one integrated ‘anatomy map’ that can be used to elicit the requisite foresight intelligence. I conclude by sharing examples of the ways that this anatomy map has been used on various project in the past 2 years to structure design protocols for futures workshops and structure policy analytics educational programmes.

Other abstracts to follow

See also:
Mathematics Research Centre
Mathematical Interdisciplinary Research at Warwick (MIR@W)
Past Events 
Past Symposia