Skip to main content Skip to navigation

WBS Research Projects Automated Simulation Output Analysis

Automated Simulation Output Analysis

Abstract light and shadows

Professor Stewart Robinson, Dr Katy Hoad, Professor Ruth Davies

AutoSimOA Diagram

Figure 1: Overview of the Automated Analysis Procedure

Appropriate use of a simulation model requires accurate measures of model performance. This in turn, requires decisions concerning three key areas: warm-up, run-length and number of replications. Simulation software, however, gives little or no guidance in making these decisions. A three year project, "Automatic Simulation Output Analysis", sponsored by the EPSRC (Grant ref: EP/D033640/1) and SIMUL8 and undertaken by Warwick Business School, is investigating the development of a methodology for automatically advising a simulation user on these three decisions. This web site outlines the approach being taken, discusses progress to-date and provides links for research outputs.


Obtaining Accurate Simulation Results

Even with the best constructed simulation model if it is not used properly it is likely that the results will be inaccurate. If the results from a simulation of a system are not accurate they will not provide a good prediction of what would happen in the real life system even if the model itself is valid. The results could be biased and misleading, with the consequence that incorrect conclusions could be drawn.

There are many decisions that need to be made about how a simulation model is to be run. These include:

  • Do I need to use a warm-up period; set initial conditions; or perhaps use a combination of both; or do I in fact want my simulation to start from empty ?
  • If I do use a warm-up period how long should this period be ?
  • Should I perform one long simulation run, or a number of independent runs (i.e. runs with different random streams, also called replications) ?
  • How long should this one simulation run be ?
  • How many replications is enough ?

In particular, the decisions on run-length, number of replications to perform in a trial and warm-up period require specific skills in statistics. Because of the variety and complexity of statistical methods used to make informed decisions on these matters, at the moment, most simulation software provide little or no guidance to users on making these important decisions. Because the setting of inappropriate run-lengths, run (replication) numbers and warm-up periods can lead to inaccurate and misleading results, it is considered important to deliver tools to simulation users that allow them to make informed decisions regarding these matters, whatever their statistical background.

It is with this in mind that Warwick Business School and SIMUL8 have collaborated on a project to create an automated tool that can analyse preliminary simulation output and hence provide recommendations on warm-up, run-length and number of replications.


Refs:

 

Project Personnel

Principal Web Site editor & Research Fellow: Dr Katy Hoad

Operational Research and Management Sciences Group
Warwick Business School
University of Warwick
Coventry
CV4 7AL
UK
Contact e-mail: kathryn dot hoad at wbs dot ac dot uk


Principal Investigator: Professor Stewart Robinson

Professor of Operational Research Associate Dean for Specialist Masters
Operational Research and Management Sciences Group
Warwick Business School
University of Warwick
Coventry
CV4 7AL
UK
e-mail: stewartrobinson at warwick dot ac dot uk


Co-Investigator: Professor Ruth Davies

Professor of Operational Research and Systems
Associate Dean for Academic Policy
Operational Research and Management Sciences Group
Warwick Business School
University of Warwick
Coventry
CV4 7AL
UK


Project Partner: SIMUL8 Corporation

Contact: Dr Mark Elder
141 St. James' Road
Glasgow
G4 0LT
Scotland