WARNING: This text has been OCRd from the original paper and so will contain many typographical errors. It should be useful for searching, however. [Ash, August 2002]. Worlds Before and Beyond Words Meurig Beynon Paul Ness Steve Russ Department of Computer Science, University of Warwick, Coventry CV4 7AL, UK Abstract The advent of computer-generated environments for simulating experience of the real world invites a reappraisal of the role of classical and neo-classical theories of computation (e.g. views of computation based on linguistic and logical frameworks). To make more effective use of computers in connection with Virtual Reality requires a shift in emphasis towards computers as state representers rather than calculators, towards the construction and analysis of environments rather than documents and towards empirical rather than theoretical approaches to knowledge representation. This paper discusses the prospects for gaining new insights into empirical activity through the systematic construction of models (typically - but not necessarily - computer based) in which real world state, as perceived by a particular agent, is imitated directly through the use of suitable metaphors. Such activity is exemplified in the construction of engineering models and the development and calibration of scientific instruments. A characteristic ingredient of the modelling processes involved in these applications is correlation of the results of experiments and observations that are performed in parallel in a real world environment and in the associated physical (e.g. computer) model. Conventional computer programming paradigms are not well-suited for constructing models by empirical methods. Amongst widely used software tools, only spreadsheets are well adapted to imitating real world state as it is captured through observation and experiment. The philosophical ideas to be introduced in the paper have been developed in connection with practical case-studies and software tools that contribute to a long-term Empirical Modelling Project being pursued at the Computer Science Department at the University of Warwick. A key concept in this research programme is the development of agent-oriented models in which the dependencies between observables that are used for communication are specified using generalised spreadsheet principles. 2 1. Introduction Our agenda is suggested by a programme of research ("The Empirical Modelling Project") that has been carried out under the direction of the principal author for some ten years. The scope of this project is wide, and is represented in papers devoted to a variety of themes, including engineering design and simulation [7,12], concurrent systems modelling [3,7], concurrent engineering [1,2] and software development [5,8]. This research has both practical and foundational aspects and has led to the development of a suite of software tools. A detailed technical exposition of our empirical modelling method is beyond the scope of this paper (see the Appendix for a brief synopsis of the Empirical Modelling Project and some illustrative screenshots). Our concern is to explore the significance of our methods in relation to the goals of Virtual Reality (VR) research. As the title of our paper indicates, our modelling method differs from classical language-based modelling. Its antecedents are to be found in the techniques for communicating designs and conceptions of the real-world through using physical objects whose features are imitative rather than symbolic. Analogue models of this nature (as developed by engineers, craftsmen and artists) can be viewed as a pre-electronic precedent for VR. Like VR models, they rely on conveying content through inviting the user to experience an environment rather than interpret a document. In this respect, they are sharply distinguished from traditional mathematical models that sit comfortably within the framework of the classical theoretical framework for computation developed by Turing. Understanding the relationship between our approach and traditional computer-based modelling methods is a challenging problem that has led us to philosophical investigations outside the scope of our original concern. In The Metaphysics of Virtual Reafity, Michael Helm [11] highlights the difficulty of defining the concept of VR. To some extent, this difficulty can be attributed to the 3 profound metaphysical issues surrounding the concept of reality. Our search for foundations and principles behind our empirical modelling method has also drawn us into this awesome territory. What is an imitative physical object, or an environment for interaction, and in what sense are these objects and environments artificial or real? Our actual and projected case-studies naturally lead us to consider the concept of reality from a variety of viewpoints. We may consider "what really happens" at a cricket match, and in what sense having keyboard skills is a personal reality. Such considerations are part of the metaphysical framework we must ultimately develop to account for our approach. On the other hand, it is convenient at the outset of our discussion to take a rather ingenuous stance on what is real and focus on reality as it is typically and perhaps naively encountered by the engineer, rather than the typist, the cricketer or the theoretical physicist. After all, whatever its ontology, the universal impact of building the Severn Bridge upon people in general is not to be denied. And though such an unsophisticated concept of reality is in some respect prosaic, it can also be invested with a mystical quality as an inexhaustible source of experience. The idea that real objects transcend any finite experience we can gain through particular modes of observation and interaction is particularly relevant to our philosophical stance. It is vividly expressed metaphorically in these words of the poet Edward Thomas: "1 cannot bite the day to the core". 2. Empirical Modelling and VR: the Engineering Perspective Most of our sponsored research to date has necessarily been concerned with engineering applications. A typical case-study in this area was the development of a Vehicle Cruise Controller Simulation (VCCS) - see [7], and the screenshot in the Appendix. In evaluating our VCCS model, we have been concerned with the degree of realism that could in principle be achieved using our methods, and the extent to which our model can be made faithful to real-life observation and interaction with a vehicle cruise controller. In this aspect, our work addresses what 4 Helm identifies as simulation and interaction in the "seven divergent concepts guiding VR research" that characterise VR [11]. In Helm's terms, simu/ation is associated with realistic evocation of sensory impressions (as in the use of photorealistic real-time texture-mapped images in a flight simulator), and interaction with the construction of [electronic] representations with which the user can interact. To put our views of simulation and interaction in context it is convenient to ask the question: in what sense is VR an essentially 20th century phenomenon? In this way, we can get a helpful perspective on what the technology of computers adds to our understanding of the modelling process, and how our contemporary explanations of the natural world influence our approach to modelling and our assessment of its significance. We can usefully regard Helm's simulation and interaction as a separation of concerns: fooling the senses, and fooling the mind. In modelling a cat, for instance, we would like to imitate the sensory experience: to convey what a cat looks like, feels, sounds and smells like. We would also require insight into the true scope for interaction: to know what sort of experiences we can have with a cat. It is consistent with our philosophical stance on reality that we should ideally expect more of VR than captures a circumscribed aspect of catness. A cat is after all an extraordinarily rich source of experience, and a fixed mode of observation and interaction cannot do justice to the exploratory nature of the cat concept. It is clear that our century has brought enormous developments in the technology we can use to imitate sensory experience. Where in former times we had to rely upon the artist's impression, the photograph, the stuffed cat or the china cat we can now construct a hologram. Perhaps it is the development of such technology that prompts us to suppose that we are ready to mimic reality, but in this we may be misled. There are potentially difficult problems - both conceptual (how many ways are there to sense a cat?) and technical (how do we imitate the smell of a cat?) - to address in imitating different modes of cat observation. What is more, though the means to evoke a particular perception of the current state of a cat is a prerequisite for constructing a VR model it is not by any means sufficient. Our experience of cats in fiction - as in TopCat, Tom and Jerry, T.S. Elliot's Cats, and Cats: the musical- illustrates that realism in interaction has a deep conceptual aspect. To what extent does power to fool the senses help us to represent behaviour in interaction? If we could equip an educated person of the first Elizabethan age with the technology for VR, their realisation of VR would surely reflect the explanatory systems of their age: the Chain of Being [14], by which events are attributed to agents in a hierarchy that begins with God and passes to the earthworm via planets, royal personages and peasants. Is it plausible that in such a VR we could construct the artefacts of our own age: the aeroplane, the television or the nuclear reactor? The explanatory systems of today likewise emphasise the subtlety of the relationship between what we experience through the senses and how the world appears to behave; much of our own science is established upon scientific observables whose nature cannot be directly experienced. The successful implementation of VR models that can be applied in engineering depends upon developing modelling techniques that are effective both in representing what we directly experience and in expressing what we think happens or expect to happen in interaction. Too great an emphasis upon the role that theories can play in developing such models is potentially dangerous in two respects. It can blind us to the possibility that our explanatory systems are only in a matter of degree less limited and incomplete than those of our ancestors. It can also lead us to presume that the only respectable ways to model using computers entail complete and unambiguous specification of operational meaning prior to implementation. In our approach to modelling, we are not obliged to subscribe to either of these points of view, but can adopt a perspective similar to that of the 6 practising engineer, whose design and modelling activity combines empirical and theoretical ingredients. 3. Computer-based Modelling for VR Empirical and theoretical approaches to modelling were well-established in practice long before the advent of computers and the conception of VR. In understanding the significance of VR as a 20th century phenomenon, it is important to determine whether computers and multi-media are merely enhancing modelling processes that we currently practise, or whether they can have a more fundamental impact on our whole conception of modelling the world. Our thesis is that the technology of the electronic age is in the process of liberating empirical approaches to modelling, which depend on the discovery and invention of physical objects with specific characteristic properties, so that they can assume greater significance in relation to theoretically-based modelling approaches. The effect of this process is to force us to reassess the status of empirical methods, and to seek a common framework within which the complementarity of empirical and theoretical methods can be fully appreciated. Exposing such a framework will restore the natural synergy between empirical and rational perspectives that predated the formalisation of mathematics in the last century, and help to undermine a prejudice towards theoretical models that is reflected in our perception of scientific explanation (cf [3,4,10]). 3.1. The Empirical Modelling Framework In an empirical approach to modelling, the characteristic idea is the correlation of two experiences: that of a situation and that of a model. Empirical methods are ubiquitous in contexts where non-verbal skills are acquired and applied. Simple examples of the kind of activity we have in mind include: • measuring the length of the edge of a brick by cutting a piece of string to size; • making a scale model of an engineering mechanism; • creating a spreadsheet to represent the student performance in examinations; • developing a scientific instrument; • learning a practical skill, such as swimming. In all these contexts, the essential principle is to establish a correspondence between two sets of observables: the real-world observables that represent the situation of interest, and the reference set of observables that is defined by features of the model. Establishing this correspondence is an iterative process, leading to successive refinement of the model and circumscription of the situation, that involves observation and experiment within the model and the situation (cf [5,6]). At all times, both model and situation are perceived in particular states. The way in which states and observables in the model and the situation correspond is itself mediated by observation and experiment. The objective in setting up this correspondence is to achieve consistency between the way in which sets of observables in the situation are indivisibly linked in change and the way in which the coresponding sets of observables in the model are indivisibly linked in change. Empirical models are distinctively different from mathematical models. Empirical modelling conceptually takes place in the context of two states with which the modeller can interact at one and the same time. As in VR, the process is based upon constructing an environment rather than a document. This means that we can use empirical modelling where neither comprehension of state nor circumscription of interaction is presumed. Empirical modelling is well-suited to VR both in respect of realism in simulation and faithfulness in interaction. An appropriate physical model can faithfully reflect both immediate and cumulative experience of the situation via interaction. This power is exploited wherever we use images or artefacts to convey the characteristics of a reality. The physical model need not necessarily be computer-based -even when it is realised electronically, the computer is only significant in so far as it serves as a physical instrument with which the modeller interacts. This is in contrast to the way in which the computer is conventionally regarded in classical computer science as merely a means to implement an abstract algorithm or computation. In effect, it is how the user apprehends the computer as a physical object that matters in empirical modelling, not the invisible mechanisms by which this object is specified. In this context, the user's apprehension of the model is essentially private and non-linguistic, even though the programming of the computer itself is rooted in language and public communication. The linguistic basis of computer models should not disguise the potential for empirical modelling outside the realm of theory and language however. We can sometimes devise a physical object to reliably imitate its context as a 'model' of its 'situation' simply by serendipity, in the same way that a tree can serve as a sundial. It is in this sense that empirical modelling gives us access to worlds before and beyond words. 3.2. The Theoretical Modelling Framework Theoretical models need no introduction. It is characteristic of a theoretical model that it is expressed in terms of a language whose semantics is preconceived. The limitations of symbolic representations in respect of sensory experience are well recognised. We appreciate that it is difficult to express a colour or a texture in words, however specialised and subtle our vocabulary. We can readily establish conventions for what values are abstractly defined in a computational model, but we cannot convey how they will be experienced. Theoretical models are more widely advocated and adopted in representing the behaviour of a system. Computer science abounds in mathematical formalisms for constructing models whose behaviour can be inferred from theory. When we develop such a model (e.g. as a differential equation, a computer program, or a logical specification) the scope for appropriate interaction is circumscribed by the real-world knowledge that informed the theory. In this respect, a theoretical modelling framework supports VR only in a limited sense. The intended scope for exploration of a theoretical model is implicit in the semantics of the theory. Theoretical models are well-suited to particular kinds of VR simulation, where the goal is clearly defined and the theory well-established, but in this context further exploration lies within the frame of reference of the theory. Related criticisms apply to Iogicist approaches to programming for AI [13]. To qualify this critique of theoretical models we must refer to the celebrated pipedream for theoretical approaches to VR: that of The Theory of Everything. To find such a theory requires a framework for observation within which all phenomena can be consistently explained. It might be, for instance, that by modelling physical interactions at the level of fundamental particles we could explain emergent properties perceived at a high-level of abstraction. This kind of reductionism is subject to technical criticism on account of issues of undecidablity and infeasibility alone [9]. A more fundamental concern is whether there is any good reason to think that our explanatory framework is any more truly comprehensive than that expressed in the Chain of Being. More plausible than the existence of a Theory of Everything is the possibility that no closed model can possibly sustain indefinite exploration. Consider for example, the range of experiments that could be carried out with a virtual knife. With unconstrained interaction we could pose all kinds of questions: • what can it cut? • can it be sharpened? • at what temperature will it melt? • can it be used as a screwdriver? • when will it cast a reflection on the ceiling? • would it be funny if used in mock knighting ceremony? • would it frighten someone if brandished? It seems quite improbable that all answers to such questions can be derived as consequences of a single theory. In practice, we make use of knowledge about the world that commands personal conviction, but is not necessarily derived from a general theory. It is helpful to distinguish two aspects of theoretical knowledge that are conflated in orthodox scientific theories. A theory expresses a conviction about what relationships between observables we expect to find when we apply the theory in an appropriate context. Because it operates within a linguistic framework with a commonly agreed semantics a theory is also a means of communication. In the empirical realm, even though experience of a model and a situation can be private and pre-articulate, there is nonetheless a place for conviction. We can be convinced that this particular movement of the arm is associated with this particular characteristic of a tennis shot. And though we may not be able to convey this movement in symbols, we may be able to teach someone else to appreciate this association. Our discussion of empirical and theoretical approaches to modelling motivates an open-ended approach to the development of VR environments for engineering applications. In such an approach, an evolving VR environment is indefinitely refined as it is explored and developed through observation and experiment. The refinement process involves ongoing consultation with the real-world situation that is being modelled, and may lead to the introduction of both empirical and theoretical ingredients. If this is such a good idea, you may well ask why such hybrid modelling methods have not already been widely developed and adopted. The correct answer to this question may be that such methods have not been feasible hitherto. It is difficult to conceive how to extend a theoretical model to take account of new experimental evidence, for instance, as is commonly found in science and engineering when a theory does not quite satisfactorily account for the experimental results. A theory that is not precisely applicable is not easily adapted, and empirical information that 11 is introduced in an exception handling mode is not easily integrated into a theoretical model. In the classical empirical framework, the recalcitrant nature of physical objects typically makes it difficult to modify their characteristics in a directed and incremental fashion. The use of computer-enhanced objects as surrogate physical objects seems set to change this state of affairs. All that is required to make this more clearly evident is a paradigm shift towards computer-based modelling that is oriented towards an empirical rather than a theoretical perspective. This is the motivating idea behind our Empirical Modelling Project. 4. Towards a Metaphysical Account In the introduction, we referred to the metaphysical framework we must ultimately develop to account for our empirical approach. The motivation for this is clear when we consider the ontological chasm that separates reality as perceived by the craftsman interacting in an empirical modelling framework and the reality of phenomena universally agreed to operate in accordance with the predictions of Newtonian mechanics (cf Gooding [10] on the empirical basis of scientific theories). The framework within which we would like to cast our explanation of the world, in so far as it is explicable to us, has to begin with the private experience of identifying and monitoring observables and indivisible linkages between observables, and develop from this point to an account of how we can first perceive other agents (whether inside or outside the model) and then communicate with some of them through physical gesture and symbolic language. Needless to say, it is premature to venture more than a sketchy and speculative discussion of such a progression of concepts at this stage, and neither our practical nor theoretical research is yet at the stage where we can pursue this path through experiment. This section is devoted to a rough preliminary map of the relevant territory. Our general outlook is consonant with the idea that the foundations for computer-based modelling are to be sought in physical realisibility (cf [13]). The mental map suggested by our empirical modelling approach is broadly specified with reference to agents. The context for empirical modelling is one in which insight is gained through experiment in an environment that has yet to be fully explored and circumscribed. For this reason, the context allows us to operate as an agent in a powerful sense - leaving us free to interact not through some preconceived and appropriately restricted interface, but entitled to change the model in essential ways with reference to an external situation. Conventional theory does not accommodate agents in this strong sense -our environment for interaction has been mapped out for us, so that the relationship between our model and the external situation it represents is fixed and belongs to a universally shared framework of knowledge. The most tractable contexts for empirical modelling are those in which the designer is the sole agent (1-agent models). In such contexts, nothing is perceived to change except through the direct intervention of the designer. In practice, most experimental environments are much noisier - there are many agents operating, and it becomes more difficult to distinguish the indivisible effects of personal actions. To account for the behaviour of such systems, we have to develop multi-agent environments and conviction about patterns and mechanisms for communication between them. The most significant feature of an empirical modelling framework is that the model and situation reside within the realm of experience of a single agent, so that is possible to introduce aspects of the model into the situation. It is on this basis that we can hope to give an account of observables whose status can only be inferred indirectly through interaction with scientific instruments, and to postulate a theory for the development of language. Throughout the entire development that we hope in due course to elaborate and illustrate, a common principle operates. In identifying observables, indivisible linkages, agents, communication patterns and symbolisms, we introduce theory wherever we presume it to be justified through repeated confirmation by experiment. The arbitrariness of this justification, and its status as an act of faith about all future experience, means that our classification of situations always has a provisional quality. For instance, who and when are we to say that there are no other agents beyond those of whom we have experience? In this respect, we are more favourably disposed towards the Chain of Being hypothesis than towards a Theory of Everything. It is this philosophical bias towards uncertainty about absolutes that guides our adoption of the term empirical modelling. 4.1. VR from the Private Perspective: the 1-agent Model The essence of empirical modelling is to be found where the experimentor acts as the sole state-changing agent. From a philosophical perspective, the most controversial aspect of our approach is the mechanism by which observables are identified, since this connects with sensitive issues of linking symbol to referent that are treated in an entirely different way in conventional logical accounts (cf [13]). Our contention is that this process of identification of variables is a matter of common experience far more plausible as a primitive intuition than the relatively arbitrary association of symbol to observable that classical logic presumes. The essential difference is that experiment and state-change plays a part in it -it is this that allows us to distinguish between one association of observables and another. I regard the hand of a clock as one observable that can be viewed in many states because I can observe the transition from state to state - as it were - continuously. Of course many paradoxes surround such a naive concept of identity, some of which, such as evaporating clouds, are part of our everyday experience. The virtue of an empirical modelling framework is that the identification of observables is provisional in nature, and is subject to revision in the light of further experience. The explicitly state-based nature of empirical modelling has many advantages. It makes it possible to postulate distinct observables that can have coincident values. It also allows us to ascribe identities to objects in which there are characteristic patterns of observables. A key process in the identification and classification of 14 observables is the recognition of dependencies: reliable associations between the values of observables that are respected in change. The concept of dependency illustrates one of several ways in which empirical models can be synthesised. In seeking to develop our metaphysical framework from first principles, it would be invidious to explain a dependency with reference to some commonly accepted theory, such as arithmetic. A dependency is a way in which the value of one observable can be reliably inferred from the values of others. This appeals to a form of computation that can be appropriately expressed as derived from empirical modelling as a form of reliable interaction in an independent situation. In functional terms, such interaction is the counterpart of use of a calculator, but might be more appropriately described in terms of a physical object with a reliable stimulus-response pattern that could be interpreted as computing an input-output relation. Such a process is not necessarily public in nature - witness the diverse ways in which different people carry out "the same" calculation. In our philosophical framework, all interpretation is fundamentally related to interaction of a single agent within an empirical modelling context. Within this context, we can account for activities such as the acquisition of mental models that are employed when we exercise a skill. The intrinsically private and non-linguistic nature of this primary experience contrasts strongly with any form of public and theoretical knowledge. The process of isolating observables and identifiable actions is an essential ingredient in developing an empirical model that coincidentally provides a platform on which we can introduce symbolic references. Whether such symbolic references can be shared is a matter to be considered in yet another empirical modelling context. 4.2. VR from the Private Perspective: the Multi-agent Model In our interactions with a situation, it is unusual for us to be concerned only with changes of state that are directly under our control. The concept of "observables indivisibly linked in change" is intended to convey the idea that changing the value of a certain observable has a predictable effect upon the values of other observables. In a single agent model, this concept of indivisibility may amount to changing the value of an observable and waiting for the system to reach a state of stable equilibrium (cf the experimental framework for the investigation of Hookes' Law). In effect, indivisibility means that the side-effects of change to one observable are inevitable and predictable rather than literally synchronised in time. An empirical modelling context is perceived as multi-agent when the changes to observables are not all attributable to actions on the part of the experimentor. Further experimentation, beyond the primary activity that identifies observables and indivisible relations, is needed to identify the characteristics of a multi-agent environment. The focus of this experimentation is on identifying constraints on the values of observables, isolating other agents responsible for change and attributing changes to observables to these agents. This reinforces the idea that our perception of how a system behaves has a strong conceptual component: even in everyday experience, our convictions about what agents are extant, which observables are absolutely constant, which observables are subject to change, when and by whom - all this is deeply bound up with our prejudices about what is plausible and what would be miraculous. The empirical modelling framework admits several conventional experimental techniques that can be used in investigating agents. The presence of other agents is typically correlated with the presence of observables, and control experiments can be carried out in the absence of agents. Gaining conviction about the responses of agents may involve the development of "scientific instruments" to extend the range of what is observable. A scientific instrument is a physical object whose response to a situation is perceived to be reliably correlated to its context (cf the sundial). The development and calibration of such a device is itself an independent empirical modelling process in which the instrument under development plays the role of the model. Once developed, such an instrument can be incorporated into the real-world situation as a source of auxiliary observables. The use of such an instrument effectively extends the range of observables. Communication between agents can also be analysed empirically by correlating the actions of agents with actions that affect observables. 4.3. VR from the Public Perspective: the Multi-agent Model The empirical modelling activities described above are private in as much as there need be no presumption that what is experienced by one person is correlated with the experience of another. To move from reality as personal experience that is consistent and carries conviction, to reality as consensus, requires more sophisticated forms of communication than are involved in communication between agents within the model. Two forms of communication to other human agents as if in sharing experience can be envisaged. The parties to a communication can act as experimentors in a common empirical modelling context. Whether communication can take place depends entirely upon the nature of the personal realities that are brought together. If the state-changing actions of one of the experimentors are sufficiently familiar to the other, it may be possible to communicate non-linguistically. A tennis coach, for example, can teach a skill by directly demonstrating the correlation between arm configuration and associated tennis shot. Alternatively, it may be possible to confirm the identification of an observable that is apparently common to the two experimentors and to establish a symbolic convention with which to reference it. In this context, one of the experimentors will attach a symbol to a reliably perceived element of experience, and interaction will proceed in the new situation in which 17 the symbols of one experimerrtor are additional observables for the other. In this connection, it is interesting to note that in articulating a symbol in response to what is being observed an agent is emulating a scientific instrument. We envisage that the cumulative effect of many layers of empirical modelling, conducted in situations that are augmented through the development of more sophisticated means of calculation, embellished through the introduction of scientific instruments, of several experimentors and of symbols leads to the basis on which classical formal foundations are established. We can enter the world of formal language where reliability is presumed, so that we can do without actual experiment. At this stage, experiment is displaced by a dialogue that takes place in an imaginary world: "in this situation, if we were to do the following, we should cbserve thus ". Though we are accustomed to regardthis reality-of-consensus as the authentic reality, and hence to view reality as mediated by language alone, it is not appropriate to disparage the reality of personal experience in this fashion. To some degree, language can take the place of interaction with an environment, to some degree, a document can be animated in the reader's imagination, but no purely symbolic representation can convey the worlds before and beyond words that elude this imperfect process of communication. 5. Summary VR aspires to fool the senses and the mind by achieving realism and allowing interaction in the exploratory sense. We do not believe that this can be done within closed modelling systems or on the basis of preconceived interpretation of symbolic actions alone. VR points to the need for a modelling process that is open-ended and rooted in personal immediate experience. We are developing an empirical modelling method that has already found wide application and appears to have considerable potential for VR. In particular, analysis of this method suggests a metaphysical account of intersubjectivity that is well-suited to VR. 6. Acknowledgements We are indebted to Simon Yung for his fundamental contributions to the Empirical Modelling Project, to Cheryl Sidebotham for her work on the jigsaw assistant, and to IBM Warwick and the EPSRC [Grant GR/J13458] for their financial support. 7. References [1] v.D. Adzhiev, W. M. Beynon, A. J. Cartwright, Y. P. Yung. A Computational Model for Multi-agent Interaction in Concurrent Engineering, Proc. CEEDA'94, Bournemouth Univ., 1994, 227-232 [2] V.D. Adzhiev, W. M. Beynon, A. J. Cartwright, Y. P. Yung. An Agent-oriented Framework for Concurrent Engineering, Proc. lEE Colloquium: Issues of Cooperative working in Concurrent Engineering, October 1994 [3] W.M. Beynon. Agent-oriented Modelling and the Explanation of Behaviour, Invited Paper in Proc. International W/S "Shape Modeling Parallelism, Interactivity and Applications", Dept. of Computer Software, TR 94-1-040, Univ. Aizu, Japan, Sept. 1994, 54-63 [4] W.M. Beynon, S. B. Russ. The Development and Use of Variables in Mathematics and Computer Science, IMA Conf. Series 30, 1991,285-95 [5] W.M. Beynon, S. B. Russ. Empirical Modelling for Requirements, Research Report #277, University of Warwick, September 1994 [6] W.M. Beynon, Y. P. Yung, A. J. Cartwright, P. J. Horgan. Scientific Visualization: Experiments and Observations, Proc. 3rd Eurographics W/S: Visualization in Scientific Computing, Viareggio, 1992, 159-165 [7] W.M. Beynon, I. Bridge, Y. P. Yung. Agent-oriented Modelling for a Vehicle Cruise Controller, Proc 1992 Eng. Systems Design and Analysis Conference, ASME PD-VoI. 47-4, 159-165 [8] W.M. Beynon, S. B. Russ, M. D. Slade, Y. P. Yung, Y. W. Yung. Definitive principles and software specification, Computer Science Research Report 146, University of Warwick 1989 [9] J. Cohen, I. Stewart. The Collapse of Chaos, Viking 1994. [10] D. Gooding. Experiment and the Making of Meaning, Kluwer 1990. [11] M. Helm. The Metaphysics of Virtual Reafity, OUP 1993. [12] P. E. Ness, W. M. Beynon, Y. P. Yung. Applying Agent-oriented Design to a Sail Boat Simulation, Proc. ESDA 1994, Vol.6, 1994, 1-8 [13] B.C. Smith. Two lessons of logic, Comput. Intell .3 (1987) 214-218 [14] E.M.W. Tillyard. The Elizabethan World Picture. Pelican 1972. The Empirical Modelling Project The Empirical Modelling Project is an ongoing research programme, based in the Computer Science Department at the University of Warwick, that was initiated by Dr Beynon in 1981. The theme of the project is broad, and has found application in human-computer interaction, interactive graphics, computer-aided design, scientific visualisation, concurrent systems modelling, concurrent engineering and software development. Over the last 5 years, the project has received sponsorship from EPSRC, The Royal Society, NSERC Canada, British Telecom, IBM, Matra Datavision and the University of Aizu in Japan to pursue several of these areas of application. Major collaborators over this period include Dr Simon Yung, Dr Mike Joy , Dr Steve Russ and several postgraduates in Computer Science, and Dr Alan Cartwright in Engineering. Our output includes contributions to over 30 international publications as well as a suite of special-purpose software tools and a large, varied and steadily growing collection of case-studies. Since its inception, the project has developed both as a foundational study in modelling and computer programming and as a powerful practical method for simulation, visualisation and concurrent systems development. The choice of the epithet "empirical" reflects the fact that our methods are rooted in observation and experiment, and accordingly approach system development from a very different perspective from that associated with formal methods and traditional mathematical models. As when constructing a spreadsheet model, the primary emphasis is upon identifying the dependencies between real-world observables, and on obtaining a representation of state that is sufficiently faithful to be useful in "what if" experiment. Its explicit emphasis upon modelling state as directly experienced rather than behaviours as circumscribed distinguishes our approach from traditional techniques of computer-based modelling. Our modelling process gives useful insight into how we presume to gain reliable comprehensive knowledge about a system from experimental evidence. In due course, we expect to be able to give a much fuller account of the relationship between rational and empirical perspectives within our modelling framework. The major practical contributions to our programme have been made by Dr Y P Yung and his elder brother Dr Y W Yung, whose work on implementation has enabled us to build and interpret a variety of notations for representing dependencies. These include notations to create automatic links, similar to those between cells of a spreadsheet, to connect the disposition of all the elements in a typical screen display, such as windows, textual annotations, lines, labels and to specify their deeper semantics, whether as combinatorial elements or mouse-sensitive regions. The screen displays in Figure 1-3 give some indication of the range of applications that can be addressed by empirical modelling using these tools. Figure 1 illustrates one of many final year undergraduate projects that have exploited our tools and methods. The screen display shows a jigsaw in which the picture dynamically changes (as in the nursery rhyme Hickory Dickory Dock), and the jigsaw pieces change accordingly. The partly obscured windows behind the main display show extracts from the associated file of definitions that make up the script that describes the current state. The definitions are automatically classified and displayed within different windows according to the type of dependency they establish. In principle, the user of such a system can redefine any of the extant definitions in an arbitrary fashion - for instance, it would be a simple matter to arrange for the size of the face of the clock to vary according to the location of the minute hand, or the speed of motion of the hands to depend upon the number of jigsaw pieces correctly assembled. The ease with which different strategies for control can be readily imposed on the model in this way suggests its potential application in educational use, for instance, in assessing the cognitive and physical capabilities of a child. Figures 2 and 3 illustrate other areas in which our modelling principles have been successfully applied: a vehicle cruise control simulation and a demonstration of the combinatorial properties of families of lines. 1 ? /