WARNING: This text has been OCRd from the original paper and so will contain many typographical errors. It should be useful for searching, however. [Ash, August 2002]. Reprint from R.A. Earnshaw B. Wyvill (Eds.) New Advances in Computer Graphics Proceedings of CG International '89 (c) Springer-Verlag Tokyo 1989 Printed in Japan. Not for Sale. Springer-Verlag Tokyo Berlin Heidelberg New York London Paris Evaluating Definitive Principles for Interaction in Graphics M. Beynon ABSTRACT This paper is an appraisal of current progress towards supporting interactive graphics within the framework of a general-purpose programming paradigm based upon definitions. It considers how the use of definitive principles relates to other work, why it appears promising, and what progress has been made towards the resolution of technical difficulties. As a sequel to [2], it re-examines potential for applications of definitive principles in interactive graphics in the light of more recently developed ideas about dealing with control issues and dynamically changing relationships in a definitive programming framework. It also takes account of new research into notations for graphics that makes use of geometrical constructions. As a subsidiary theme, the paper contrasts the support for reference and representation of geometric relationships in various kinds of interactive graphics systems. Key words: interactive graphics, spreadsheets, geometric constructions, constraint-based graphics systems, functional programming, reference, animation 1. INTRODUCTION Most existing interactive graphics systems focus on providing the user with a toolkit that makes it possible to draw complex diagrams; they reflect a view that is oriented towards tools rather than frameworks. The emphasis is upon programming as a "means to an end", viz the depiction of a complex diagram. Though graphical interfaces are fashionable, and user-computer interaction concerns are deemed important, the underlying idiom often resembles batch programming. The user is an unequal partner who makes use of an interactive interface for convenience, but is not thereby enabled to intervene more significantly and directly in the computational process. Modern applications for graphics demand a broader perspective. It has become necessary to think of "interaction in the large": to consider the issues of managing many thousands of drawings, perhaps over many years of development, within the broader semantic context of such applications as engineering design and manufacture. In developing interactive graphics systems that can support large applications, it is surely important to first identify simple and powerful principles for their design. More sophisticated tools per se are no panacea. Experience has shown that developing tools based upon ostensibly less powerful techniques can enhance the range of application: c.f. [7] " ... the use of geometric constructions eliminates the need for solving large systems of non-linear equations inherent in declarative constraint-based systems. Consequently, L.E.G.O. can be used to model comparatively more complex objects." ~ A framework within which existing approaches to interactive graphics can be interpreted and integrated must depend upon generic techniques with wide - if not universal - applicability. Unification of principles in interactive graphics appears hard to attain. The methods used are notoriously diverse; there is a conspicuous lack of formalisation and standardisation, and ad hoc techniques are prevalent. The problem of formulating an abstract view is compounded by the very different functions that graphical images can serve: as in computer-generated art, medical image analysis, symbolic representations in architecture or circuit design, or technical drawing. The many different types of hardware and software support for graphics pose complementary problems. Nor are existing techniques for formal specification necessarily well-suited for interactive applications. This paper aims to appraise current progress towards supporting interactive graphics within the framework of a general-purpose programming paradigm based upon definitions: "definitive programming". Previous papers dealing with related work include [1,2,3]. Though many of the issues addressed were considered in [2], there have been several subsequent developments that motivate a re-examination. As described in [4,5], the concept of definitive programming has itself been enriched through the identification of an abstract machine model that can support far more sophisticated computation than the "pure definitive notations" of [1]. This significantly enhances the scope for representing dynamic data relationships, and dealing with concerns - such as animation - where control structures are required. Of additional interest is the parallel development of interactive graphics systems based upon geometric constructions [7,13]: work that can be directly related to the definitive programming approach - with mutual benefit. For instance, definitive programming provides a much broader perspective within which to consider the use of "imperative constraints", whilst research on construction-based modelling suggests new and better solutions to the technical problems of dealing with complex operators raised in [2]. 2. FUNDAMENTAL ISSUES FOR INTERACTIVE GRAPHICS The focus in this paper is upon graphics systems that involve interaction in a significant sense. A system that supports "visual programming" is not necessarily such a system: the main function of a graphical interface may be topermit convenient editing of a program that routinely generates graphical images in an autonomous way. To invoke the concept of "significant interaction" is itself to beg a question: "Is it possible to make an objective claim that one system supports richer interaction between the user and the computer .than another?" This paper argues that it is - provided that the reader will accept the thesis that a spreadsheet gives better support for interactive calculation than a pocket calculator. To appreciate the context in which "interaction" is being interpreted, the user may think of a major design project involving the abstract description and evaluation of a complex artefact such as a building. A very large numberof drawings may be developed in the design process, and the timescale may be such that drawings have to be revised many months after they are first drawn. Naively, the user will need to be able to specify relationships between objects and components of objects that are to be stored and maintained by the computer. Several key issues arise: How are the relationships to be established? When is the maintenance of relationships carried out? To what extent are relationships maintained through autonomous action on the part of the computer? How is the current status of the design to be represented to the user? How can data relationships be represented in such a way that semantic analysis or simulation can be conveniently performed? The demands that a user would ideally wish to make upon a design system are very great: ~: 293 ~reted and :ability. s used are )n, and ad ~unded by erated art, .r technical ~hics pose ~ecessarily ,'tics within lefinitions: 3]. Though mbsequent f definitive :t machine definitive lamic data uctures are ics systems e definitive )gramming 'imperative and better - relationships should be easily perceptible, and conveniently modified - it should be possible to record partial information about relationships conveniently - relationships should be expressible at many levels of abstraction - it should be possible to accommodate a temporary failure to meet constraints, to compensate at a later stage, and conveniently make consequent changes retrospectively. For the user, a transaction with the system will typically be an incremental change - a single addition to the sequence of perhaps many thousands of previous transactions. The architect who moves a wash-basin a metre nearer to the door will not expect the plans for the entire building to be reprocessed, but might reasonably expect to be advised that the waste pipe is obstructed by a balcony on the floor below. Many existing programming paradigms for interactive graphics are better suited for the complete reappraisal of a design after every trivial amendment than to modest and selective processing proportionate to a minor transaction. To some extent, this is a symptom of a more profound difficulty: how to represent relationships between objects so subtly that a trivial action has some impact, but not too much. Such problems of maintaining relationships that are both consistent and admit a multitude of very minor modifications have promoted the development of systems that avoid the embarrassment of user-intervention wherever possible. Thus a sophisticated design system might invoke a computational scheme of enormous ingenuity that involved reconfiguring all the waste pipes and balconies on an entire wall, at considerable computational cost, and perhaps in violation of the user's aesthetic preferences. 3. CURRENT APPROACHES significant system: the rogram that concept of ~ctive claim Lputer .than thesis that a The above discussion indicates that the effective representation of geometrical relationships is essential for significant interaction in a graphics system. Many existing approaches fail to address this issue adequately. When the emphasis is on supporting a repertoire of drawing operations (as in say MacDraw), the graphical image is described by the cumulative effect of procedural actions, and the geometric relationships that can be established are limited to the independent manipulation of explicitly defined groups of points and lines. Within such a framework it is not possible to support relationships between geometric entities at an appropriate level of abstraction. y think of a )lex artefact ign process, :ter they are objects and Several key atenance of ~utonomous presented to ." analysis or It is clear that an interacti~;e graphics system that gives effective support for abstract relationships must be based upon higher-level primitives. Construction-based modelling recognises this by adopting fundamental geometric constructions as the primitive operations, and describing structural relationships in terms of these. The formulation of data relationships over an appropriate system of data types and operators at a reasonable level of abstraction is common to many approaches: their difference lies in the way that this "underlying algebra" is exploited. Four interrelated approaches to modelling geometric relationships are particularly relevant to this paper: - functional programming principles (c.f. [9] p255), - equational / constraint-based principles (c.f. [6]), - construction-based modelling [7,13], y great: - definitive programming principles [2,3]. Systems to represent planar line-drawings within these paradigms would typically be based upon an underlying algebra consisting of scalars, points, lines, and shapes comprising multisets of points and lines, together with elementary geometric operators e.g. specifying the line joining a pair of points, the angle between two lines, or the union of two shapes. A brief outline of the different superstructures that can be built upon this foundation within these paradigms will be useful. Following [9], a functional programmer might represent an abstract shape by a function point x point --~ shape (i.e. as the set of points and lines determined by a choosing a particular coordinate system as represented by a pair of points), and define complex shapes by applying higher-order functions to basic shapes. During interaction, the current status of the functional programming system will be determined by what functions have been defined, and the image currently depicted will be determined by what function evaluations have been performed. Geometric relationships are specified and modified by editing a script that defines appropriate higher-order functions, and redisplay is effected through function re-evaluation. In a constraint-based system, geometric entites are represented by variables, and geometric relationships are expressed as equational constraints between variables over the underlying algebra e.g. insisting that four points lie on a circle, or that two lines are parallel. Ideally, these constraints are considered to be purely declarative in nature: they prescribe the relationships that must hold without burdening the user with details of how they are maintained. Interaction involves editing the current set of constraints, invoking a constraint-satisfaction process that first attempts to reconfigure the geometric entities appropriately, and subsequently leads to their redisplay. In a construction-based modelling system, the relationships between data are formulated through a sequence of "imperative constraints". In effect, the locations of geometric entities are specified relative to each other through explicit prescriptions that are directly or indirectly expressed in terms of the operators of the underlying algebra. The constraints established in this way are conceived and stored as procedural fragments that encapsulate the description of a graphical image, and can be edited for purposes of reconfiguration or re-display. In a definitive programming idiom [2], the points, lines and shapes that make up the graphical image are represented by a system of variables of the appropriate type. The value of a variable can either be specified explicitly, or implicitly by an algebraic expression in terms of the values of other variables and constants. The system of variable definitions is free of cyclic reference, so that the variables can be partially ordered by a data dependency relation: v P ladder(n,a,b) = if n=l then line(a,b) else picture(line(n.a,b),ladder(n-l,a,b)). In this specification, a and b are parameters representing vectors, n is an integer to specify the number of rungs, and P designates the data type "picture". It is an elegant and concise description, but is many ways inappropriate in an interactive graphics setting. To depict the rungs of a ladder, a function evaluation such as ladder(n,<1,2>,<2,1>) must be performed. To reference a particular rung of the ladder, as might be required in simulating "climbing the ladder", is impossible without embellishment of the original specification. To model a real ladder, that might consist of two congruent hinged sections that could be locked into a V-shaped or linear relation, it would not be enough to use such a functional specification - even in conjunction with a DoNaLD shape variable. Of course, it is not infeasible to elaborate functional methods to partially redeem the situation (c.f. [10]), but there appear to be fundamental limitations associated with the strictures of referential transparency. By contrast, a procedural approach to the specification of the ladder() function makes the problems of attaching references to the individualrungs of the ladder less acute. In effect, it is relatively to easy to formulate a procedural description of a function that returns not only a value of type shape to represent the ladder, but a family of variables of type line to represent the rungs. To illustrate the form that such a specification might take, consider the two following specifications - the one written in the construction-based L.E.G.O. idiom, the other in the definitive DoNaLD style: (define-function ladder(n a b)) (line n.a n.a+(b-a) 1) (write-function) (if (n>l) then (ladder n-1 a b) (end-function) openshape ladder(int n, point a,b) within ladder { line L = [n.a, n.a+(b-a)] if n>l then { shape Ladder = ladder(n-l,a,b) } } The same considerations that applied to the comparison of the "bisection of the line" construction discussed above again apply. The dual semantics that makes it possible to view a L.E.G.O. specification from a procedural and a definitive perspective is helpful here in interpreting the specification on the right as defining an operator together with a system of references to the value it returns. By introducing such operator specifications it is possible to formulate a DoNaLD definition: shape Ladder; Ladder = ladder(7, {1,2},{2,1}) nctional ion of a ed with ction in wolved. for the ~cedural ¢ing [9], nctional ~cify the concise ~pict the ~ired in original Dns that such a rse, it is 0]), but erential kes the ect, it is : only a !present :he two ~e other so that Ladder/L refers to the top rung, Ladder/Ladder to the remaining set of rungs, of which Ladder/Ladder/L is the topmost etc. This is the kind of referencing facility that is required when establishing relationships between complex objects that cannot conveniently be explicitly defined. 6. DEFINITIVE PROGRAMMING IN THE LARGE: CONTROL In this paper, the case for "programming with definitions" has so far been made on technical grounds. The primary argument has been that conventional interactive graphics involves editing functional, procedural or constraint-based specifications that typically correspond only in a very obscure way to the cognitive models the user requires. Particular attention has been focussed on the limited way in which conventional paradigms for interactive graphics support references to objects and their constituent parts. Some of the problems can be attributed to the vague semantics of variables, and the dichotomy between declarative frameworks in which references are too inflexibly established, and procedural environments in which they are too impermanent. The result is that interactive systems are frequently ill-equipped for significant user intervention. In what respects does definitive programming potentially offer better prospects? To an extent, the ideas introduced above meet the need for better support for the abstract description and referencing of graphical images, and allow incremental changes to relationships to be subtly and efficiently represented. The representation of dependencies between data using definitions is a particularly significant concept, since it enables the user to prescribe, and to anticipate, the effects of amending a specification. The significance of using definitions to support interaction goes beyond technical considerations alone, however. A central thesis of current work on definitive programming is that a definitive system is a useful cognitive model not merely for user-computer interaction, but for the actions of an agent in a concurrent system [4,5]. In effect, a single system of definitions is best conceived as articulating the effects of a particular action or intended action. Only by such a generalisation of the concept of "definitive notations for interaction" does it become possible to express the fact that an architect's expectations on moving the wash-basin might or might not encompass an automatic compensating movement of the mirror, and could lead to the invocation of an automatic process to reconfigure the external balconies. As this illustration suggests, though it may not be possible for the user to predict the outcome of an automatic computation initiated during an interaction, it is necessary to give the user both a degree of control over such computation, and a good appreciation of its effects. The perspective taken in this paper is that the description and manipulation of geometrical relationships throughout the entire user-computer interaction should be seen as one consistent (non-terminating) computation comprising many simple transactions. In this interactive process, the user and the computer participate via sequences of homogeneous actions, and the effect of each individual action is transparent to the user. This means, in particular, that the user knows the effective geometric relationships throughout the entire interaction, even to the extent that it is possible for the user to suspend and intervene during an automatic computation. .e line" view a here in ~tem of sible to The appropriate abstract machine model adopted for this purpose is the abstract definitive machine (ADM), as described in [4,3,5]. A full discussion of the ADM is beyond the scope of this paper, but a brief sketch of its use in the animation of a simple concurrent system will be used to illustrate the key ideas. In many respects, this ADM simulation resembles examples developed using the NoPumpG software [12]. Suppose that two blocks bl and b2 are connected by a string of length d, and that the blocks are independently controlled by two handlers. The behaviour of the pair of blocks can be described by the ADM program in Figure 1. The handlers are represented by entities: each entity comprises a set of variable declarations and definitions, and a set of guarded actions. In this context, each action consists of a sequence of redefinitions. The operation of the ADM is such that in each machine cycle the guards of all actions within currently instantiated entities are evaluated, and the actions associated with true guards are performed in parallel (subject to non-interference). The entity handler10 for instance, may at any stage be holding bl - as defined by the variable hl, and when holding the block may be pushing left (pll) or right (prl). The movement of the blocks is determined by the bmover0 entity, and depends upon the directions in which the blocks are currently being driven (dr1, dll, dr2, d12). The ADM control program in Figure 1 is taken from [5], where a fuller discussion may be found. For brevity and convenience, some simplifications have been made: the use of parametrised entities has been avoided, and only the specification of one of the block handlers (handler10) is given. The use of the colon (:) to separate actions indicates that precisely one of these actions is to be performed when the associated guard is true. The specification of bmover0 illustrates how the context within which variables are redefined must be altered to reflect conditions such as whether the blocks are touching (t12), and whether the string is taut (st). For instance, if bl is being driven to the left, and the string is taut, the action "dll and st -> p2=pl+d; pl= I pl I-1" is enabled, indicating that the movement of bl to the left must necessarily drag b2 to the left also. As it stands, the specification does not include any method for resolving interference between actions, as for instance would arise if b2 were simultaneously being driven to the right (c.f. [5]). The entity bstateO records the current position and status of the blocks: their positions (pl, p2) and whether the string either should snap or has been snapped (nostr). The simulation is very simply animated by complementing the ADM specification with a file of DoNaLD definitions to describe the required display. The DoNaLD definitions of the openshapes bl and b2 that represent the blocks~ and the line str that represents the string can be abstractly viewed as an additional entity within the ADM specification. The block display registers the location of the blocks (as specified by the variables pl and p2), and the handler-block relationship (as specified by hl, dll, dr1 etc). The simulation snapshot depicts a situation in which bl is being driven to the right, and the block b2 is being held, for instance. CONCLUDING REMARKS This paper has argued the case for giving serious consideration to the development of definitive programming as a medium for describing interactive systems, and perhaps indicated something of its potential. There are many issues still to be explored, and work is in progress on a variety of related topics, including applications to CAD, and to the simulation of concurrent systems. Good semantic models for graphics and interaction deserve closer consideration. There are many indications in this paper of the apparent inadequacy of our present theoretical foundations for programming in respect of data representation and manipulation (c.f. [11]). An intriguing historical sidelight on this issue is the demise of the concept of a mathematical variable as "representing a variable quantity" during the arithmetisation of analysis in the 19th century [8]. Perhaps the development of better methods of programming for interactive graphics can d~imulate reconsideration of the formal status of the rich - if obscure - concept of variable that geometrical intuition formerly inspired. pl, p2) 301 entity • In this is such [ties are bject to bl - as it (prl). )on the ~ay be use of andlers one of tefined ~-), and tring is the left ference to the i a file of the ng can lisplay mdler-uation entity handler10 { definition dl = dll or dr1, dll = hl and pll, dr1 = hl and prl, pll = 0, prl = 0, hl =0 action not hl -> h1=1, hl and not dl -> hl=0: pll =1: prl=l, dll -> pll = 0, dr1 -> prl = 0 entity bstate() { definition pl, p2, d, st = not nostr and (p2-pl)==d, t12 = (p2-p1)==1, nostr = 0 action not nostr and (p2-pl)>d -> nostr=l entity bmover0 { action dll dll dr2 dr2 dr1 dr1 d12 d12 and not st -> p1=lp11-1, and st -> p2=pl+d; pl=lpll-1, and not st -> p2=lp21+1, and st -> pl=p2-d; P2=lp21+1, and not t12 -> pl=lpll+l, and t12 -> p2=p1+1; pl=lp11+1, and not t12 -> p2=lp2l-1, and t12 -> p1=p2-1; p2=lp21-1, bstate0; bmover0, handler10; handler20 openshape bl within bl { point O O = {500+~/p1"100, 500} point NE,NW, SW,SE NE = O + {50,50} NW = O + {50,-50} line n,s,e,w n = [NW,NE] s = [SW,SE] point N, E, S, W, X N = if ~/hl then (NE+NW) div 2 else O S = if ~/hl then (SE+SW) div 2 else O E = if ~/hl then (NE+SE) div 2 else O W = if ~/hl then (NW+SW) div 2 else O X = if ~/dr1 then E else if ~/dll then W else O line WE, NX, SX = [W,E], [N,X], IS,X] } int pl,h 1,drl,dll,p2,h2,dr2,di2,nostr line str str= [bl/E, if nostr then bl/E else b2/W] N NE X S ent of .~rhaps k is in :ion of Above: The ADM control program Bottom right: Simulation snapshot Middle right: Detail of the block1 display Top right: DoNaLD display specification re are :etical ]). An latical in the "active :ept of Figure 1: A block moving simulation using definitive principles ACKNOWLEDGEMENTS I am indebted to Mike Slade and to Edward Yung for their support in designing and implementing software prototypes for DoNaLD and the ADM, and to Steve Russ for many helpful discussions on the subject of reference and interaction. REFERENCES 1. W M Beynon Definitive notations for interaction, Proc hci'85 CUP 1985, 23-34 2. W M Beynon Definitive principles for interactive graphics, NATO ASI series F:40, 1987, 1083-1097 3. W M Beynon, D Angler, T Bissell, S Hunt DoNaLD: a line drawing system based on definitive principles, Univ of Warwick RR#86, 1986 3. W M Beynon, Y W Yung, Implementing a definitive notation for interactive graphics, New Trends in Computer Graphics, ed N Magnenat-Thalman, D Thalman Springer-Verlag 1988, 456-68 4. W M Beynon, M D Slade, Y W Yung Parallel computation in definitive models, in Proc Conpar'88 (to appear) 5. W M Beynon Definitive programming for parallelism, CS RR#132, Warwick Univ, 1988 6. ABorning The programming language aspects of ThingLab, a constraint-oriented simulation laboratory, ACM Transactions on Programming Languages 3(4), 1981, 353-387 7. N Fuller, P Prusinkiewicz Geometric Modelling with Euclidean Constructions, New Trends in Computer Graphics ed N Magnenat-Thalmann, D Thalmann, Springer-Verlag 1988, 379-391 8. P Geach, M Black Philosophical Writings of Gottlob Frege. 9. P Henderson Functional Programming, Prentice-Hall International 1980 10. J Hughes Why Functional Programming Matters, PMG Report #16, Chalmers Univ of Tech & Univ of Goteborg, 1984 11. W Kent Data and Reality, North-Holland 1978 12. C Lewis Using the NoPumpG primitive, Dept of Computer Science and Inst of Cog Sci, Univ of Boulder 13. T Noma, T L Kunii, N Kin, Henomoto, E Aso, T Yamamoto Drawing Input Through Geometrical Constructions: Specification and Applications, New Trends in Computer Graphics, ed N Magnenat-Thalman, D Thalman, Springer-Verlag 1988, 403-415 14. T W Reps Generating Language-Based Environments MIT Press 1984 t7, ~f Tech ng and r many , New 1988, oc ~88 387 ['rends 3, 379- Meurig Beynon is currently a lecturer in computer science at the University of Warwick. His research interests have included geometric aspects of partially-ordered algebras, mathematical algorithms, and the algebraic theory of boolean circuits. His most recent research has been concerned with the development of a programming paradigm based upon definitions ("definitive programming") that has been applied to the design of prototype software for interactive graphics and CAD applications, and for modelling and simulating concurrent systems. The current focus of this research is on exploiting definitive principles as a method for software specification, and as the basis of a general-purpose model for parallel programming. Beynon is the author of 22 refereed papers on mathematics and computer science. He was an SRC Postdoctoral Research Fellow at University College Swansea from 1973-5, and a visiting Research Fellow at British Telecom Research Laboratories in 1986. He was the principal organiser of the 2nd British Colloquium for Theoretical Computer Science in 1986, and the secretary of the colloquium committee from 1986-8. Beynon received his BSc and PhD in mathematics from King's College, London in 1969 and 1973. Department of Computer Science, University of Warwick, Coventry CV4 7AL, UK. ,ci, r