Skip to main content Skip to navigation

Chapter 9 of 'Music and Consciousness' (eds. David and Eric Clarke) OUP 2011

From Formalism to Experience: A Jamesian Perspective on Music, Computing, and Consciousness

Meurig Beynon

This is a version of Chapter 9 in Music and Consciousness: Philosophical, Psychological, and Cultural Perspectives (ed. David and Eric Clarke), OUP, July 28th 2011, ISBN: 9780199553792, 157-178.
Permission to make digital or hard copies of part or all of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page.

Introduction: repudiating dualism

In his essay ‘Does consciousness exist?’ William James identifies a commonly held (mis)conception: that consciousness is ‘one element, moment, factor—call it what you like—of an experience of essentially dualistic inner constitution, from which, if you abstract the content, the consciousness will remain revealed to its own eye’ (James 1912a: 8). In response to this notion, James writes:

my contention is exactly the reverse of this. Experience, I believe, has no such inner duplicity; and the separation of it into consciousness and content comes, not by way of subtraction, but way of addition— the addition, to a given concrete piece of it, of other sets of experiences, in connection with which severally its use or function may be of two different kinds. (9)

The key sentiment in this quotation is expressed in the phrase italicized by James himself. It represents a subtle form of repudiation of a perceived duality between form and content that is also often invoked in relation both to music and computing. It is a useful point of departure for thinking about consciousness in music.

Dualism in music is perhaps most potently represented in the saying colloquially attributed to a certain school of piano teachers: ‘Now that we’ve learnt the notes, it’s time to put in some expression.’ There is a presumption here that there is such a thing as merely ‘playing the notes’, and that it would pose no problem to a pianist of the stature of Emil Gilels or Sviatoslav Richter to be asked to play a piece of music without any expression. In this connection, James’s point would be that there is but one experience that can be viewed as ‘playing the notes’ and as ‘making music’, and, setting aside the issue of how successful the novice pianist may be at doing either, one cannot help but simultaneously attempt to do both. The implications of James’s repudiation of such dualism go deeper. They relate to a blurring of distinctions and a problematization of concepts that is endorsed by much contemporary musicology. As illustrated by Giles Hooper’s recent account of the discourse of musicology, it is deemed facile to speak without qualification about intra- and extra-musical factors, of high-art and popular music, of classical and romantic compositions, of authentic editions and performances (Hooper 2006: Chapter 1).

In the following account I set out to show how James’s thinking, in association with an alternative foundational approach to computing, provides the basis for a treatment of the theme of music and consciousness that can embrace many varieties of musical experience and interpretation without compromising integrity. As an amateur musician with a casual interest in musicology, I am not best placed to justify this claim directly. Instead I shall venture a more oblique justification, drawing upon my professional background in computer science to highlight parallels between composing or performing music and Empirical Modelling—a specific way of using computing technology to build artefacts that has been developed under my direction over the last 20 years. The musical illustrations and references I discuss are drawn from the German classical and romantic traditions, which are most salient in my own experience as a pianist and accompanist with a particular interest in chamber music and songs. But while this reflects my area of relative musical competence, it should not necessarily be taken as indicating that the ideas developed apply only to the narrow musical culture of Western score-based tonal music on which I shall focus.

Music, computing, and consciousness: moving beyond formalism

The advent of computing has been closely associated with the rise of consciousness studies. Much of the thinking in the latter field has been inspired by computational theories of mind that originated in the study of artificial intelligence. Though research on music has not had the same prominence as topics such as natural language in this context, it has been the focus for studies both linguistic (e.g. Longuet-Higgins 1978, Steedman 1984, Winograd 1968) and connectionist (e.g. Desain and Honing 1992, Gjerdingen 1990, Todd and Loy 1991). There are obvious reasons why these approaches appeal. Music has already developed notations and conventions for interpretation that closely resemble the formal languages of computing in certain respects. Musical scores and computer programs are both ways of representing rich state-changing activities in an abstract manner, and both have an associated realization by way of an external behaviour.

When viewing music from a formal perspective musical structures inferred from the score have primary significance. Composition and performance are understood with reference to established rules and conventions. Musical analysis in this spirit is well represented by work such as William Caplin’s Classical Form (1998). Here the underlying mathematical model of harmonic relationships, as represented in the well-known cycle of keys, plays a crucial role in framing the rules of classical form. By treating music as a rational activity based on formal representations and transformations it is possible to give an effective account of it in terms of familiar abstractions of computer science. For instance, Geraint Wiggins and his collaborators have developed an application program interface that can be used to give computer support for musical analysis and reasoning about music (Wiggins et al. 1993). This relies on introducing abstract data types to express the mathematical properties of the elements of Western tonal music.

A more ambitious objective inspired by a formal view of music, closely related to the theme of musical consciousness, is the application of artificial intelligence techniques to generate music. David Cope’s Virtual Music (2001) is one of the best known examples of research of this kind. Cope’s approach, challenging his readers to distinguish human compositions from computer-generated pieces solely on the basis of studying the music itself (‘The Game’), has stimulated much controversy—see for instance the responses by Hofstadter (2001, n.d.) and the review of Cope 2005 by Wiggins (2008). As Margaret Boden has observed, the idea that mechanical methods of constructing music can produce results of aesthetic value is perceived by some as threatening deeply-rooted ideas about the nature of human creativity (Boden 2006b). Boden views computationally inspired approaches to generating music as challenging ‘romantic notions’ about creativity in composition. The abstract concept of music that this viewpoint endorses is well-suited to the way in which computer-generation of music is effected. What guides computer-generated composition is an abstract specification based on metrics that are applied to musical constructions, and a process for organizing notes into patterns that is framed as an algorithm. In this respect, it is well-matched to the classical view of computer programming, where all negotiation of meaning is preparatory to the process of construction.

The kinds of ‘scientific’ explanation of musical experience to which some researchers in artificial intelligence aspire are not representative of humanistic conceptions of music. In Music and the Mind the psychologist Anthony Storr recognizes the need to take ‘the expressive aspects of music’ into account, and observes: ‘the language used by both philosophers and scientists is neutral and objective. It eschews the personal, the particular, the emotional, the subjective ... Whilst it is perfectly possible to study music from a purely objective, intellectual point of view, this approach alone is insufficient’ (Storr 1993: 38). Storr himself gives far more attention to cultural, physiological, and psychological aspects of musical experience than to formal aspects.

Radical empiricism commends an alternative perspective that is neither formalist and positivist nor as sceptical of theoretical metanarratives as is postmodern musicology. It is based on a reorientation in which concrete and immediate experience of artefacts is fundamental rather than peripheral, and on interpretation that, while never absolute, is amenable to pragmatic authentication. It addresses the need identified by Storr for a philosophical outlook that does not eschew ‘the personal, the particular, the emotional, the subjective’. And while encompassing these characteristics it also proposes pragmatic criteria for making judgements about the objectivity of interpretation that may help to address Hooper’s concerns regarding recent hermeneutic tendencies in musicology (Hooper 2006: 39).

Musicology and computing science: comparisons and analogies

The philosophical shift in perspective to radical empiricism in thinking about music and consciousness is effected by considering a different analogy that relates music and computing: that between a musical artefact as concretely experienced (for instance, in performance or listening), and an artefact based on computer technology viewed from a phenomenological rather than an abstract computational perspective. The principles and practical tools of Empirical Modelling (discussed at greater length below) have been developed with a view to just such a reorientation in thinking about computing.

A computer program, as traditionally conceptualized using Turing’s notion of computation, is much more prescriptive of a specific behaviour than a musical score. Putting aside the fact that one computer may outperform another in terms of speed or data storage capacity, and that there are physical limits on the capabilities of any computing resource, there is an abstract notion of specification of a program which can be correctly implemented so as to perform essentially the same function on different computer architectures and with different interfaces. By comparison, the performances of the same musical work by different artists and on different instruments are extremely diverse in character.

Experiential considerations that are peripheral to whether a program meets its abstract specification are typically of the essence in determining whether a piece of music is effective. Music is not intended to be used but to be experienced. Its full appreciation has always relied on broader intelligence about the context in which works were conceived, composed, and performed. Developments in technology that support musical composition through the direct crafting of concrete sound experience, using electronic representations rather than musical scores, detract from the idea that musical artefacts resemble abstract programs expressed in a formal language. The diversification of musical cultures and the advent of new modes of music-making based on unconventional paradigms have had a similar impact.

As Hooper (2006) observes, musicology has negotiated a shift of emphasis from formal to experiential and contextual dimensions. Computer science, as a much younger discipline with strong historical roots in abstract mathematics, has yet to make such a shift. Modern applications of computing meanwhile expose the limitations of the narrow ‘classical’ conception of a computer program. As new digital technologies for interaction and communication develop, greater emphasis is given to the experiences that the computer can generate. Visual and auditory effects become more significant, and, in mediating these, real-time response and aesthetics are relevant. A more profound impact of these developments in computing culture has been to subvert the naïve idea of computer user, and the notion of systems meeting narrowly specified functional needs. It is inconceivable that computer science could embrace a postmodern framework of interpretation, but the limitations imposed by its current narrow interpretative framework are quite apparent. In establishing an appropriate science for computing in its full generality, the need for a perspective that embraces formal and experiential aspects without enfranchizing wholesale deconstruction is endorsed both by computer scientists and researchers in many related disciplines (see for instance Cantwell-Smith 2002, Jackson 2006, Latour 2003, McCarty 2005, Winograd and Flores 1986).

In the early 1970s access to computers was through a corporate culture, and applications were predominantly of a computational data processing variety. The need to make the most efficient and effective use of what were at that time limited and expensive resources focused attention on issues such as the design of algorithms, the structure of programs, and the abstract processes of software construction. The advent of the personal computer, and the technological developments that brought computing into prominence in everyday life, transformed the practice of computing and challenged its epistemological framework. It has become difficult to understand the role that the classical theory of computation is playing—and potentially can play—in a computing culture dominated by rich digital artefacts, where issues of embodiment and human experience come much higher on the agenda.

To make a speculative analogy, the corporate computing culture of the 1970s might be seen to have had a counterpart in the courts of the eighteenth-century musical patrons. The context in which music was commissioned and performed influenced its character, tending to promote its formal and decorative aspects. The turn of the nineteenth century saw the rise of the artist as an autonomous subject; this was a period during which instrumentation developed significantly in ways that afforded new potential for orchestral sound. It also saw the development of the pianoforte as a bourgeois domestic instrument that in some respects had a role analogous to that of the personal computer, giving individuals access to effects that had previously required large organized groups of instrumentalists. During this period, it became possible for ambitious musical compositions to be conceived that reflected the greater autonomy of the composer.

Beethoven’s innovative treatment of musical forms and meanings serves as a useful illustration. This included elaborations of established forms (as discussed in Caplin 1998); deviation from the traditional organization of the harmonic palette, enabling the organization of tonal canvases on an unprecedentedly large scale (as discussed in Tovey 1944); and the assimilation of narrative elements (as discussed by Seaton (2005) in his commentary on the ‘Tempest’ Sonata). Significant as these innovations were, it is in emancipating new potential modes of musical meaning that they perhaps had the greatest novelty and impact. Beethoven’s own testimony concerning the personal meaning invested in his music—corroborated by his sketch and conversation books, and embellished as his reputation grew—itself helped to stimulate this. The close association of music with the ‘great’ composers and their personal lives licensed attitudes that for many years inhibited critical analysis of cultural influences on the construction of musical meaning (see Cooke 1963, Garnett 1998, Hatton 1994). Various recent approaches in musicology have derived some of their force from a reaction to such neglect.

It is the parallel between musicology and computer science, rather than that between musical and computing artefacts, that is crucial here. Musical compositions and modern-day computing products are necessarily hybrid, having both formal and experiential characteristics. It is no more appropriate to classify one product of computer technology as strictly a ‘program’ and another as strictly an ‘artefact’ than it is to classify one musical composition as ‘classical’ and another as ‘romantic’. But our conceptions of computing and musical artefacts cannot be divorced from the ways in which they come to be constructed and analysed—both must evolve together. In computer science there is topical need for a stance that can justify a more pluralistic interpretation of computing artefacts. What is more, this pluralism has to be consistent with the need in conventional applications for meanings that are sufficiently objective to endorse traditional computational activity. In musicology, there is a topical need for a more principled and discriminating stance on the interpretation of musical artefacts. What is more, this must take account of the objections that led to earlier musicological outlooks being placed under critical scrutiny.

Conjunction and experience: perspectives from James

Endorsing plurality in interpretation while giving integrity to a pluralistic world is a central concern in James’s philosophical outlook (cf. James 1909). I shall develop this theme by first reviewing how James’s thinking relates to the shift in perspective on music that can be associated with the classical and romantic musical traditions; I shall then briefly consider how computer science can benefit from a similar shift in perspective.

The philosophical stance behind James’s vision of consciousness is that of radical empiricism. In an empiricist outlook it is commonplace to think of entities being separated in our direct experience. In listening to a song we hear the sound of the piano and the sound of the voice, and distinguish these. James’s contention, as set out in his preface to The Meaning of Truth, is that relations connecting the entities we perceive as separate are also given in experience: ‘The relations between things, conjunctive as well as disjunctive, are just as much matters of experience, neither more nor less so, than the things themselves’ (1911b: xii–xiii). The connection that the pianist makes between a note on the stave and a note on the keyboard can be such a relation. Such conjunctions can also relate to less mundane aspects of experience, as in a listener’s emotional response to a chord.

James’s perspective is well-attuned to the sensitivities of contemporary musicologists in key respects. This can be seen in relation to what Jenefer Robinson identifies as ‘the most fundamental question’ in the introduction to her Music and Meaning: ‘Can music, without the help of words (as in song, opera, or program music), signify aspects of human life and experience “beyond” the music? And if so, how?’ (Robson 1997: 4). In the spirit of the quotation from James with which I began this chapter, we can assert that the separation of experience into ‘music’ and ‘beyond the music’ is a specious duality. Nor does the idea of a relation directly given in experience decisively endorse or contradict the kinds of rational explanations for musical responses that are proposed by Boden (2006b), are postulated by Cope (2005), or underlie Cooke’s attempt to decipher ‘the language of music’ (Cooke 1963). But, in keeping with James’s view that ‘the “truth” of our mental operations must always be an intra-experiential affair’ (James 1912b: 202), the claims for each of these possible construals of musical meaning are subject to pragmatic validation, and rest ultimately upon matters of personal experience.

James’s philosophical stance gives a positive answer to the first of the two questions posed by Robinson (‘Can music ... signify aspects of human life and experience “beyond” the music?’), but not in such a way as to decisively refute the sceptical scientist or formalist. Reliable and immediate associations between abstract music and extra-musical signifiers are commonplace in many musicians’ experience, but to what extent these associations can be regarded as objective is much harder to establish. Some associations will be entirely personal, perhaps relating to life events of a totally individual nature. Others will be connected with cultural factors, perhaps derived from a common tradition of musical education. Possibly some associations have deep cognitive or physiological roots. What distinguishes James’s epistemological stance in such matters is his contention that ‘subjectivity and objectivity are affairs not of what an experience is aboriginally made of, but of its classification’ (James 1912c: 141).

James also offers an explicit answer to Robinson’s subsidiary question (‘If so, how?’). In his essay ‘How two minds can know one thing’ he draws attention to the impossibility of explaining the origin and nature of relations given in experience: Experiences come on an enormous scale, and if we take them all together, they come in a chaos of incommensurable relations that we can not straighten out. We have to abstract different groups of them, and handle these separately if we are to talk of them at all. But how the experiences ever get themselves made, or why their characters and relations are just such as appear, we can not begin to understand. (James 1912d: 133). It might be plausible to construe ‘we can not begin to understand’ in the light of the much more rudimentary knowledge of neuroscience a century ago. However, given the pragmatic nature of James’s outlook, and the emphasis he has added to the text, this is surely not his intended meaning. Conjunctive relations in our experience can indeed be established in the most accidental and unpredictable ways. Their empirical and potentially transient nature is entirely in keeping with James’s contention that ‘radical empiricism … refuses to substitute static concepts of the understanding for transitions in our moving life’ (1912e: 238–9).

Neither is the substance of James’s observations about consciousness undermined by advances in neuroscience. Gerald Edelman, for instance, cites the need for models of consciousness that reflect what he identifies as Jamesian properties: ‘Consciousness is a form of awareness, is continuous but continually changing, is private, has intentionality, and does not exhaust the properties of its objects’ (Edelman 2005: 164). In Edelman’s view, ‘computer or machine models of the brain and mind do not work’ (114). In their place, he proposes an alternative organizing principle—based on the (controversial) notion of a ‘phenomenal transform’—in which a conscious process depends upon its underlying neural activity in a fashion that echoes James’s notion of conjunctive relation (76–8).

James’s view of consciousness is predicated on our capacity to make separations within our immediate experience and yet apprehend these separate components as one. An apposite view of consciousness corroborated by experimental research in neuroscience is again offered by Edelman: ‘One extraordinary phenomenal feature of conscious experience is that normally it is all of one piece … Any experienced conscious moment simultaneously includes sensory input, consequences of motor activity, imagery, emotions, fleeting memories, bodily sensations, and a peripheral fringe’ (61). This bringing of unity to diversity relies on a complementary phenomenon: the fact that ‘consciousness can be modulated by focal attention’ (61).

The tension between attending to an experience as a whole or focusing on specific ingredients within it is familiar to every experienced musician. The objective of rehearsal may be to integrate many aspects of a musical experience, but the process may involve the independent exploration of each aspect in detail. Realizing the form of music involves subordinating the details and specific contextual ingredients of musical experience; while appreciating music as an experience entails being responsive to all the relations that are conjured by each momentary sound.

It is the reference here to ‘momentary sound’, rather than to formalistic musical elements (such as notes or phrases), and to all the relationships to which this can allude through conjunction in direct experience, that opens the Pandora’s Box of contemporary musicological concerns. To what extent does this sound encompass the tuning of the instrument, the extraneous noises made by the performer, or the coughing of the audience? To what extent does it embrace the contextual factors that are contingent upon the situation, such as the acoustics of the hall, the topical life events that might impinge upon the imagination of the performer uniquely on this occasion, or the actual thunderstorm that coincidentally accompanied the thunderstorm in a performance of Beethoven’s Pastoral Symphony I once attended?

The relationship between a formal intellectual stance on experience and an open pluralistic interpretation of experience is the primary concern in James’s A Pluralistic Universe. With these two perspectives in mind James writes:

The only way in which to apprehend reality’s thickness is either to experience it directly by being a part of reality one’s self, or to evoke it in imagination by sympathetically divining some one else’s inner life. But what we thus immediately experience or concretely divine is very limited in duration, whereas abstractly we are able to conceive eternities. (James 1909: 250–1)

James promotes a view of experience that goes beyond what, in relation to music, could be conceptualized in conventional musical terms. In particular, he alludes to a distinction between time as felt and time as conceptualized (developed at length in his discussion of Zeno’s paradox of Achilles and the Tortoise) that would subvert the idea of individual notes as atomic elements of musical experience. In keeping with the Jamesian notion that knowing— rooted as it is in directly experienced conjunctions—is of its essence personal, I shall illustrate the above discussion with reference to my own musical experience.

A personal case study

In the course of preparing this chapter I have in parallel carried out some informal experiments in musical performance to better understand my own musical consciousness. My main access to music is through the score. Within the musical tradition with which I am most familiar I can read and play from the score much of the piano literature to a passable standard. With the score in front of me I am aware of the structure of a composition, can parse complex phrases and chords without difficulty, and appreciate harmonic progressions and tonality. Without the score, despite being able to find great enjoyment as a listener, I have by contrast scarcely any conscious conceptual grasp of this rich musical content. Any kind of reconstruction of the score, or realization of a performance, from the sound alone—whether in playing from memory, or playing by ear—is extremely problematic. With a view to better understanding this limitation, I have been trying to memorize music with which I am very familiar.

My observations have highlighted for me the complexity of the interaction between many aspects of the experience of playing the piano that come together fluently in my experience when I play from the score, but are otherwise for the most part obstinately dislocated. They give practical insight into what it means to say that relationships can be given in personal experience. I recognize that my ability to sight-read music is informed by two component skills that are not specifically concerned with music as an aural experience at all: a facility for identifying visual patterns that has been acquired through long experience of reading conventional chords and harmonic progressions; and dexterity at reconfiguring and moving my hands that is predominantly muscular in nature and does not in general require me to look at the keyboard.

By comparison, when trying to play from memory I am deprived of the visual experience that is quite clearly my primary gateway to musical knowledge and competence. There are figurations and coordination patterns that I can only physically play with the score in front of me. I have difficulty in reliably discerning any coherent disjunctions in my experience. For instance, unused to looking at my hands while playing, I cannot directly apprehend a connection between playing specific notes and hearing the associated sound as confidently and immediately as I connect patterns in the score with configurations of the hands.

These experiments illustrate key characteristics of James’s notion of conjunctions-given-in-experience: the significance of connections that are immediately registered rather than laboriously figured out; the relevance of practice and rehearsal; the essential personal character. There is no way in which you can corroborate my account by studying it as a propositional statement. Yet, despite its subjectivity, its authenticity could to some extent be confirmed by a suitably qualified observer inviting me to demonstrate the competences I claim. As for the authenticity of my claims to incompetence, they pose a greater problem, though I know only too well that the absence of certain kinds of directly perceived relationship in my musical experience is genuine.

It is in relation to broader conjunctions, such as those that impute subjective and/or emotional content to music, that the ‘romantic turn’ in the conception of music discussed above becomes topical. Where composers explicitly testify that their compositions have been conceived with associations drawn from ‘external’ experience, similar considerations to those raised above pertain. We have no direct or decisive means to authenticate their claims. But, insofar as it makes sense to speak of corroboration, it is to interpreters and performers who realize the musical experience, rather than musical analysts, to whom we must turn. It is only the performer who can venture to make sense of music as an object whose properties cannot be exhausted—in keeping with Mahler’s cryptic dictum “Das Wichtigste steht nicht in den Noten” (‘The most important part [of music] is not in the notes’) (Nikkels, 1999: 327); or with Schumann’s paradoxical injunction to play the coda to the final movement of his Piano Sonata Op. 22 prestissimo (‘as fast as possible’) with subsequent indications to play immer schneller und schneller (‘ever faster and faster’).

In assessing claims to musical meaning beyond formal explication, we may posit the performer as an experimenter. In this context the intentional fallacy is a necessary, if entirely speculative, part of the hypothesis, since the music is assumed to have invoked a conjunction in experience in the consciousness of its composer. Consistent with the romantic stance of James articulated in the preceding quotation, the performer’s attention is attuned to ‘sympathetically divining some one else’s inner life’; is focused on what is ‘immediately experienced’; and is ‘very limited in duration’. But whereas the formal analysis of music begins with the score as a given, the consciousness of each moment that is demanded of the performer may be fragile and hard-earned.

By way of illustration, consider the pianist’s task in performing the opening bars of Schumann’s ‘In der Nacht’ from his Fantasiestücke, Op. 12 (Example 1). To be able first to juggle the surging arpeggio figurations in groups of eight semi-quavers between two hands in such a way as to emphasize the sighing quaver–crochet beats, then to transfer the arpeggio figuration entirely to the left hand and superimpose a triplet figure in the right demands practised skill of the performer, qualities of the instrument, a suitable context for listening, and clarity of mind. Beyond that, the effect will not—in compliance with Schumann’s annotations—express fervour (mit Leidenshaft) or evoke being ‘in the night’ without the most careful attention to an elusive balance of touch, dynamic and rhythm.

In der nacht opening bars

By far the most challenging aspect of interpreting ‘In der Nacht’ is dealing with the associations that are least directly linked to the formal structure. The idea that the notes are intrinsically and objectively associated with being ‘in the night’ is but a romantic notion. The piano teacher will nevertheless incite the imagination of a pupil by using imagery in ways that unquestionably contribute not merely to mastering a pattern of sound but to achieving a phenomenal effect. Though it may of itself have little to do with the imagery of night, prior experience of watching waves breaking on the shore and hearing the undertow of other waves receding may help in conjuring the state of consciousness to which the pianist aspires in these opening bars.

As the above discussion illustrates, a significant factor in the effective exposure of conjunctions is understanding of an ‘extra-musical’ nature. This is especially relevant where music has an explicit programme or text. In Schumann’s song ‘Mondnacht’ from his Liederkreis Op. 39) the sentiment that the music ostensibly accompanies is explicit in von Eichendorff’s eponymous poem. Each of the three verses is a different take on what it is to experience a moonlit night. In verse 1, whose opening is quoted in Example 2(a), it is ‘as if heaven had laid on earth a kiss’ (als hätt der Himmel die Erde still geküsst); in verse 2 it is that ‘the breeze sighs over the meadows and the waving corn’ (Die Luft ging durch die Felder, die Ähren wogten sacht); and in verse 3 it is as if ‘my soul spreads out its wings … and flies home’ (meine Seele spannte, weit ihre Flügel aus … als flöge sie nach Haus). The pulse of the song is maintained through some four hundred stately semiquavers. Embedded within the texture of the accompaniment, in tension with the melody but in such a way as to accentuate the sense of momentary transitions in experience, a single note is at one point repeated more than 40 times. This invites the performers to enlist or refresh conjunctions in experience repeatedly, setting up a field of possible interpretations to which the words serve as a guide.

Mondnacht extracts

Rehearsing the accompaniment to ‘Mondnacht’ is an exercise in managing consciousness while realizing an experience. In this rehearsal, the words colour the way in which each note is interpreted. The superposition of the three different takes on one experience is suggested in the music by repeating a pattern of simple phrases three times, but leaving the cadence at the end of the first two iterations unresolved (like the semicolons in my textual description). The sentiment of flying home is conveyed by moving decisively towards a closing perfect cadence on the final repetition (see Example 2(b)). Rehearsal is the means to discovering what effects are under your control, and how to exercise that control most expressively. Each of the individual semiquavers has its significance in relation to the whole, but their execution is one of many aspects that need to be absorbed into the subconscious for a performance to be effective. The notion of conjunction in experience applies quite as much to matters of detail (such as how the pianist shapes the dynamics of notes within each chord and phrase) as to the overall interpretation. How successful the performance is perceived to be will be highly dependent on matters over which it is hard to gain conscious control, such as what associations come to mind in the particular context of performance.

The above discussions of musical performance exemplify the experience-led crafting of an artefact in association with sense-making. Such an activity is nowhere better represented than in music-making, whether in composition or in performance. Conceiving or performing music invokes moment-by-moment experience. The art of the composer or the performer may be to minimize or to conceal the effort invested in crafting each moment. It may also be to draw the focus of attention to a specific moment by placing it in a context that invites reflection and response.

Empirical Modelling and musical phenomenology

An approach to artefacts in which the primary emphasis is phenomenological rather than functional is not only ill-matched to a computational model of mind but also quite unlike any established way of conceiving computing artefacts in certain key respects. Even in processes like agile programming, in which development involves interaction with partially completed prototypes, there is no counterpart of the coherent journey through states of mind on which both the composer and performer set out to travel step-by-step. This is unsurprising when we consider Brian Cantwell-Smith’s observation concerning the largely mysterious nature of the connection between a piece of software as traditionally conceived and the external world to which its constituent components refer (Cantwell-Smith 1987: 215). By contrast, Empirical Modelling (EM) is a body of principles and tools that has been developed with a view to shifting from a functional to a phenomenological perspective on interaction with computer artefacts. I shall focus here on sketching the analogies that EM establishes between musical and computing artefacts1.

Empirical Modelling is to be practiced and appreciated as an activity that involves tracking states of mind. In this respect it is—at any rate in aspiration—very similar in character to music-making. Its primary focus is on an open-ended process of exploration that involves identifying through rehearsal the stable entities in experience (‘observables’); the relationships that express the way in which changes to these observables are perceived to be directly linked (‘dependency’); and the means by which these observables are subject to change (‘agency’). The artefact and the patterns of interaction that the modeller weaves around it evolve in parallel, and serve as means of exposing tacit understanding (cf. Polanyi 1961, 1983). To echo the Jamesian characteristics of consciousness, as identified by Edelman, such modelling serves to explore—and celebrate—the respects in which the properties of its object cannot be exhausted.

The effect of such activity is to create a network of dependencies between observables, somewhat similar to the defining formulae that relate the cells of a spreadsheet, or to the physical dependencies between keys and vibrating strings in a piano. At any given instant, the network itself metaphorically resembles a single sustained sound, a moment in time, or a state of mind. The most significant feature of a dependency between observables, which is typically expressed by formulating an explicit definition, is that it specifies implicit computational activity to update values (cf. the updating activity in a spreadsheet)—activity that is latent in the model and is hidden from the modeller. As far as the modeller is concerned, such a dependency simply expresses the way in which one change to an observable entails another. And since—on account of the speed at which the update is carried out—this entailment is perceived as an atomic connection, it establishes a form of conjunctive relation such as James identifies in his account of consciousness.

Erlkonig visualisation

The most significant music-related model that has been built with EM tools to date is a model of Schubert’s celebrated song Erlkönig (D. 328)2. The observables in this model correspond quite precisely to the features to which a musician might attend in a performance. They relate to each specific moment in the song, and include display components—as shown in Figure 1—to convey (i) the current key, (ii) the current role being played by the singer, (iii) the words being sung, and (iv) some characteristic features of the accompaniment. The principal observables and dependencies that can be found in this model are depicted in the dependency graph in Figure 2, where each node corresponds to an observable and each incoming edge at a node emanates from an observable upon which its value depends. As can be seen from Figure 2, all the components of the visualization are linked by dependency to the current point in the score, as recorded in the observable termed noofbeats: the ‘number of [triplet quaver] beats’ from the beginning of the song. By way of illustration, Figure 1 corresponds to the moment at which the Erlkönig makes his final appearance, and the current value of noofbeats at this point in the score is 933.

Erlkonig dependency graph

The main novelty in the Erlkönig model is the way it visually conveys the harmonic subtlety of Schubert’s setting. For this purpose, it exploits an animation of the traditional cycle of keys together with a colour wheel that associates colours with keys according to a mathematical formula (see Figure 1). This animation is unconventional in that at certain points in the song there is tonic major–minor ambiguity that is quite characteristic of Schubert, but is especially prominently and unusually expressed in Erlkönig. To convey this harmonic effect faithfully visually, it is appropriate to distort the geometric figure that displays the cycle of keys in such a way that the nodes representing major–minor tonalities associated with the same tonic are from time to time dynamically conflated. Such a distortion is in progress in Figure 1, where the outer circle representing the minor keys is rotating anticlockwise and contracting in such a way that the nodes representing C major and C minor will coincide. This reflects the tonal ambiguity at the point where the Erlkönig issues his final ultimatum to the child.

The entire Erlkönig model was constructed manually by analysing the score—a process that more closely resembled the rehearsal that leads up to a musical performance than conventional computer programming. There was no direct counterpart of the compiling and debugging of a global algorithmic specification of the model behaviour that is characteristic of programming. In musical terms, the mapping of dependency establishes a simultaneous vertical conjunction similar to an aggregation of harmony, texture and context, while the linear melodic elements are first realized through tracing out manually the changes to the primary observables. Automating the execution of the visualization in the model in conjunction with a recorded performance of the song then requires only a synchronization of the current location in the song with a point in the performance. This can be simply achieved through adapting the digital audio output so that the value of the observable noofbeats is incremented on each beat.

In this process of model development, the metaphor of rehearsal is most apt. There is the same sense of gradual building from mastery of momentary state towards gestures extending over time, sometimes developed individually and subsequently integrated. There is a prominent sense of the layering of consciousness. The primitive relationships that are to be expressed by dependency (such as that which determines the current bar number from the number of beats elapsed, or associates a specific colour with the current key—cf. Figure 2) are first consciously crafted, then composed in an incremental fashion to establish more complex relationships. At each stage, what has been successfully expressed through dependencies becomes a part of the modeller’s subconscious environment. This happens both in the mind of the human modeller, in that attention shifts from the mechanism that was used to establish the dependency to the higher level relationships between observables that the dependency establishes, and in the concrete model itself, where the mechanism is no longer being explicitly observed and invoked but serves to update relationships automatically.

As in rehearsal, an important aspect of the shifting focus of attention during model-building is the manner in which—in Jamesian terms—new conjunctions become topical. This is the kind of phenomenon highlighted earlier, in the discussion of Schumann’s ‘In der Nacht’. As the rehearsal proceeds, the performer’s attention moves—as far as his or her skill allows—from individual notes, to musical figures, to synchronizing and balancing parts, to melodic lines, to musical structures and to affective impact. Although this uplifting of attention can be facilitated by studying the composition as a whole, the experience of conjunction itself has to come from the performance. This mode of acquiring understanding (‘understanding forwards’) is identified by James as quite characteristic of radical empiricism, in keeping with that ‘[refusal] to substitute static concepts of the understanding for transitions in our moving life’ mentioned above (James 1912e: 238). A parallel can also be drawn with certain styles of composition, whereby—rather than stating themes that are subsequently developed, as in classical sonata form—a composer first presents thematic material in fragmentary form that is only later integrated into a coherent statement (of which an example might be the first movement of Sibelius’s Second Symphony).

Perhaps the most important affinity between EM and music is the capacity it affords for live migration and negotiation of meaning. The observables in a network of dependencies need have no specific declared meaning—they represent whatever they can enlist from the modeller’s imagination according to the patterns they exhibit when some change to an observable or dependency is introduced. In contrast, the primitive ingredients of a conventional program are themselves behaviours with specific functions optimized to best serve the specific function of the whole. In realizing a behaviour, a conventional program’s engagement with the environment is then intentionally similar to that of a musical recording: it plays obliviously, quite possibly immaculately, but not in such a way as to admit ready integration with any other musical recording.

In EM one model can be composed with another simply by combining two or more networks of dependencies. In the first instance, this combination may involve no more than treating several unrelated networks as if they were one. Much more expressive power can be realized by adapting or extending the dependencies within individual networks so that they become interrelated. A simple, if somewhat contrived, example designed to illustrate this principle adapted an EM model originally developed to emulate an educational program called JUGS (Beynon et al. 1989). This model featured two containers that held integer quantities of liquid, and incorporated definitions of all the observables and dependencies necessary to express operations such as prescribing the capacities and contents of the containers, filling and emptying containers, and pouring liquid from one to the other. The observables in this model were taken in conjunction with a totally independent model of a standard music keyboard. By adding a small set of dependencies it was possible to relate the content of the containers to locations on the keyboard and to modify the visualization of the containers so that they both had the same ‘capacity’ but were of very narrow width3. In this way, the containers could be viewed as representing the strings of a guitar, and the combined model as depicting how the notes that might be played by selecting frets on a guitar would correspond to notes on the keyboard.

James: further prospects

In this chapter, I have set out in broad terms respects in which James’s philosophical stance of radical empiricism holds promise in relation to epistemological problems that currently face both musicology and computer science. The parallels I have drawn between musical compositions and computing artefacts motivate an account that can unify musical and computing perspectives on consciousness. Any such unification must contend with the paradigm that computer science has traditionally endorsed: the computational theory of mind. Boden identifies this theory with the claim that ‘mind is explicable by whatever theory turns out to be the best account of what computers do’ (Boden 2006a: 1428)—which is a paraphrase of Chrisley's characterization of Transparent Computationalism as ‘the claim that the best account of cognition will be given by whatever theory turns out to be the best account of the phenomenon of computation’ (Chrisley 2000: 106). The analogy between computers and musical instruments, the points of convergence between activities such as musical rehearsal and Empirical Modelling, and the modern development of new musical instruments with much greater technological sophistication and scope for autonomous and adaptive behaviour lead me to suggest an alternative thesis: that (to adapt Chrisley’s words) ‘mind is explicable by whatever theory turns out to be the best account of the phenomenon of music, or of what musical instruments do’. Defending this thesis serves to highlight some of the key characteristics of a Jamesian outlook as they relate to Edelman’s view of consciousness and to EM.

Edelman’s emphatic opposition to the notion of the brain as computer has stimulated fierce criticism (see for example Boden 2006a: 1200–5). I have been no more successful than other—better qualified—reviewers in understanding Edelman’s research in point of detail, but my experience of EM comprehensively endorses his contention that investigating how conjunctive relationships between experiences are established in the brain is quite distinct from devising a new variety of computation. Computational frameworks are archetypally concerned with prescribing behaviours that achieve functional goals through manipulating inputs and monitoring outputs that have preconceived, abstractly specified interpretations. By contrast, the studies to which EM and Edelman’s research relate are essentially concerned with creating dispositions in which the human observer perceives semantic connections. In EM the emphasis is on explicitly crafting artefacts through model-building and managing the contexts for observation in such a way as to recruit conjunctive relations in immediate experience. In Edelman’s research, the focus is on determining through experiment how such perceived relations correspond to the organization of neural structures in the brain.

The mind resembles a computer or a musical instrument according to whether we see the brain as performing computation or establishing dispositions. I speculate that Edelman’s preference for the latter interpretation stems in part from his experience as a practicing musician who might in other circumstances have become a professional violinist4. To the musician, making meaning moment-by-moment through the engagement of mind and body with an instrument is quite unlike performing a computation. A computational interpretation relies on being able to invoke causality without ambiguity. As has been illustrated in previous discussions, it is impossible to determine to which of a wealth of factors a musical performance owes its effect.

The principle that we can have no absolute knowledge of causality is one of the fundamental premises of James’s pragmatic stance: ‘no philosophic knowledge of the general nature and constitution of tendencies, or of the relation of larger to smaller ones, can help us to predict which of all the various competing tendencies that interest us in this universe are likeliest to prevail’ (1912f: 180; see also 178–80). This is not to dismiss the importance of the attributions of agency we make when linking the way in which we depress the piano key with the resulting sound, or when—within the context of a particular musical tradition—we associate a certain emotional response with a specific harmonic progression. In respect of musical idioms and compositions, where melodic, harmonic and rhythmic features that are astonishing in one context are commonplace in another, the musician develops a capacity to entertain different realities, and inhabit each as if it were the sole reality. Different concepts are apposite to each context, and have to be assimilated into the subconscious as if they could not be otherwise. But the essential need for the musician to be alive all the while to new possibilities accords well with James’s assertion that ‘as reality is created temporally day by day, concepts ... can never fitly supersede perception … The deeper features of reality are found only in perceptual experience’ (James 1911a/1996: 100, 97). In this way construals of music can legitimately represent perspectives that range freely from the formal to the experiential.

It may also be that deeper exploration of the relevance of radical empiricism for music can offer reciprocal support for some of James’s thinking. As Taylor and Wozniak (1996) observe, James’s philosophical writings have not enjoyed the same degree of respect and influence as his contributions to psychology (most notably James 1890/1950). Marianne Janack (2004) attributes the suspicion with which many later philosophers view James to his failure to acknowledge the epistemological border between philosophy and psychology. In particular, James insists upon considering reasons for belief in conjunction with causes for belief, maintaining that we cannot freely elect to believe or disbelieve propositions on the grounds of their rationality alone. Janack's concise interpretation of James's accounts of the relationship between reality and belief (cf. for instance James 1890/1950, Vol. 2: 297) is that what carries conviction for a person must have for them the sting of reality (Janack 2004). The confluence of formal and experiential factors in music discussed in this chapter highlights the way in which musical appreciation similarly blurs the duality between our intellectual and our passional natures.

Other objections to a Jamesian stance centre on its treatment of language. In response to James’s account of ‘pure experience’, in which the notion of a conjunctive relation plays a fundamental role, the philosopher Graham Bird writes: ‘We are left ... with a puzzle about the role or sense of “pure experience”. It is evidently of great importance in James’s account, and yet also totally inarticulate’ (Bird 1986: 108). Bird goes on to cite Wittgenstein’s observation to the effect that ‘a nothing would do as well as something about which nothing can be said’ (Wittgenstein 1953: 304). The presumptions about the expressive reach of language implicit in such a criticism may be viewed with some scepticism by musicians. They are unlikely to agree with Boden when she writes: ‘the constraints of music, complex though they are, are more amenable to definition than those involved in literature’ (Boden 1995).

An alternative perspective on language that is much closer to the spirit of radical empiricism and EM is framed by Don Paterson in an essay expressing his understanding of what it is to be a poet. Having first characterized primal experience as preceding language and being imbued with a sense of ‘everything as everything else’, he writes:

When we allow silence to reclaim those objects and things of the world, when we allow the words to fall away from them—they reassume their own genius, and repossess something of their mystery, their infinite possibility. Then we awaken a little to the realm of the symmetries again, and of no-time, eternity … [W]hen the things of the world … that we have contemplated in this wordless and thoughtless silence reenter the world of assymetrical concept, of discrete definition, of speech and language—they return as strangers; and then they declare wholly unexpected allegiances, reveal wholly unsuspected valencies. We see the nerve in the bare tree; we hear the applause in the rain. These things are, in other words, redreamt, they are reimagined, they are remade. This I think is the deepest meaning of our etymology as maker. (Paterson 2004)

Perhaps it is no coincidence that Paterson is also a performing musician.


I am much indebted to David Clarke for editorial help in preparing this chapter.


  1. Further discussion of EM in its relation to humanities computing and to music in particular can be found in Beynon et al ( 2006 ) and Beynon ( 2006 ); the association of EM with radical empiricism is the subject of Beynon ( 2005 ). See also the website of Warwick University’s Empirical Modelling Research Group. (URLs of all websites mentioned in the notes are given in the references).
  2. A screenshot from the model similar to that displayed in Figure 9.1 can be seen in context in an online poster (see the directory kaleidoscopeBeynon2005 in the Empirical Modelling Projects Archive), and a variant of the interactive model can also be accessed online via the relevant webpage of Empirical Modelling Web Eden.
  3. See the online JUGS poster in the directory kaleidoscopeBeynon2005 in the EM archive.
  4. For more background, see the biography of Edelman available at: www.notablebiographies. com/supp/Supplement-Ca-Fi/Edelman-Gerald-M.html (accessed 6 July 2010).


Beynon, W.M. (2005). Radical Empiricism, Empirical Modelling and the nature of knowing. Pragmatics and Cognition, 13, 615–46
Beynon, W.M. (2006). Mathematics and music—models and morals, in R. Sarhangi and J. Sharp (eds.) Bridges London: Mathematical Connections in Art, Music, and Science, Conference Proceedings, 437–44 (St Albans: Tarquin Books). Available at: (accessed 30 March 2010)
Beynon, W.M., Norris, M.T., Russ, S.B., Yung, Y.P., and Yung, Y.W. (1989). Software construction using definitions: an illustrative example. Computer Science Research Report No. 147 , University of Warwick. Available at: (accessed 30 March 2010)
Beynon, W.M., Russ, S.B., and McCarty, W. (2006). Human computing: modelling with meaning. Literary and Linguistic Computing 21, 141–57
Bird, G. (1986). William James. ‘The Arguments of the Philosophers’ (series). London: Routledge and Kegan Paul
Boden, M. (1995). Creativity and unpredictability. Stanford Humanities Review, 4, special issue: Constructions of the Mind. Available at: (accessed 30 March 2010)
Boden, M. (2006a). Mind as Machine: a History of Cognitive Science, Vol. 2, Oxford University Press
Boden, M. (2006b). What can AI teach us about arts and letters? Seminar, Centre for Computing in the Humanities, King’s College London, 23 November
Cantwell-Smith, B. (1987). Two lessons of logic. Computational Intelligence, 3, 214–18
Cantwell-Smith, B. (2002). The foundations of computing, in Scheutz, M. (ed.), Computationalism: New Directions, 23–58, (Cambridge, MA: MIT Press)
Caplin, W.E. (1998). Classical Form: a Theory of Formal Functions for the Instrumental Music of Haydn, Mozart, and Beethoven (New York and Oxford: Oxford University Press)
Chrisley, R.L. (2000). Transparent computationalism, in M Scheutz (ed.), New Computationalism: Conceptus-Studien 14, 105–21. Available at: (accessed 30 March 2010)
Cooke, D. (1963). The Language of Music. (Oxford: Oxford University Press)
Cope, D. (2001). Virtual Music: Computer Synthesis of Musical Style. (Cambridge, MA: MIT Press)
Cope, D. (2005). Computer Models of Musical Creativity. (Cambridge, MA: MIT Press)
Desain, P. and Honing, H. (1992). Music, Mind and Machine: Studies in Computer Music, Music Cognition and Artificial Intelligence. (Amsterdam: Thesis Publishers)
Edelman, G. (2005). Wider than the Sky: A Revolutionary View of Consciousness (London: Penguin Books)
Empirical Modelling Projects Archive. Available at: (accessed 6 August 2010)
Empirical Modelling Research Group. Available at: (accessed 6 August 2010)
Empirical Modelling Web Eden. Available at (accessed 6 August 2010)
Garnett, L. (1998). Musical meaning revisited: thoughts on an ‘epic’ critical musicology, Critical Musicology Journal. Available at: (accessed 30 March 2010)
Gjerdingen, R.O. (1990). Categorisation of musical patterns by self-organising neuron-like networks. Music Perception, 7, 339–70
Hatton, R. (1994). Musical Meaning in Beethoven: Markedness, Correlation, and Interpretation (Bloomington, IN: Indiana University Press)
Hofstadter, D. (2001). Staring EMI straight in the eye—and doing my best not to flinch, in D. Cope (ed.) Virtual Music: Computer Synthesis of Musical Style, 33–82, (Cambridge, MA: MIT Press),
Hofstadter, D. (n.d.). Sounds like Bach. Available at: (accessed 29 March 2010)
Hooper, G. (2006). The Discourse of Musicology. (Aldershot: Ashgate)
Jackson, M. (2006). What can we expect from program verification? IEEE Computer, 39 (10) : 53–9
James, W. (1890/1950). The Principles of Psychology. 2 Vols. (New York: Dover Publications)
James, W. (1909). A Pluralistic Universe. (New York: Longmans, Green, and Co.)
James, W. (1911a/1996). Some Problems of Philosophy: A Beginning of an Introduction to Philosophy (University of Nebraska Press )
James, W. (1911b). The Meaning of Truth. (New York: Longman, Green, and Co.)
James, W. (1912a). ‘Does consciousness exist?’, in Essays in Radical Empiricism. 1–38 (New York: Longmans, Green, and Co.)
James, W. (1912b). ‘The essence of humanism’, in Essays in Radical Empiricism, 190–205
James, W. (1912c). ‘The place of affectional facts in a world of pure experience’, in Essays in Radical Empiricism, 137–54
James, W. (1912d). ‘How two minds can know one thing’, in Essays in Radical Empiricism, 123–36
James, W. (1912e). ‘Is Radical Empiricism solipsistic?’, in Essays in Radical Empiricism, 234–41
James, W. (1912f). ‘The experience of activity’, in Essays in Radical Empiricism, 155–90
Janack, M. (2004). Changing the epistemological and psychological subject: William James’s psychology without borders. Metaphilosophy, 35, 160–77
Latour, B. (2003). The promises of constructivism, in D. Ihde and E. Selinger (eds.) Chasing Technoscience: Matrix for Materiality, 27–46, (Bloomington,IN: University of Indiana Press)
Longuet-Higgins, H.C. (1978). The perception of music. Interdisciplinary Science Reviews, 3, 148–215
McCarty, W. (2005). Humanities Computing. (New York: Palgrave MacMillan)
Nikkels, E. (1999). Mahler and Holland, in D. Mitchell and A. Nicholson (eds.) The Mahler Companion, 326–37 (Oxford: Oxford University Press).
Paterson, D. (2004). Rhyme and reason, T S Eliot Lecture 2004; published in abridged form in The Guardian Review, 6 November 2004, 34–5. Available online at: (accessed 6 August 2010)
Polanyi, M. (1961). Knowing and being. Mind, 70 (new series), 458–70
Polanyi, M. (1983). The Tacit Dimension. (Gloucester, MA: Peter Smith)
Robinson, J. (1997). ‘Introduction: new ways of thinking about musical meaning’, in Robinson (ed.), Music and Meaning, 1–22, (Ithaca, NY: Cornell University Press)
Seaton, D. (2005). Narrative in music: the case of Beethoven’s ‘Tempest’ Sonata, in J.C. Meister (ed.) Narratology beyond Literary Criticism: Mediality, Disciplinarity, 65–81 (Berlin and New York: Walter de Gruyter)
Steedman, M.J. (1984). A generative grammar for jazz chord sequences. Music Perception 2, 52–77
Storr, A. (1993). Music and the Mind. (New York: Random House)
Taylor, E.I. and Wozniak, R.H. (eds.) (1996). Pure Experience: The Response to William James. (South Bend, IN: St Augustine’s Press)
Todd, P.M. and Loy, D.G. (1991). Music and connectionism. (Cambridge, MA: MIT Press, Cambridge)
Tovey, D.F. (1944). Beethoven. (London: Oxford University Press)
Wiggins, G. (2008). Review of D. Cope, Computer Models of Musical Creativity. Literary and Linguistic Computing, 23, 109–16
Wiggins, G., Miranda, E., Smail, A., and Harris, M. (1993). A framework for the evaluation of music representation systems. Computer Music Journal, 17(3) , 31–42
Winograd, T. (1968). Linguistics and the computer analysis of tonal harmony. Journal of Music Theory 12, 2–49
Winograd, T. and Flores, F. (1986). Understanding Computers and Cognition: A New Foundation for Design. (Boston, MA: Addison-Wesley)
Wittgenstein, L. (1953). Philosophical investigations, trans. G . Anscombe (Oxford: Blackwell)