Interactive Composition: An Interdisciplinary Musical Approach
Abstract. Digital interaction in composition establishes a creative dynamic between musical writing and reading. Composition is interactive between the computer (through the screen-text interface) and the text (through the text-screen interface). To varying degrees, new digital technologies and computer sciences continue to offer ways of controlling more precisely sound and its instrumentation. Sequencers, samplers and digital audio mixers enable sound to be produced and manipulated in a perfect, deterministic manner. What relationships can one have today with these machines? To what does musical ‘writing’ refer when composition is mediated through such technologies? The score is not necessarily the only written trace of a work. Graphics, diagrams, code, formulae and algorithms constitute the essential materials for production, but they are not considered a complex language of signs designed exclusively for musical production. This paper defines interactive composition as a creative dynamic between writing and reading music, based on the empirical foundation of the digital reading of musical scores transcribed onto computer, the effect produced by listening to sound sequences, and the memorisation and identity of these sequences in the temporal continuum.
Being a composer today requires a critical examination of the techniques used to create works of art. The art of composition is no longer limited to elaborating a musical score, with its specific rules of writing and grammar, its laws and symbols, but rather encompasses what is generally called – from conception to completion – ‘musical creation.’ This general expression means that sound techniques extend beyond writing techniques in contemporary music. To varying degrees, the new digital technologies and computer sciences continue to offer composers more precise means of controlling sound and instrumentation. By enlarging the areas governing the presentation of sound – conception, manipulation, treatment, environment, broadcasting – today's musicians have all the advantages they need to create the music of their time. In the complex chain that is musical creation, they may choose to position themselves ‘upstream’ of conception by focusing on thought and technique, ‘downstream’ of the completed work by focusing on performance and interpretation, or in the middle by focusing on feelings and emotion. Once these choices have been made – and have been fixed by the sound production and, as the case may be, the mix – then the music can start to flow. These different stages all form part of the work's foundations.
Moreover, in contemporary music, the score often does not translate the entire work, which is not limited to its written representation but also includes non-transcribable sound sources. Once listening has produced a compromise and the written source has been recorded and fixed in time, then there is no need to write any more. And when a musical score attempts to transcribe new sound sources, it often lacks the necessary notational precision and finds itself using approximate and ambiguous symbols. It is this gap which makes the evaluation of electronic music different from that of acoustic music. This deficit is reduced by production values, though these cannot be transcribed in the same sense that notes, chords and rhythms can. This paradox had already been noted between the 1940s and the 1960s and was reflected in proportional notation, random composition, the forging of new symbols and the representation of the electronic elements of concrete music by symbolic notation. Nonetheless, it remains a part of the evolution of techniques incorporated into music. The rules that govern composition and that made the musical text intelligible no longer apply to the more heterogeneous sound treatments found today. What is more, the automated world of techno-creation frees the composer from dependence on the performer. The machine ‘performs’; it executes orders just as it executes sounds and sound sequences.
The digital sequencer is the tool that best illustrates the interactive approach to writing, from its encoding to digital reading. The looping and random repetition of sequences belong to a technique for remembering sound gestures (MIDI) which ultimately takes shape in the mind like an image with unique characteristics. The reading of a sequence generates an instantaneous sound environment which is not without influence on the creative process. At no other period in the history of music have composers had the possibility of playing the parts assigned to different instruments with such mechanical precision. Traditionally, composers sought inspiration in the piano (the polyphonic instrument of choice): they used it to revise phrases, work out the best harmonic and melodic relationships, or even improvise the beginnings of a new piece. However, this tinkering was not employed systematically and was not used to develop the writing process in specific directions. Composers could, however, spend time getting used to the sound and ‘feel’ of various instruments. However, while this traditional method of hearing sounds was not without influence on the process of composition, it is different from what we are describing, even if only because a work's instrumentation – both from the point of view of writing and listening – is now controlled by just one medium, one interface: the computer keyboard. This practice combines, together and simultaneously, a musical thought and its imminent realisation. When composing, we are now doing the opposite of what we used to do. Traditionally, we used to listen to and draw inspiration from the piano in order to ensure a continuity of sound. In doing so, we remained dependent on certain habits and reflexes, be they the position of the fingers on the piano keyboard or the aural analysis of melodic and harmonic progression. We would write, we would listen, and the score would take shape. With the MIDI keyboard, we input the data and focus mainly on its accuracy. Only then, on listening with playback, does the temporal evolution, the exposition of ideas, the instrumental, dynamic and spatial balance reveal itself, like an object which takes shape suddenly, a sort of distant reply to what was written. The evolution of the writing is accompanied by the aesthetic experience of listening to the MIDI sequences. Of course, the difference lies in the nature of the resultant soundscape and its potential for manipulation.
New technologies provide a diverse range of tools which cover the principal stages of musical creation, from conception to performance. Certain production techniques, including sequencers, samplers and other digital audio mixers, though for the most part used in commercial music (easy-listening, pop music and other ‘techno-creations’), can also benefit composers of contemporary music. And, like the different stages in technical development – from the autonomy of sound vis-à-vis the musical note to the arrival of electronic sound production techniques and their integration into contemporary music, up to computer assisted composition – the attempts by composers to create the music of their time logically follows this evolution. In its broadest sense, computer music is not confined exclusively to processing and treating sound but more generally covers everything which concerns digital representation, synthesis, sampling, filtering, special effects, mixing, editing, sound checks, broadcasting, production, etc. Contemporary musical creation is caught in a permanent contradiction. On the one hand, there is the actual research into musical language, which concerns musical grammar and the more general notions of system and compositional method. On the other hand, there is research into physical acoustics involving scientific techniques and knowledge of engineering, mathematics and computer programming. Finally, and encompassing all the rest, there is the research into the field of aesthetics, the assigment of tasks emerging from digital creation, various modes of applying digital technologies to musical creation, the production of special sound effects linked to the acoustic environment and the statistical and computer-aided analysis of works in musicology. In short, anything that may conceivably be used by contemporary composers to create a musical work becomes a legitimate aesthetic experience. Music is a whole. It is this wholeness that makes it as aware of the different contexts where it is performed as of the inspirational force that these contexts can provide. To a certain extent, and in keeping with its heteronomy, the ear is liberated. It is no longer a question of simply hearing, but also of seeing and touching. And, in liberating itself, music dissipates into the representational mass from which it emerged. The integration of these new representations into music is often criticised by purists, though they should form an integral part of the composer's profession.
There is indeed a close relationship between the processes and techniques that the contemporary musician must master. From this relationship emerges an ambivalence which, in various compositional techniques, is reflected in musical creation. This ambivalence arises from the composer's critical perspective: in listening to music, he may favour attentiveness to sound independent of content, or he may favour the expression of a more elaborate thought system. His judgement therefore oscillates between an awareness of sound as sound, independent of its message, and the expression of an idea and the desire to formulate a message and illustrate the effects of writing. However, in order to make the score intelligible – in order for the message to be expressed – and because writing is the interface between conception and realisation, it privileges duration, sequence, the moment, sound. And, when listening, sound is opposed to time by the seduction of the sound environment. And if one cannot speak of opposition between these attitudes, but rather complementarity, it may be that electronic music based on a transition-less manifestation of the sound environment can suggest a possible compromise.
With regard to a written musical score performed by electronic musical instruments (samplers, synthesisers, drum machines) via MIDI, the computer generates digital sounds. But, concretising written music by transcribing data from the musical score to the computer represents digital music in its strictest sense, because it involves digital performance. In the same manner that broadcasting a publicly performed musical work requires the conversion of analogue/digital acoustic signals by means of recording techniques and, as a result, the passage of sound through the elementary microphone/amplifier/loudspeaker transmission chain, digital music requires these same stereophonic devices if it is to be broadcast. The quality of the sound reproduction system is therefore not without influence on composition, just as conception influences digital performance. As with recording techniques, which are concerned with getting the best balance between sounds in order best to emphasise the effects of writing, corrective listening, until recently limited to electronic music, can extend sound conception. This latter is, at this point, no longer textual, as it is a question of optimising the sound of the compact disc. Technical adjustments to the sound before public performances, which allow the composer to fine-tune the sound environment during rehearsals, are also repeated after the public performance if the work is to be recorded on CD. The technique of recording masters in professional studios generates a sign code which, because it borrows from different musical sources without necessarily reflecting conventional musical language, could be called ‘interdisciplinary.’ This musical polymorphism includes different aesthetic experiences of the process of musical creation and generates different forms of musical appreciation. Ideally, each of these individual experiences will lead to one shared experience, the finished work. At this level, judgement is not questioned but heightened in order to serve the infinitesimal precision of hearing. What is more, in terms of creation in general, the full dimensions of music are only revealed in the relationship between the computer platform used for playback and the listening environment (not at the level of writing, but on this side of its execution). It continues to settle on the computer, directed towards the ultimate digital reading which will fix its laws and determine it as object. If technical adjustments – level checks, filters, editing, mixing, effects all applied via computer screen – made to the recording remain secondary, they are nonetheless not entirely separable from the creative process. It would seem that processing and producing the signal in electronic music requires an ability to distinguish the acoustic and electronic elements of each voice – and this is obviously something required of professional musicians. Because, for the artist, the work must seek to attain some form of perfection.
These different sound and musical production techniques – editing, mixing, special effects, recording techniques, etc. – are founded on a shared principle, reading or playback. Used in professional recording studios or in public concerts as a form of sound-check, playback is opposed to live performance. We are not using playback in its standard meaning of ‘mimed broadcasting of previously recorded material.’ Instead, we are using it literally: playing back, rehearsing, reproducing, playing a sound, a sample, a sequence, a piece of music through speakers – in other words, reading it. Aside from the sound checks preceding the recording of masters mentioned above, the composer has the possibility of hearing what was written and entered into the computer an unlimited number of times. For pieces that are written as they are listened to, as is the case with much electronic music, playback represents a distinct form of musical expression: digital writing, characterised by interaction between sonic invention and formal composition. Analogue writing, representing traditional composition and pre-dating computer transcriptions, is a quite different practice. It relies on compositional treatises to facilitate understanding of the text. These rules – which were born from certain cultural ideals – were designed to orient writing in the direction of a discursive logic. Although founded on natural principles, the homogenous logic underlying symbolic transcription can alienate the ear. Its aim is to enable the performer to reproduce the writing in all its minute, devotional detail. Such practices lose their authority when viewed from the perspective of playback, as the sound environment interferes with the conception process and with sequences already entered into the computer. It is because they do not share the same aspirations that digital writing cannot simply replace analogue writing. Equally, distinguishing compositional practices from one another implies that they employ different forms of musical expression. The same applies to the distinction between writing and reading, if it is indeed the case that reading, in the case of the playback of written sequences, aims to respect the text as well as explore sound.
In the context of ‘digital writing’ (the sequencer and sampler being controlled by MIDI instruments), reading can cause minor variations to the writing. A level of interaction between writing and reading appears and allows different interpretations of the text by, for instance, the modification of the sound parameters per se as well as their evolution in time – MIDI controllers and program change. Whether the composer wants to replace one timbre with another, apply reverb, modify accents, transpose sequences, repeat them as often as desired at different intervals, or even change the tempo, the changes will be heard directly and immediately and will affect the composition. By providing a different means of approaching the creative process, playback would seem to bring writing closer to its object, as the passage from writing to reading evidently passes through listening. Once it is digitised, the musical text is also ‘instrumentalised’ and enters the realm of sound. Once present in the sound environment, the text being digitally read can undergo virtual changes without its written form being affected. The passage from writing to reading (recording) is fluid and flexible and, depending on his approach, the composer can become an active instrumentalist (recording sequences in real time) or a passive instrumentalist (recording sequences step by step). By immediately listening to a sequence transcribed from a written score to the computer, the composer works in close relationship with his outer ear. His judgement is based on the external perception of music (as opposed to the inner ear). The direct transition between writing and reading (the digital performance) neutralises an element in the relationship of the composer to his work, that of human performance. The latter, which requires physical capability as well as intellectual understanding (virtuosity being neither solely a matter of speed nor of precision), cannot be embodied otherwise than in the digital model of the composer, that is to say by compromising his inner ear. Digital techniques change the way music is presented and so change the music itself. This radical change affects the two poles of creation: writing and reading, conception and realisation, creation and interpretation. This evidently has an effect on writing practices.
It is not a question of discussing a situation where music is composed entirely on computers; it may be by invoking the elementary tools of the composer – paper, pencil, eraser, musical instrument – that one can better measure the impact of the new practices of conception and creation. On the contrary, the computer is to be conceived of as a means of channelling music – with the surprising and ambiguous ability of being both performer and instrument – designed to reproduce the written score. If paper and pen are still needed to render an idea concrete (thought is still indispensable in composition), the ear can decide otherwise. By judging what was written to be inadequate – in advance of hearing it – the ear can give it a more musical inflection unrelated to the laws of composition. In this way, sound (the sound environment) gains the edge over time (the duration programmed into the system), reducing writing to the process of ordering events in sequence. From this point of view, using this new tool to create will have an effect on compositional practice and writing techniques, and therefore on the rules of musical language, its theory. The laws of analogue writing essentially aim at giving the discourse temporal coherency and, in this context, sound remains a homogenous link in the discursive chain. However, analogue writing sets itself apart from digital writing by the presence of an intermediary representational space, an expressive interstice, prior to completion: reading. The composer writes, transcribes, listens and adjusts. Digital reading, as an auditory aid to composition, leads to an interactive dynamic between the creative hand and the ear. Without this type of interaction, only the composer's imagination allows him to guarantee the quality of the internal aural representation of the score. This is why, before the advent of these new writing technologies, only pieces which could actually be performed were written. However, if playback performances are devoid of individual interpretation, then there is no other reading than the composer's. They are effectively less supple, as they eliminate an expressive dimension between writing and reading: what was written is read and this reading, above all, confirms what was written. It could be said that this does justice to composers as they now get to perform and interpret their own work – even though it is by means of an instrumental medium which makes no reference to the canon of traditional instruments. The computer is simultaneously a tool for composing and performing. Indeed, it forms a sort of instantaneous writing which, with its successive loops, gives a colour and an identity to the whole sequence. And this exclusively auditory evaluation, which links the composition to the varied world of the senses (sensory memory comes before sound memory), can have an influence on the evolution of the creative process. The new digital audio technologies give composers the possibility of interacting with the text at the listening stage by making simple adjustments to the settings which most often refer directly to the nature of sound rather than its plasticity, to the dramatisation of its movement in musical representation (narrative and discourse). It becomes natural to add or remove elements based on the judgement formed by the outer ear – the ear which receives what it hears from the outside – and so follow the evolution of the work in the course of its being written. It is, to employ an evocative image, written concrete music.
It is of course not a question of isolating performance and the non-negligible human factor from the exploration of music between its writing and reading, but of attempting – from this interactive viewpoint – to think of its neutralisation in favour of listening to sequences and their instantaneous performance. There is a new phenomenon at play which moves beyond the simple practice of writing: the emergence of the machine's mechanical performance in the service of a writing practice. Its only justification is that it is motivated by external hearing and that it brings to the surface a practice which has long been obscured by functional rules linked to traditional writing practices. Traditional practices are themselves legitimised by an appeal to the inner ear and a respect for the text. As it happens, contemporary music is often attacked for not taking the audience into account, for flaunting its intellectual nature and for affirming that its aesthetic aims are removed from the emotion proper to art. Importing scientific thought into the musical apparatus has, at least, the merit of redefining the boundaries of what is musical and what is of interest to composers, from conception to completion. It is naïve to consider that the work of art is created as a mere object of communication, as though it was just a question of repeating the tradition without questioning it. Indeed, contemporary artistic creation invites the artist to adopt a subjective position with regard to established aesthetic values which take their legitimacy from affective reality, the artist's sensibility. It must be accepted that artistic research concerns integrating tradition as well as adapting ideas to new situations encountered by the artist. This justifies the notion of reforming compositional practice by redirecting interest towards the field of instant listening opened up by playback. In effect, the expression of musical ideas requires research into musical language and other forms of knowledge, together with the various modes of musical expression.
Let us not claim that playback is a means of conducting specifically musical research. Contrary to the generally accepted idea, what is more important in music is not what we perceive or feel (and very often one cannot rationalise the musical discourse) but rather the way we perceive or feel music. And this fluid process of perception is constantly modified during playback, that is to say as the score is being written. One obvious example regarding this particular compositional practice is the fact that the acoustic image of a given sequence is not repeated identically to itself and may eventually impose its new shape on the listening process, thus becoming a necessary stage in the creative process. This means that reading is in constant conflict with writing and, from this tension, a new stylistic energy is imparted to the composition. The automated reading of musical texts somehow provides a coherent answer to composers for whom traditional composition no longer meets the aspirations of the public. Worried about the quality of performance when faced with the rare opportunity of performing the work in concert, wanting to be free from the constraints of having to work with musicians, the composer may choose to use the computer because it remains faithful to its programmer: it makes no mistakes and is therefore an ideal performer, it is accuracy personified. It is perfection, but without emotion. With a computer, it is not a question of ‘rehearsal’ in the sense of rehearsing the performers (as the machine already ‘knows the score’), but in the sense of rehearsing the music itself, where what counts is the sound and the way the sound sounds. It is not a question of obtaining co-operation between the different instruments but of finding agreement between the different elements of the score. What is the sonic (to avoid using ‘musical’) identity of each voice? With this in mind, digital technology offers many opportunities for humanising the rigidly mechanical (dressing the listening environment, softening digital sounds, etc.). Therefore, the mechanical reading of the text can overcome its own unfeeling mechanicity.
While auditory memory, by means of the inner ear, allows the representation of sounds in their absence (as though they were images that only a type of plasticity without drawing nor colour could translate), it does not reproduce the physiological process in its totality. If one may speak of a muscular inertia of sounds perceived internally, such sounds nonetheless endure, static, strictly on the mental level, without flowing through the body's nervous system. They are ‘intellectualised’ before being physiologically assimilated. Though affective reality is organic in nature, perceptual resistance to sounds and to their succession in time is psycho-physiological in nature. The interaction born of the writing/reading relationship therefore allows us to pass from the status of representation to that of sonic reality. In the field of musical perception (psychological and physiological), a written fortissimo will never be as unpleasant as when heard, a written tremolo never as frightening as when heard, a written tutti never as effective as when heard. Instead, they will seem out of place and awkward. There is a difference in the way we assimilate, for example, literature and lived sensations: the former is assimilated mentally while the latter are assimilated psycho-physiologically. In this way, from the point of view of the intellect, what will seem inconsistent or disrespectful of the logic of writing, will appear natural when heard. It is on listening that a genuine coherence in the progression from state to state can be felt by our senses and emotions. However, the assimilation of stimuli generated by the music, and initiated by various physical and psychic processes, is never the same for different people, even if the stimuli objectively remain the same frequency.
Having recourse to interactive composition implies a different conception of writing, one which undoubtedly gives more importance to the improvisation which takes place on listening to a work, as the score is elaborated through the reading and rereading of sequences by the computer. In such a practice, the composer is not only guided by the sovereign logic of writing but also by the immediate impact of hearing sound sequences. The relationship to the musical material changes, with intuition regaining a significant place in musical creation and limiting the speculative aspect of writing. In line with this interactive practice, the musical text on the one hand and the seduction of the sound environment on the other, we must ask ourselves how intelligibility interplays with sensitivity, given that they are at opposite poles of the approach to musical composition. If sequences are characterised by a distinct sound morphology and its infinite repetition, their immediate reading implies an external auditory experience. From the point of view of receptivity, the identical repetition of a sequence tends in effect to bring intuition (what the ear perceives straightforwardly) to the foreground. The transition from writing to reading in interactive composition tends therefore to objectify the sequence and to make it an independent event. Removed from the field of writing, subject to a narrative involution, the sequence recovers a peculiar sonic plasticity, whose repetition brings it close to the concept of sonic matter, even if it is initially a matter of textual articulation. The writing/reading interaction makes the text more material, more open to sensations. In any case, it tightens up the links between the abstraction proper to writing and the concrete soundscape. And, if sound intuition is more authentic when read than when written, being based on empirical sensation, the same cannot be said of form or duration intuition, as these require the interconnection of sequences, and the emergence of an overall view of the sonic articulation, that is to say the ‘memorising memory.’ It is in this sense that, in order to respect the coherence of the creative process, the interaction between writing and reading must remain dynamic from the beginning to the end of the composition.
It would seem that for over twenty years now, the MIDI norm has helped bring the interactive composition proper to electronic music (though, as we have shown, it is not limited to this) and traditional composition closer together, at least in principle. It allows the score and its performance to be created directly on the screen. Now all we need do is wait for electronic music to be transcribed on paper for the circle to be completed. This evolution in the practice of making music is not without consequences for musical production itself. It has sped up the process of sonic invention and has fundamentally renewed the formal principles underlying so-called serious music, which had already been strongly challenged. It will in any case allow us to measure the true gap between the composer's inner universe and external reality. It will show that the great movements of modern music – modalism, atonalism, post-atonalism, serialism, post-serialism, minimalism, etc. – were in fact only steps, fashions, in the process of opening up classical music and the affective reality experienced on hearing. It will declare the adoption of these styles as so many inevitable steps in the freeing of sound. But it will also witness the performing limits of machines and the necessity of resorting to humans in order to consecrate sound and achieve music. Based on the respect of the text and sound exploration, interactive composition, that which established itself in the transition from writing to listening, seems to offer an alternative to musical intelligibility. Writing, when thought out and directly experienced with automatic reading, fulfils its objective all the more authentically: extracting the audible meaning from thought and expressing its underlying emotion. If writing, as a principle of communication, substitutes itself for sound and its reception, then the reading which is its personification tends to authenticate – on hearing it – the meaning of that which is written. But what can musical science hope for if writing is not necessarily subjugated – in its founding principles – to the demands of the ear? Within the limits of this article, we have not attempted to claim the existence of a system containing its own solutions. Interactive digital composition is, in the context of the interdisciplinary, a stepping stone towards other approaches to the artistic experience.
About the Author
Composer Frank Pecquet is a professor in computer graphics and sound environments in the Visual Art Department at University Paris 1 Panthéon-Sorbonne and a researcher in the aesthetics of contemporary arts. Frank Pecquet can be contacted by email at frankmc@noos.fr
About the Translator
Colin Bell is a freelance translator and can be contacted at crjbell@oceanfree.net.