The Egg The Cart The
Horse The Chicken: Cyberwriting, Sound, Intermedia
Hazel Smith, University of Canberra
Roger T. Dean, University of Canberra
About the authors...
This article, which is based on our hypermedia work, The Egg The Cart The Horse The Chicken, focuses on the way new technologies are shifting page-based notions of literature. Firstly, it argues that hyperlinking and animation allow for new kinds of textual structure and linguistic mobility. Secondly, it suggests that the merging of words, sound, and visual image within the domain of the computer can create new types of meaning through semiotic juxtaposition and exchange. Thirdly, the essay discusses ways in which creative uses of hypertext can be exploited educationally.
Hypermedia is drastically changing definitions of literature and extending the way that we write and read. Hyperlinking, animation, and interactivity allow cyberwriters to experiment with structural and linguistic innovations, use dramatic spatial effects, and engage an audience, making writing and reading fluid and multiple activities. Hypermedia also allows for the linking of word, sound, and image as "intermedia" within the space of the computer, so that the domain of verbal meaning is extended through exchange and juxtaposition with other semiotic systems. In this article we will discuss the way in which new technologies expand notions of literature with reference to our work The Egg The Cart The Horse The Chicken at www.australysis.com/eggsite/theegg/eggfinalv2.html. The text and animation is written in Flash by HS and the sound in Max by RTD.
An interactive demo (~300 KB) of the The Egg TheCart The Horse The Chicken.
Requires Shockwave plugin.
An external link to the authors' work at http://www.australysis.com/eggsite/theegg/eggfinalv2.html
A fundamental idea behind the piece (hence the title) is the way in which linear systems are constantly disrupted by non-linearity. This is written into the piece at a formal level by the use of the hyperlinks, animation, and a split screen, all of which tend to disrupt the normal reading processes. Another central and related idea is that of causality, and the ongoing difficulty of distinguishing it from correlation or succession, particularly in biological phenomena (in fact, the title is related to a section heading in a preceding scientific article by one of us (Eaton and Dean 2000)).
2. Writing for the Screen
Hypertext provides a fertile base for cross-genre work because prose, poetry, and theoretical ideas can all be interlinked, creating a synergy between different types of writing and different ways of approaching the same ideas. The Egg The Cart The Horse The Chicken consists of a multi-dimensional and fictocritical/postcritical network of interlinking poetry, prose, aphorisms, short narratives passages, and historical and theoretical material. The texts adopt a wide range of voices and attitudes, ranging from the satirical to the surreal, consequently decentring the notion of any predominant and unifying authorial presence. Thematically the texts also move between different times and places, thereby linking past, present and future, the global and the local.
The demands and possibilities of writing for the screen are very different from those of the page and result in a distinct topography of writing and reading. Amongst these possibilities is the way the screen can be spatially divided into two or more units that provide for textual simultaneity and juxtaposition. In The Egg The Cart The Horse The Chicken the screen is split horizontally, and an independent sequence of texts is organised in both the upper and lower frame, grouped into short linear 'scenes' that form a 'movie' in each frame. The sequence in the upper frame can be disrupted by clicking on hyperlinks (usually marked in capital letters), which allow the reader to jump to texts other than the ones that follow each other in the movie sequence. There are, however, no hyperlinks in the movie playing at the bottom of the screen, which runs straight through and then recycles when finished. Consequently the juxtaposition of the texts on the two different screens is also variable. Readers or 'screeners' will take their own route through the text each time, and no two routes will be the same, unless screeners choose not to use any links but rather simply let the movies run.
Many other spatial effects and activities, impossible on the page, are possible on the screen. Animation creates kinetic potentials, and these reform not only writing practices, but also reading strategies, as viewers are free to scan texts and take a non-linear path. Words/letters /or blocks of texts can move across the screen at any pace, disappear off the edge of the screen, appear in one part of the screen, disappear and reappear in another part. Words/letters /blocks of text can also morph into, or overlay, each other. Such manoeuvres often destabilise any concept of fixed meaning. While cyberwriting sometimes straddles the edges of readability in challenging ways, it also has the potential to create a rich semantic context in which there is a fast proliferation and plurality of meanings and associations.
3. Screening The Egg
The Cart The Horse The Chicken
In The Egg The Cart The Horse The Chicken animation occurs within both screens, sometimes at the same time, sometimes at different times, creating unpredictable shifts in emphasis. Within each movie, some scenes stand on their own, but other scenes are grouped into clusters based on related texts or transformations of the same text. So
Here the information provided in the second frame changes the import of the first version of the frame, and this added portion of the text is then treated iconically in the third frame (it becomes a mirror image).
In some cases a text that is relatively prosaic in one scene undergoes considerable deconstruction in the next, drawing it closer to the genre of concrete poetry. So
Sometimes animation occurs within a single text, but elements move, appear, or disappear:
Although animated texts are sometimes non-interactive, The Egg The Cart The Horse The Chicken invites interaction; the pace of animation in the top screen is quite slow and can be interrupted by clicking on hyperlinks. Consequently, the reader/screener has a continuing choice between letting the movie run of its own accord and interrupting its flow by clicking on a link. The dominant aesthetic is, therefore, synecdochical, but, whereas in a synecdoche, the part usually stands for the whole, in hypertext, the synecdoche is dynamic: the sum of the parts, always greater than the whole, does not add up to fixed and unified object.
Hyperlinking tends to create
rhizomatic structures that interlink disparate ideas. It is based on metonymies
(associative structures) that are centrifugal in essence, and push outwards,
pulling against the centripetal motion of metaphor, which is based on perceived
similarities between different objects. This tension between metaphor and metonymy
is important in all literary works and is particularly foregrounded in poetry
(Smith 2000). The Egg The Horse The Cart The Chicken is constructed to
produce a continuous tension between metaphor and metonymy: the words egg, horse,
cart and chicken are all metaphorical magnets, but also, paradoxically, the
origins of sprawling hyperlinked metonymies. In the following passage the egg,
as a metaphor for fertility, is already somewhat undermined because the egg
is "pickled" and separated from the woman:
In addition, rather than developing the egg/fertility metaphor, the metonymic links lead firstly to a passage about the privatisation of women's bodies, and secondly to a text about female desire in the Roman empire. The result is a huge network of associations, in which numerous concepts, ranging from non-linear structures in science to the atemporal nature of memory, intersect.
4. Cyberwriting as Intermedia
So far we have only talked about the verbal element of The Egg The Cart The Horse The Chicken, but it has a significant intermedia aspect. Intermedia juxtaposes word, image, and sound, which allows semiotic exchange to take place. Semiotic exchange occurs when different semiotic systems converge (and no one system is allowed to predominate): the result is that different semiotic systems take on one another's characteristics (Smith and Dean 1997). In this piece, as in some, but not all of our other web works (Smith, Dean et al. 1998; Smith, Dean et al. 1998; Smith and Dean 1999), we are less concerned with introducing visual images, and more concerned with the visual quality of the words themselves. The screen is the basic unit, and the texts are designed to fit onto the screen: this means that the texts take the form of bite-sized "screenfuls." The signifier is emphasised as much as the signified; words become visual objects, and spacing, fonts and colors play an important part in the reader's experience, creating a synergy between the meanings of the words and the way they look. Similarly the colors emphasise or contradict the meanings of the words, often extending or interrogating them. Important also in The Egg The Cart The Horse The Chicken is the heterogeneity of the colors and the way they match the diversity of the texts: there is no one color scheme. Instead, like the texts, colors recur, recombine, and counterpoint each other, forming their own metonymic patterns. Color is integral to the structure of the piece; the lower screen is in black and white, while the upper screen is in color.
The sound field of the piece likewise engages in semiotic exchange: it challenges conventional and predictable ideas of rhythmic and repetitive genres, but, at the same time, it has a continuity that contrasts with the abrupt discontinuities of the text and animation. Written and then autogenerated in the program Max, it was rendered by using the sets of standard MIDI instruments available on the contemporary Macintosh and PC. Because we wanted the sound to be continuous while the text is discontinuous, we chose to have a non-interactive sound field for this work, though some of our previous works have used interactive sound. Ideally, the sound piece would be rendered directly from a MAX file, so that it would vary every time it is played; but this would require all users to download a much larger MAXPlay (freestanding) file, which we judged undesirable.
MAX is an extremely flexible platform for the algorithmic programming of almost any kind of musical process, and it has been extended to permit raw sound manipulation in MSP and image manipulation in NATO. MIDI, or Musical Instrument Digital Interface, is an acronym used to describe a standard set of codes that pass through interfaces between computers and synthesisers and samplers. These codes provide meta-data to trigger the performance of musical instruments or other sounds. In The Egg The Cart The Horse The Chicken an autogenerated performance was recorded as a MIDI-file in Quicktime, a standard multimedia technology and plug-in available for all PCs. This file comprises instructions for sounding the standard inbuilt computer musical instruments. Thus it can be played by each user, and it sounds similar on different computers.
The resultant sound field involves a systemic (minimal) musical approach, in which algorithmic construction of rhythmic pitched motives and their rhythmic impact change with time. A sequenced probabilistic method is used to control the number of notes in the repetitive unit motive (ranging gradually from seven to eleven equal-length notes), and to introduce deviant notes intermittently. (In deviant notes, an individual pitch in the repeating pattern changes during one rendering of the pattern, and then generally returns to the original pitch in the next rendering.) However, eventually the frequency at which the deviations occur is so high that more often than not the note changes again on the next repetition rather than returning to its original pitch. The sequenced probabilities also control the speed of the notes and, therefore, the duration of each rendering of the motive. The music gradually accelerates and, at the point that the pattern becomes eleven notes, the duration of the unit pattern is briefly almost identical to its duration at the outset with only seven notes.
The sound field reverses
in mid-stream, so that the reader/listener can experience many different rhythmic
juxtapositions with the text and animation. The reversal plays a mirror image
of the sound score, in that the events occur in reverse sequence, but each individual
event sounds identical in the forward or backwards direction (in other words,
the sound files do not play in reverse, but rather musical events occur sequenced
in reverse). The reversal was timed so that most viewers would still be screening
the text when it occurs, and hence would experience new sets of sound-animation
juxtapositions. Overall, the sound field sustains, yet queries, linearity and
predictability: cause is not clearly distinguished from correlation or sequence.
Many aspects of this construction destabilise the expectations of the minimal music genre typified in the work of Steve Reich and others. For example, the genre generally involves a tempo which is fixed within the precision of human performers. Normally the rhythmic interest in minimal music is generated by repetition, but also by simultaneous juxtaposition of rhythmic patterns of different durations, for example of 3, 4 and 15 units of length, which come into phase with each other only once every 60 units. Instead, the 'Egg' music involves sequential rhythmic perturbations (See discussion in (Dean 1997)). The sound of The Egg The Cart The Horse The Chicken, like the text, challenges norms of stability, unity, and regularity.
5. Digital Creativity
Many interesting educational ideas are intrinsic to an approach such as we have developed above. For example, science is often perceived by lay people as a system of definitive knowledge, although it can be argued that it is one of fallibility. It can be as dangerous to rely on a received scientific opinion as on any other. Our emphases in the themes and construction of the 'Egg' upon unpredictability, and non-linearity, are corollaries of this view.
In a very practical way,
we and other web-artists have sought to enhance the multiplicity of interpretation
available to a screener. In an educational context, this approach is often desirable
when we want to encourage learners to generate new ideas and interpretations
and explore dissonant thoughts about topics in which prejudice abounds. On other
occasions, it may be undesirable, for example, when our teaching goal is the
transmission of relatively unambiguous knowledge, such as algebraic formulae
or a programming language.
Like the linear narrative, the efficacy of traditional 'chalk and board' teaching has been challenged and is often replaced or extended by interrogative and non-linear approaches, such as 'problem based learning' and collaborative learning. Educators need to turn attention to a fundamental issue in web education which seems as yet little studied: the impact in educational hypertexts of linearity versus non-linearity (and of the associated issues discussed above) upon the efficiency of learning (Campos 2000; Moreno and Mayer 2000). Because the web offers us multiple approaches to learning, our studies of its educational impact may differ from discipline to discipline. Empirical studies of learning efficiency based upon the different types of hypertext also need to be explored before we can fully exploit this new potential for learning.
Campos, M. (2000). "The Hypermedia Conversation: Reflecting Upon, Building and Communicating Ill-defined Arguments." Interactive Multimedia Electronic Journal of Computer-Enhanced Learning (IMEJ) 2(2).
Dean, R. T. (1997). "Polyphonies of Pulse: on the control of pulse and meter in computer-interactive improvisation." MikroPolyphonie: http://farben.latrobe.edu.au/mikropol.
Eaton, J. W. and R. T. Dean (2000). Diabetes and atherosclerosis. Atherosclerosis: Gene expression, cell interactions and oxidation in R. T. Dean and D. T. Kelly (eds.) Oxford, Oxford University Press: 24-45.
Moreno, R. and R. E. Mayer (2000). "A Learner-Centered Approach to Multimedia Explanations: Deriving Instructional Design Principles from Cognitive Theory." Interactive Multimedia Electronic Journal of Computer-Enhanced Learning (IMEJ) 2(2).
Smith, H. (2000). Hyperscapes in the Poetry of Frank O'Hara: difference, homosexuality, topography. Liverpool, Liverpool University Press.
Smith, H. and R. T. Dean (2001). Intertwingling. HOW2 1 (5). http://www.scc.rutgers.edu/however/v1_5_2001/current/special-feature/intertqt/startqt.htm
Smith, H., R. T. Dean, et al. (1998). Walking the Faultlines. Cyberquilt: a CD Rom Anthology. San Francisco, International Computer Music Association.
Smith, H., R. T. Dean, et al. (1998). Wordstuffs: The City and The Body, http://www.abc.net.au/arts/stuff-art/stuff-art99/stuff98/10.htm.
Smith, H. and R. T. Dean (1997). Improvisation, Hypermedia and the Arts since 1945. London, Harwood Academic.
********** End of Document
|IMEJ multimedia team member assigned to this paper||Yue-Ling Wong|