Writing

When tools like the stethoscope and electrocardiogram are appropriated for use beyond their intended purpose – for example in music and dance performance – how does the training of foreign users shift the framing of sound, and therefore what types of sounds from the body and apparatus come to be analysed or considered? Can we qualify the difference between a doctor and a composer listening to the heart through a stethoscope? How do the motives and methods of practitioners inform what they hear, and how they touch – to the exclusion of sensory processes beyond their frame of reference? As a choreographer and composer working with heart rate sensors, we do not seek to reveal or represent the invisible workings of the heart. Rather, in our performances and installations, the sonification and haptification of heart rhythms – along with the noise produced by the apparatus and its use – are a means to compose a context for intimate listening and touch between performers and visitors at the edges of appropriateness.

In this article we propose a biorelational framework for performance with biosensors, in which interactions are not based on causality, control, and representation, but rather, manifest through shared awareness and agency across multiple, fluid assemblies of self, other and environment. The transdisciplinary scope of this study traces trajectories from the performing and somatic arts into philosophy, biomedicine, cognitive science and human-computer interaction. A brief survey of common approaches to interaction design with biosensors will contextualize discussion of our current practice-based research and creation project, ‘Choreography and Composition of Internal Time’. In this project, we are examining temporal relationships between physiological processes, such as heart rate and breath, with rhythms in movement, music and mediated environments.

In this article we discuss the ethical and æsthetic implications of the appropriation of biomedical sensors in artistic practice. The concept of cross-disciplinary appropriation is elaborated with reference to Guattari’s ethico-æsthetic paradigms, and Barad’s metaphor of diffraction as methodology. In reviewing existing artistic projects with biosensors, we consider ways in which the recontextualization of technologies, and likewise techniques, can both propagate and violate disciplinary expectations and approaches. We propose that by way of critical appropriations of biosensors in artistic practice—that is to say, de- and re-contextualizations of biosensors that acknowledge the shift of ecology and epistemology—artists have a vital role to play in troubling reductive representations of bodies, and furthermore, destabilizing the ethico-æsthetic boundaries of differently constituted disciplines.

O.—odot—is a portable media programming framework based on the OSC data encoding. It embeds a small expression language which allows writing and executing programs in OSC structures. The integration of programming and declarative functional descriptions within data transfer protocols enables structured and expressive communication in media systems: program snippets can be distributed in OSC messages, which evaluate to further OSC messages in the different communicating software. We present experiments using this framework in the OpenMusic computer-aided composition environment, and illustrate via case studies some advantages of such integrated system.

We present a practice-based research project, which examines temporal relationships between physiological processes with human movement and mediated environments. Ongoing research will provide a foundation for the creation of performances in which real-time heart rate data from contemporary dancers is used to generate click tracks for live musicians. The complex constructions of time in this project necessitate novel visualization strategies throughout the research, development, creation, and production process, including: time domain display of data with rich and facile interaction using bokeh; visualization of biometric data as a form of biofeedback; and unique representations of time in music notation.

We present computer-aided composition experiments related to the notions of polyrhythmic structures and variable tempo curves. We propose a formal context and some tools that enable the generation of complex polyrhythms with continuously varying tempos integrated in compositional processes and performance, implemented as algorithms and user interfaces.

We present a re-realization of David Wessel’s Antony (1977). Antony was originally composed using a novel set of additive synthesis techniques developed for the 4A and the PDP 11/40. Those techniques continued to play a role throughout Wessel’s life as an improviser, and inspired many of his students and colleagues. Our re-realization of Antony involves using those same techniques to reinterpret and perform the very piece that they were used to create.

We present recent work on odot, a system that extends Open Sound Control and facilitates the rapid and dynamic construction of Message-Oriented Middleware providing an interoperability layer for communication between applications. Unlike traditional middleware systems, odot, when embedded in a host environment, provides a node where computation can take place, allowing middleware to take shape dynamically as the needs of the system develop.

EFFICACe est un projet de recherche centré sur les outils de composition assistée par ordinateur (CAO), explorant les rapports entre calcul, temps et interactions dans les processus de composition musicale. Nous présentons différents travaux en cours dans le cadre de ce projet, concernant l’application de processus interactifs pour le traitement et l’ordonnancement des séquences temporelles, le traitement et la spatialisation sonore, ou encore les interfaces homme-machine.

Media/arts programming is often experimental and exploratory in nature and requiring a flexible development environment to enable continually changing requirements and to facilitate iterative design in which the development of software impacts the design of a work of art, which in turn produces new requirements for the software. We discuss agile development as it relates to media/arts programming. We present aspect-oriented programming and its implementation in Max/MSP using Open Sound Control and the odot library as tool for mobilizing the benefits of agile development.

We present work on the “o.io” system, a suite of tools for hiding vendor-specific and protocol- specific details of controllers and actuators and for replacing diffuse documentation and heterogeneous ontologies with harmonized, situational schema carried along in real-time with gesture and actuator control values as Open Sound Control (OSC) bundles. We introduce useful general design patterns and object-oriented tools that support them. We conclude with details of support of two particular devices that illustrate the potential of “o.io”, the QuNeo and Bluetooth LE heart rate monitors.

This paper introduces “o.expr,” an expression language for dynamic, object- and agent-oriented computation of gesture signal processing workflows using OSC bundles. The use of o.expr is shown for a range of gesture processing tasks. Aspects of o.expr, including statelessness and homoiconicity, simplify agile applications development and provide support for heterogeneous computational networks.

This paper addresses the problem that most electrophones and computer-based musical instruments are ephemera lasting long enough to signal academic and technical prowess. They rarely are used more than in a few musical performances. We offer a case study that suggests that longevity of use depends on stabilizing the interface and innovating the implementation to maintain the required stability of performance for players.

An effective programming style for gesture signal processing is described using a new library that brings efficient run-time polymorphism, functional and instance-based object-oriented programming to Max/MSP. By introducing better support for generic programming and composability Max/MSP becomes a more productive environment for managing the growing scale and complexity of gesture sensing systems for musical instruments and interactive installations.

A new media programming style is introduced that brings efficient run-time polymorphism, functional and instance-based object-oriented programming to Max/MSP and related visual dataflow languages. Examples are presented to illustrate new, unusual and effective applications of the approach of using OSC messages for object representations and data flow for method delegation.

We present a parametric method for the variable control of tempo with specification of shape and phase alignment constraints, and its implementation as a graphical interface written in C for the Max/MSP environment. This tool aids in the construction and management of polyphonic streams of independent, fixed or smoothly-varying tempos suitable for live performance or computer-generated scores. We also present a brief history of how polyphonic tempo manipulation has been used in context and compare our work with two recent projects to implement similar tools.

We present two complementary approaches for the visualization and interaction of dimensionally reduced data sets using hybridization interfaces. Our implementations privilege syncretic systems allowing one to explore combinations (hybrids) of disparate elements of a data set through their placement in a 2-D space. The first approach allows for the placement of data points anywhere on the plane according to an anticipated performance strategy. The contribution (weight) of each data point varies according to a power function of the distance from the control cursor. The second approach uses constrained vertex colored triangulations of manifolds with labels placed at the vertices of triangular tiles. Weights are computed by barycentric projection of the control cursor position.

We present an additive synthesis engine that can produce smooth transitions between timbres without glissandi or audible transients. With this algorithm, one can explore the musically rich area of hybrid timbres that arise as one timbre “migrates” into another. This method operates in real-time using live or pre-recorded sinusoidal analyses as input. Parameterization of the migration behavior enables interactive control. We present the history of this engine that begins with work by David Wessel at IRCAM in the 1970s. We discuss two implementations: a statistical method based on sampling from a discrete probability mass function derived from a sparse input spectrum, and a stochastic local-update method using parallel Markov Chain Monte Carlo processes, which enables control over spectral flux, spectral smearing and migration rate. We conclude with a summary of notable musical uses in contemporary electroacoustic music.

Timbre can affect our subjective experience of musical dissonance and harmonic progression. To this end, we have developed a set of algorithms to measure roughness (sensory dissonance), and pitch correlation between sonorities, taking into account the effects of timbre and microtonal inflection. We proceed from the work of Richard Parncutt and Ernst Terhardt, extending their algorithms for the psychoacoustic analysis of harmony to include spectral data from actual instrumental sounds. This allows for the study of a much wider variety of timbrally-rich acoustic or electronic sounds which was not possible with the previous algorithms. Further, we generalize these algorithms by working directly with frequency rather than a tempered division of the octave, making them available to the full range of microtonal harmonies. The new algorithms, by yielding different roughness estimates depending on the orchestration of a sonority, confirm our intuitive understanding that orchestration affects sensory dissonance. This package of tools presents rich possibilities for composition and analysis of music that is timbrally-dynamic and microtonally-complex.

We describe a tool for real-time musical analysis based on a measure of roughness, the principal element of sensory dissonance. While most historical musical analysis is based on the notated score, our tool permits analysis of a recorded or live audio signal in its full complexity. We proceed from the work of Richard Parncutt and Ernst Terhardt, extending their algorithms for the psychoacoustic analysis of harmony to be used for the live analysis of spectral data. This allows for the study of a wider variety of timbrally-rich acoustic or electronic sounds which was not possible with previous algorithms. Further, the direct treatment of audio signal facilitates a wide range of analytical applications, from the comparison of multiple recordings of the same musical work to the real-time analysis of a live performance. Our algorithm is programmed in C as an external object for the program Max/MSP. Taking musical examples by Arnold Schoenberg, Gérard Grisey and Iannis Xenakis, our algorithm yields varying roughness estimates depending on instrumental orchestration or electronic texture, confirming our intuitive understanding that timbre affects sensory dissonance. This is one of the many possibilities this tool presents for analysis and composition of music that is timbrally-dynamic and microtonally-complex.