ADC 26
SPEKANA: Field Recordings as Compositional Material
Composer-directed analysis and mapping for ambient music
Abstract
Field recordings contain spectral, temporal, and dynamic patterns, together with spatial cues, that can inform composition beyond the direct use of recorded sound. Drawing on spectral and post-spectral practices, and on ambient music’s attention to atmosphere and listening conditions, SPEKANA connects listening, audio analysis, and composer-directed mapping within the DAW.
Developed as a C++/JUCE VST3 plugin, SPEKANA uses short-time Fourier analysis and residual-based spectral peak detection to extract prominent frequency components. Captured pitches can be transferred to DAW instruments as MIDI or exported as MusicXML. Users can adjust the time scale and map pitches to a selected palette.
SPEKANA also tracks spectral centroid, spread, flux, flatness, RMS level, and stereo energy balance. These features can be exported as MIDI CC data and mapped to synthesizer and effect parameters in Ableton Live through SPEKANA Bridge, a companion Max for Live device.
SPEKANA offers a practical workflow for turning the characteristics of recorded environments into musical material.
Research context
Ambient music & Deep Listening. Brian Eno’s ambient music and Pauline Oliveros’s Deep Listening inform this project’s attention to atmosphere, environment, and listening as a compositional practice. [1] [2]
Workflow gap. Existing spectral tools provide detailed analysis, but moving from analysis to editable compositional material can involve separate environments and data transfer. [3] SPEKANA brings analysis, MIDI/MusicXML output, and parameter mapping into a composer-directed DAW workflow.
Methodology
SPEKANA connects two kinds of material: discrete pitch observations and continuously changing audio descriptors. The composer decides how each is used.
Listen and select
Choose a field recording, an excerpt, and moments or changes worth following. The source and the listening decision remain part of the process.
Analyse and map
Inspect spectral peaks and descriptor curves. Choose which features to retain, where to send them, and how to set the mapping range and response.
Compose and revise
Use live MIDI for another track, MusicXML for notation, or MIDI CC via SPEKANA Bridge for parameter control. Audition and edit the result within the composition.
MIDI and MusicXML provide pitch material; descriptor curves provide MIDI CC control through SPEKANA Bridge. Time scale and pitch-palette choices shape how the derived material enters the composition.
Flowchart text version
The DAW host audio buffer enters processBlock(), which splits into analysis and audio pass-through. The pass-through branch goes directly to the DAW host audio output.
Buffer-based analysis measures stereo energy and downmixed mono RMS. Frame-based spectral analysis uses mono downmix, a 2048-sample frame with a 512-sample hop (75% overlap), Hann windowing, a 2048-point FFT, and temporal smoothing (α = 0.9).
The spectral path splits into peak detection and spectral features and descriptors. Peak detection is followed by mapping to scales. Mapped pitches, spectral descriptors, and buffer-based measurements feed visualization, capture, and output.
From analysis to material
The following are possible mappings for compositional exploration, rather than fixed interpretations of the recording.
| Observation | Possible musical use | Composer’s decision |
|---|---|---|
| Spectral peaks | MIDI notes, chords, or pitch bend | Choose peaks, register, and scale or chord constraints. |
| Selected events / spectral change | Note timing, duration, or activity patterns | Select events and temporal spacing. Flux alone is not a rhythm transcription. |
| RMS / energy contour | Note velocity, expression, or gain | Set the dynamic range and response; energy is not identical to perceived loudness. |
| Centroid, spread, flatness, flux | MIDI CC for filter cutoff, bandwidth, noise mix, or modulation | Choose the destination, range, polarity, and smoothing. |
| Stereo energy balance | Pan or spatial-effect control | Interpret channel balance as a control signal, not a reconstruction of the original space. |
Descriptor definitions and spectral-change analysis are discussed in the FluCoMa documentation. [4] The mapping choices above are proposals for this project.
Time & pitch mapping
1. Time scale
Adjust the time scale to shape the pace at which the derived musical material unfolds.
2. Pitch palette
Map captured pitches to a selected palette, such as a scale or chord. The composer chooses the pitch collection in relation to the piece.
These choices connect the recording’s characteristics with the temporal and tonal organisation of the composition.
Spectral-capture interface
Select a feature to locate it in the interface.
The spectrogram displays the changing frequency content of the host audio. Freeze markers indicate captured moments.
Captured pitches appear as chords on the embedded staff. These chords can be reviewed and exported as MIDI.
The peak list shows note names and frequencies. 12-TET and 24-TET modes are available; quarter-tone notation uses arrow accidentals.
Figure 2. The earlier spectral-capture interface. The newer descriptor and Bridge modules are not shown in this screenshot. Click the image to enlarge.
Demonstration plan
Use the same field-recording excerpt across three conditions, documenting the composer’s choices and keeping other settings fixed where possible. These comparisons are planned; no results are claimed here.
Walkthrough
Show the source recording, analysis, mapping choices, and resulting DAW material. Video to be added.
Comparison conditions
Condition 01
Source and baseline mapping
Present the original recording alongside a documented mapping with no additional time scaling or scale/chord constraint. State the existing pitch quantisation, instrument, and parameter settings.
Comparison audio and observations to be added.
Discussion
What does the recording inform?
The proposed contribution is a way to trace how a recording influences pitch organisation, temporal development, dynamics, and parameter movement. A descriptor does not contain a ready-made musical meaning; that relationship depends on the composer’s mapping and subsequent editing.
Where is agency located?
Agency lies in selecting the source, choosing and rejecting observations, defining mappings, adjusting time and pitch constraints, and deciding what enters the piece. Automation can assist these steps without making the work an autonomous composition system.
Interpretation and limitations
Spectral peaks are not necessarily perceived fundamentals; spectral change does not by itself establish rhythm; and stereo balance is only one cue in a recorded spatial image. Transformed material may also lose the recognisability or context of its source. These limits are part of the compositional enquiry.
Evaluation
Document how each mapping changes the musical material, which relationships to the source remain audible, and which decisions the composer revises. Comparisons should examine different outcomes rather than assume that slower or more scale-constrained material is inherently more ambient.
Compositional examples, process observations, and findings remain to be supplied. No claim of improved creativity or agency has been established by evaluation here.
Paper & references
Manuscript
Paper PDF to be added.
Implementation
- Eno, B. (1978). Ambient Music. Liner notes for Ambient 1: Music for Airports. Linked text is a reprint.
- The Center for Deep Listening at Rensselaer. Deep Listening. Includes an excerpt from Pauline Oliveros’s Quantum Listening: From Practice to Theory (To Practice Practice) (1999).
- IRCAM. Other Analysis. AudioSculpt 3 documentation: spectral-analysis results exported as SDIF for use in OpenMusic or Max/MSP.
- FluCoMa. SpectralShape and OnsetFeature. Project documentation.