TZ7

18 Nov 2025

Tonnetz Autonnetz Controller

This project turns an ESP32-S3 (N16R8) into a self-playing / touch-reactive Tonnetz instrument. It records a short sample through an INMP441 microphone, pitch-shifts the capture across a 13×13 Tonnetz lattice, and excites those one-shots via a hybrid of capacitive-touch input (three CAP1188s) and a relaxed cellular automaton. LEDs, audio playback, and sensing each live on their own FreeRTOS tasks to keep interaction responsive.

Hardware at a Glance

  • Audio out: PCM5102 codec on PCM_BCLK_PIN 42, PCM_LRCLK_PIN 40, PCM_DIN_PIN 41, mute gate on PCM_MUTE_PIN 39.
  • Audio in: INMP441 I²S mic on MIC_SCK_PIN 18, MIC_WS_PIN 16, MIC_SD_PIN 17 recording at 48 kHz / 32-bit.
  • Cap touch: Three CAP1188s on I²C (I2C_SDA 13, I2C_SCL 14) feeding 24 edges that AND into 91 virtual “cells”.
  • LEDs: 91 WS2811 pixels on FASTLED_PIN 1 with per-pixel Tonnetz metadata.

Runtime Layout (src/main.cpp)

  1. setup() wires everything together: initializes FastLED, Tonnetz tables (setup_TNZ()), CAP1188s (i2cScan(), printCapWhoAmI(), setSensitivity()), I²S I/O, captures an initial second of audio (captureAudio()), runs pentatonic detection (detectPentatonicFromRecording()), builds per-cell one-shots (buildTransposed()), seeds the automaton, and launches three tasks (ledTask, audioMixer, capTask).
  2. loop() simply advances the cellular automaton via updateCAAsync() unless a re-record gesture is active.

Key Functional Blocks

Tonnetz + LED Mapping (setup_TNZ, markLEDsFromCA, ledTask)

  • setup_TNZ() maps every LED index to lattice coordinates, assigns MIDI numbers (axis offsets of +3/+4 semitones), tonnetz classes, and hues.
  • markLEDsFromCA() paints active cells in their Tonnetz hue and leaves exponential white trails for recently released cells (g_trail16).
  • ledTask() (core 0) throttles LED refresh, applies the red capture overlay, and decays trails at TRAIL_FADE_MS time constants.

Audio Capture & Preparation (captureAudio, detectPentatonicFromRecording, buildTransposed)

  • captureAudio() streams 1 s of mono audio from I²S, stores it in PSRAM, high-pass filters it, and makes it available for later pitch-shifting.
  • detectPentatonicFromRecording() runs multiple Goertzel passes to build a 12-bin chroma vector, scoring all major/minor pentatonic candidates; the best mask limits autonomous CA notes while touch can bypass it.
  • buildTransposed() pitch-shifts the capture for every Tonnetz cell that’s inside the detected scale. It uses resamplePitch() which performs linear resampling, trims the buffer to clean zero crossings (trimToZeroCross()), and enforces a max length (ONE_SHOT_SAMPLES).

Voice Management & Audio Rendering (ensureCellBuffer, birthCell_impl, audioMixer)

  • ensureCellBuffer() lazily pitch-builds buffers for out-of-scale notes when touch requests them.
  • birthCell_impl() starts voices either from touch (birthCell_TOUCH) or the CA (birthCell_CA), applies a short immunity window so fresh touches are not killed immediately, and records timestamps for LED trails.
  • audioMixer() (core 1) sums all active one-shots with Hann attack/release windows, auto gain scaling, optional Schroeder-style reverb (Reverb struct), and writes stereo frames out over I²S.

Cellular Automaton (updateCAAsync, seedInitialCA, countNeighboursLocal)

  • updateCAAsync() enforces lifetimes (CA_CELL_LIFETIME_MS), runs a limited number of random trials per tick (CA_TRIALS_PER_TICK), ensures population caps (MAX_SIMULT_VOICES), reseeds after moments of silence, and respects touch immunity. Autonomous births obey the detected pentatonic mask; manual touches do not.
  • seedInitialCA() injects a few random notes at boot or after prolonged silence.

Capacitive Touch + Re-Record Gesture (readCAPOnce, capTask, triggerReRecordAsync)

  • readCAPOnce() polls the three CAP1188s, maintains the auto-calibration loop (requiring 5 s of zero readings before calibrationComplete goes true), and fills the CapSignals bitmap.
  • capTask() (core 0) debounces each LED cell, spawns touch-triggered voices, logs active channels for debugging, detects the “hold ≥10 cells for ≥800 ms” re-record gesture, and keeps scanning even while recording by delegating capture to another task.
  • triggerReRecordAsync() starts reRecordTask(), which mutes the amp, clears LEDs/trails, records a fresh buffer, re-runs key detection, rebuilds the pitch-shift tables, then unmutes playback.

Tasks and Timing

Task Core Purpose
ledTask 0 Refresh LED matrix, apply record overlay, decay trails.
audioMixer 1 Mix active voices, feed PCM5102, apply reverb.
capTask 0 Poll CAP1188 sensors, debounce, trigger births, watch gesture.
reRecordTask 1 Spawned on demand; performs 1 s capture + rebuild.
loop 1 (Arduino) Calls updateCAAsync()

[README truncated]

Recent Activity

  • 17f851f attempt at including midi and varying sample start point on playback, as well as (2025-11-18)
  • e62db91 tweaks to parameters to make smoother and encourage more interaction (2025-10-23)
  • 5676e48 rerecord working when holding multiple cells (2025-10-18)
  • 82c2770 no scale restriction touch working (2025-10-18)
  • e3318cd touch integrated for notes in scale (2025-10-18)

Languages

  • C++: 94%
  • C: 4%
  • Python: 2%