earshot: ears for coding agents

First, it hears the file

It puts a sound into words: calibrated tags with the closest alternative readings, AudioSet labels and measurements. It says what it hears, not what you hoped: this camera shutter from the bank read as a snare hit and a box lid closing.

Also in this step: match, rank, find, compare, ab, realness, measure.

252 sounds, stored as code

Every one renders by id with no models and no network. Tap a sound to hear it.

Its launch film, timed by earshot

The sound effects in this 35 s film are takes rendered from earshot’s bank, all seven of 0.4’s new families among them, placed with the launch kit over a licensed music track. The one exception, at 24 s, is a real CC0 recording, followed by earshot’s re-make of it in code. Then earshot drew the sheet below from the finished file: frames, labelled events, spectrogram, loudness and sync to the cuts.

The earshot sheet of the launch film: seven frames, the music’s sections with their genre tags, labelled events such as kick drums, wooden block hits, a punchy impact hit and a water drop, a spectrogram, the waveform, a loudness curve and sync markers against 23 cuts and 20 cues — `earshot sheet`, run on the finished launch film. Its genre tags for the one music track (‘tech house music’, then ‘upbeat corporate pop music’, then ‘epic trailer music’) are hints, as the list further down says.

Made to land on the frame

cut

Tap a mark to hear that role’s sound.

Taps lead the cut, whooshes follow

The launch preset says where each role sits against the cut: a tap 60 ms before it, a knock 40, a click 11; a pop-in thump 30 ms after it, a whoosh peak 37, a swell peak 42.

The numbers are medians measured on a private set of licensed reference clips. Each role also names its level over the music bed, 3 to 10 dB.

SoundKnobs, from the bank

dur trimpitch F#4, 370 Hz

pitch range 250 to 1200 Hz

dur shapeapex

no pitch: “a noise pass-by: its band moves with the motion, it has no note”

dur trimpitch C5, 523.25 Hz

pitch range 130 to 2100 Hz

Knobs that land exactly

dur lands exactly. apex puts a whoosh’s loudest moment within 5 ms of the target (measured within 2.5 ms), and pitch takes a note name or Hz.

Knobs edit each recipe’s own parameters and never resample. Noise sounds refuse a pitch and say why.

kit.lock.json
"mix": {
  "level": "bed",
  "duck_db": 0,
  "stop_down": {
    "before": ["hit", "drop",
               "whoosh.long", "swell"],
    "depth_db": [6, 12],
    "length_s": [0.3, 0.5],
    "release": "on the hit: the music comes back with it"
  }
}

The music steps down before a hit

Instead of ducking under every effect, the music drops 6 to 12 dB for 0.3 to 0.5 s before a hit, a drop, a big whoosh or a swell, and comes back on the hit.

In a private set of licensed reference clips, 53% of strong hits come after a dip of 6 dB or more in the half second before them, against 9% of random moments. The bed model is a median.

pop_1.wavui_pop/softseed 10.180 spop_2.wavui_pop/softseed 20.172 spop_3.wavui_pop/softseed 30.172 spop_4.wavui_pop/softseed 40.180 s

A new seed is a new take

A seed other than the stored one is a new take of the same sound: the noise, jitter and micro-timing change, the character stays.

A kit renders its takes once and caches them, and a take that fails the audit is re-rolled on a new seed. The run here is the four pops from the cut list above, spaced 114 ms apart.

$ earshot sfx check w.wav pop.wav e5.wav
3 / 3 clean
w.wav: clean
pop.wav: clean
e5.wav: clean

Every render is audited

The audit looks for clicks, cut tails, clipping, DC, comb ring, a re-rising tail, a tonal pitch that climbs more than 4 semitones and, for hits, stacked attacks or a build before the hit.

236 of the 252 sounds are clean at their own seed, all 51 of 0.4’s among them. The presets use none of the others.

knock/wood, as rendered by the bank0.15 s

A recording becomes a recipe

fit turned this knock into two strikes and 152 modes, and the module it wrote renders exactly what the fit rendered. CLAP cosine to the source: 0.968.

One fit of one short code-made sound, not a benchmark. The benchmark is below.

How well it hears

98%: of “strong” verdicts named the right kind of sound on a held-out library of 125 real recordings, and 61% the exact variant. Nothing was tuned on that library; the strong band rests on only 3 wrong pairs.
0.92: held-out ROC-AUC for a real recording against a code-synthesized one, with whole sound families held out. A fifth of clips get “uncertain”, and the model is easy to fool.
93%: of the sound effects placed in a 60 s film mix are found as events, 89% of the quiet ones under music. Event tags are hints: right about half the time for exposed events and a quarter of the time under the music.
8 of 15: times the author correctly picked out the code-made sound in a blind test of the first recipes: a coin flip. One listener and 15 trials is a small test, not a proof, and only the made impact was spotted.

A radio sound recordist in a long coat and hat holds a microphone up to a steaming pile driver on a frozen harbour, 1930s

How close a fit gets

The replication benchmark fits a private set of licensed reference clips: 116 clean one-gesture effects in 24 categories, not in the repo. It was tuned on 78 dev clips; two held-out sets were each opened once.

Clips	earshot 0.3	earshot 0.4.0
Dev (78): median CLAP, fits graded high	0.884, 19	0.921, 40
Held-out (38): median CLAP, high	0.878, 7	0.909, 14 when opened (first-round engine); 0.903, 14 for 0.4.0 on the opened set
Fresh held-out (28, harder: more music under the effects): median / mean CLAP	0.853 / 0.844	0.851 / 0.860
Fresh held-out: mel gap to the clip, grades high / good / fair / poor	0.87 dB, 2 / 6 / 14 / 6	0.58 dB, 5 / 6 / 13 / 4
Fresh held-out: takes that pass the take check, take CLAP	8, 0.809	7, 0.863

On the harder fresh set 0.4.0 is level with 0.3 on the median and ahead on the mean, the spectrogram gap and the grades (up on 7 clips, down on 2), but fewer of its takes pass, the bed check can drop a bed the fit needed, and onsets came out blunter (1.00 octave off against 0.53).
Measurement only: CLAP and the mel gap are proxies; nobody listened. The 0.4.0 rows were measured before the release review’s hybrid side fix and were not measured again.
Compare engines on the fresh set, not its numbers with the other rows: its isolation bar is lower. Both held-out sets are now opened; a next round needs a new one.
Grades: high is CLAP 0.92 or more with a mel gap of 2.5 dB or less (the bank’s ship bar), good 0.88 and 3.5, fair 0.80 and 5.0, poor the rest.

What it can hear

Trust it

Loudnessearshot measure
Timing and sync to pictureearshot timeline
Whether a single exposed sound is the kind you asked forearshot match
Picking among candidatesearshot rank
Obvious code-synthesized sourcesearshot realness

Treat as a hint

Tags of events inside a busy mixearshot timeline
Music genre and moodearshot describe

It cannot judge

Taste or emotionyour ears
Provenance or licencesyour ears
Speech contentyour ears
Anything above 14 kHz, from the modelsyour ears

earshot can say “a strong match for a deep thocky key press, 3 ms attack, lands 4 ms after the cut”. It cannot say the film feels premium.

The bank was chosen and checked by measurement, and measurements are models and heuristics, not ears. A film’s own listening pass still decides.

Two men in a yard in front of diamond-patterned doors: one saws a plank on a wooden crate while the other, in a white coat, holds a microphone towards the saw — Sound effects recorded by hand for radio, 1930s.

Give your agent ears

Paste these in a terminal: they clone the repo, install earshot for the CPU and check the install, the models and your CPU settings. The first model use downloads two open models, about 1.1 GB.

Read the README on GitHub

Install earshot

git clone https://github.com/Leonxlnx/earshot && cd earshot
python -m venv .venv
.venv/bin/python -m pip install torch --index-url https://download.pytorch.org/whl/cpu
.venv/bin/python -m pip install -e ".[mcp,llm,dev]"
.venv/bin/python -m earshot doctor --load

Then add it to Claude Code as an MCP server with 16 tools:

claude mcp add earshot --scope user -e EARSHOT_THREADS=3 -- /path/to/earshot/.venv/bin/python -m earshot mcp

Python 3.10+ and ffmpeg / ffprobe on PATH. CPU only.

Ears for coding agents.

First, it hears the file

First, it hears the file

Then it checks the work

Next, it makes the sound

Or it turns a recording into code

Last, it times the whole film

252 sounds, stored as code

Its launch film, timed by earshot

Made to land on the frame

Taps lead the cut, whooshes follow

Knobs that land exactly

The music steps down before a hit

A new seed is a new take

Every render is audited

A recording becomes a recipe

How well it hears

How close a fit gets

What it can hear

Give your agent ears