CodeConcerto

Hear the bug. Fix the song.

CodeConcerto is an AI-assisted code-review studio that turns program structure into music, makes verified bugs audible at their exact source lines, and lets developers hear the composition resolve after a fix.

Product tour · 90-second judge walkthrough · Presenter clickstream · How it works · Run locally · MCP tools

The short version: CodeConcerto transforms source code into a deterministic, multi-track score. Functions lead the melody, loops drive percussion, variables form the bass, and branches move the harmony. OpenAI adds contextual semantic review; verified failures bend the groove into a localized glitch-hop cue. Fix the code and the dissonant layer disappears.

Hackathon pitch deck

Open the 12-slide pitch deck (PDF) · Download the editable PowerPoint / Google Slides source

The deck presents the product tension, code-to-score insight, working review loop, musical Git diff, hybrid OpenAI architecture, MCP surface, product readiness, and roadmap as a concise launch-style story for judges.

Why CodeConcerto?

Code review is visually dense. A developer must scan syntax, control flow, scope, findings, and diffs through the same channel: the screen.

CodeConcerto adds a second channel—sound—without replacing conventional review. Repetition becomes rhythm, architecture becomes phrasing, and a semantic break becomes an unmistakable change in the arrangement at the same moment its source line is highlighted.

This creates a closed feedback loop:

1. Understand the program through a structural score.
2. Notice the exact moment the logic stops resolving.
3. Inspect the synchronized source line and explanation.
4. Fix the issue with a minimal AI-assisted patch.
5. Replay the program and hear the stable version.

90-second judge walkthrough

After starting the app, use this path to see the complete product story:

For the timed narration, proof points, preflight checklist, and 30-second fallback, use the presenter clickstream.

1. Keep the default Async checkout bug example selected.
2. Choose a musical style and click Sonify code.
3. Watch the piano roll and source-line highlight move together.
4. When the groove collapses into the playful glitch-hop stumble, click Pause & inspect.
5. Read the line-specific explanation under Code Review and Fix.
6. Click Resolve with Codex & hear the fix.
7. Compare the repaired score—the glitch track disappears and the music resolves.
8. Switch to Diff Mode to hear additions rise and deletions descend.
9. Export the result as a real multi-track MIDI file.

No API key is required for the included demonstration, structural composition, visualization, Diff Mode, MIDI export, or deterministic JavaScript/TypeScript findings.

Product tour

Compose and inspect Paste real source, choose a musical style, and watch every generated voice move through the synchronized score.

Hear a code review Diff Mode gives additions a bright ascending voice while deletions fall into a darker register.

Hear the bug. Read the cause. Apply the fix. Replay the resolution.

What makes it different

Capability	What CodeConcerto does
Multimodal code review	Adds an audible representation of architecture, repetition, control flow, and failure while preserving the source and written findings.
Hybrid correctness layer	Merges deterministic AST and lexical-scope analysis with OpenAI semantic review instead of trusting either layer alone.
Exact source synchronization	Keeps playback, piano roll, source highlighting, the active finding, pause, and resume on the same timeline.
Explainable bug sound	Uses a low-stress glitch-hop transition—syncopation, microtonal detuning, bitcrushing, and a tape-stop/scratch gesture—rather than an arbitrary alert sound.
Fix-to-resolution loop	Applies a minimal patch, recomposes the source, and lets the developer hear the warning layer disappear.
Musical code review	Turns a real before/after diff into opposing voices: additions ascend while removals descend.
Open tool surface	Exposes the composition and analysis engine through five MCP tools and exports editable MIDI.
Graceful fallback	Remains useful without an OpenAI key and falls back to synthesized Web Audio if sampled playback cannot initialize.

Code-to-score mapping

Program signal	Musical voice	Meaning
Each function	Its own theme	A motif derived from the function's name — so the same function always sounds like itself.
Call sites	Theme quotation	Calling a function quotes its motif, making the call graph audible.
Recursion	Theme an octave up	A self-call restates the motif in a higher register.
Cyclomatic complexity	Dynamics	Denser functions play higher and louder.
Loops and operators	Rhythm	Repeated work becomes pulse and percussion.
Variables and data	Bass	The bass walks the current chord, so data supports the harmony.
Branches and conditionals	Harmony	Decisions move the arrangement to the next chord in the progression.
Comments and structure	Ambient layer	Context and program shape create the musical bed.
Verified semantic breaks	Glitch track	The beat stumbles at the affected source line using detuned, bitcrushed cues.
Added diff lines	Ascending voice	New code moves upward and brightens.
Deleted diff lines	Descending voice	Removed code falls into a darker register.

Harmony follows a real progression per mode — I–V–vi–IV in major, i–VI–III–VII in minor, i–IV–i–VII in dorian — rather than restating one chord under every branch. Melodic notes on strong beats snap to the tones of the chord underneath them.

How it works

1. Deterministic structural analysis

@babel/parser and @babel/traverse build an AST for JavaScript and TypeScript, then two passes run over it:

Program structure (app/lib/analysis.ts) resolves every callable — functions, methods, arrows — with its span, parameters, async-ness, export status, per-function McCabe cyclomatic complexity, nesting depth, and the names it calls. From those calls it builds a call graph, marks recursion, identifies entry points, and scores the file on the standard maintainability index. The composer turns that structure into motifs, sections, and dynamics.

Semantic detectors find what scope resolution alone cannot: floating promises (grouped into one finding per function, carrying every affected line), sequential await inside a loop, unreachable statements, assignment used as a condition, eval and new Function, empty catch blocks, coercive equality that is not the deliberate == null idiom, unused function-scoped bindings, var, and complexity, nesting, and length smells.

Findings are split by consequence. Correctness, security, and performance issues bend the music. Maintainability and style notes are returned as advisories — reported in the studio's code-health panel, deliberately never sonified, so a stray var cannot wreck the song. Scope and pattern findings that carry a concrete patch win any collision with the broader semantic pass, so one root cause is never reported twice.

The browser cannot afford to ship Babel, so app/lib/structure-lite.ts reproduces the same ProgramStructure shape from a single dependency-free scan. It arranges a score the instant you paste code, handles indentation-delimited languages Babel cannot parse at all (Python, Go, Rust, Java), and a test asserts it agrees with the AST pass on shared input.

2. OpenAI semantic review

When OPENAI_API_KEY is available, the server calls the OpenAI Responses API with strict JSON Schema output. The model reviews async control flow, data integrity, correctness, and security, returning line-specific explanations, minimal suggestions, and musical descriptions.

AI findings are merged with deterministic findings. The deterministic layer remains authoritative for scope errors the model may omit, and the entire experience falls back locally when the API is unavailable.

3. Deterministic composition

The composition engine maps language, program structure, nesting, complexity, and findings into notes, tracks, key, scale, tempo, velocity, and duration. Every function receives a motif derived from its name, stated at its declaration and quoted at each call site; the harmony walks a real chord progression; melody snaps to chord tones on strong beats. The same input and style produce the same score, making demonstrations and tests reproducible.

4. Production-minded browser audio

Tone.js performs locally bundled grand-piano and drum samples through genre-specific dynamics, filters, ambience, humanization, and effects. The bug transition deliberately drops expected drum hits and introduces off-grid microtonal synth notes, bitcrushed texture, a tape-stop, and a scratch gesture. A Web Audio synthesis path provides a resilient fallback.

5. Fix, compare, and export

The repaired source is recomposed immediately so the listener hears the semantic warning layer disappear. Diff Mode uses a line-based longest-common-subsequence comparison, and the MIDI writer exports each musical role as an editable track for GarageBand, Ableton, Logic, FL Studio, and other DAWs.

Feature set

• Real-time code-to-music composition with five musical styles: 🎻 Classical, 🎹 Electronic, 🌊 Ambient, 🎷 Jazz, and 🎸 Rock
• Per-function motifs, call-site theme quotation, and audible recursion
• Modal chord progressions with chord-tone snapping instead of a static triad
• AST-based structure: per-function cyclomatic complexity, nesting, call graph, entry points, and a maintainability index
• Twelve semantic detectors beyond scope resolution, split into sonified findings and non-sonified code-health advisories
• Dependency-free structural fallback that also covers Python, Go, Rust, and Java
• OpenAI semantic review with strict structured output
• Exact line-level glitch cues for verified findings
• Sampled piano and drum playback with genre-aware processing
• Syntax-highlighted editor with severity markers in the gutter that jump to the finding
• Scrubbable timeline, click-to-seek on the score, and per-track mute, solo, and master level
• True pause and resume across audio, visualization, source highlighting, and findings
• Keyboard transport — space, ⌘/Ctrl+Enter, S, ←/→, E, M, ?
• Drag-and-drop or open a source file straight into the studio
• One-click fix and automatic resolved playback
• Git-style Diff Mode with distinct addition and deletion voices
• Multi-track MIDI export
• Five stdio MCP tools for editor and agent workflows
• Responsive UI, keyboard focus states, reduced-motion support, and local project-owned illustrations
• Deterministic offline behavior for the core demo

Quick start

Requirements

• Node.js 22.13 or newer
• npm
• A modern browser with Web Audio support

Install and run

git clone https://github.com/Maulik176/CodeConcerto.git
cd CodeConcerto
npm install
npm run dev

Open http://localhost:3000.

Enable OpenAI review

The app works without an API key. To enable the contextual OpenAI reviewer:

cp .env.example .env.local

Then set the following locally:

OPENAI_API_KEY=your_api_key_here

Never commit .env.local; it is already excluded by .gitignore.

Diff Mode

Switch the studio from Compose to Diff Mode, paste the before and after source, and select Sonify diff.

• Added lines become a bright ascending instrument.
• Deleted lines become a darker descending instrument and bass voice.
• Added loops and branches retain their rhythmic and harmonic identities.
• The complete review can be replayed or exported as multi-track MIDI.

MCP integration

Start the stdio MCP server with:

npm run mcp

CodeConcerto exposes five tools:

Tool	Purpose
sonify_code	Create a structured musical score from source code.
analyze_structure	Explain functions, loops, branches, data, complexity, and semantic risks.
compare_code_musically	Compare before and after source as ascending and descending voices.
sonify_git_diff	Read working-tree, staged, or base-branch Git changes and compose the real diff.
export_midi	Write a multi-track .mid file from code or a diff.

A typical stdio client configuration points to npm in this repository:

{
  "command": "npm",
  "args": ["run", "mcp"],
  "cwd": "/absolute/path/to/CodeConcerto"
}

Verify tool discovery, unresolved-reference analysis, diff sonification, and MIDI output with:

npm run test:mcp

Technology

Layer	Technology
Interface	Next.js 16, React 19, TypeScript, GSAP, Phosphor Icons
Deterministic analysis	Babel parser and traversal with lexical-scope resolution
AI review	OpenAI Responses API with strict JSON Schema output
Composition	Custom deterministic score and MIDI engines
Audio	Tone.js sampled playback with Web Audio fallback
Tool protocol	Model Context Protocol SDK with stdio transport
Build and deployment	Vinext, Vite, and Cloudflare-compatible workers

Repository map

Path	Responsibility
app/CodeConcertoStudio.tsx	Interactive studio, transport, mixer, synchronized visualization, and fallback audio.
app/api/analyze/route.ts	OpenAI Responses API integration and deterministic fallback.
app/lib/analysis.ts	Program structure, call graph, complexity, and the semantic detectors.
app/lib/static-analysis.ts	Lexical-scope resolution and the merge that turns every pass into one review.
app/lib/structure-lite.ts	Dependency-free structural scan for the browser and for non-JS languages.
app/lib/maintainability.ts	The maintainability index, shared by both structural passes.
app/lib/language.ts	Language detection and each language's tonal signature.
app/lib/music.ts	Motifs, progressions, sections, diff composition, glitch events, and MIDI generation.
app/lib/realistic-audio.ts	Sampled Tone.js performance, humanization, seek, mixing, and effect routing.
app/lib/highlight.ts	The editor's dependency-free syntax highlighter.
mcp/server.mjs	Five MCP tools exposed over stdio.
tests/analysis.test.mjs	Structure, detectors, advisory split, and highlighter tests.
tests/composition.test.mjs	Motif, progression, section, mixer, and MIDI tests.
tests/rendered-html.test.mjs	Server rendering, asset, and end-to-end regression tests.

Validation

npm run lint
npm run test:unit   # analysis + composition; no build required
npm test            # builds, then runs the whole suite including server rendering
npm run test:mcp

test:unit imports the libraries directly and runs in about a second, so it is the fast loop while working on analysis or the composer. npm test builds the worker bundle first and adds the server-rendering suite on top.

The suite covers program structure and its agreement with the dependency-free fallback, every semantic detector, the sonified-versus-advisory split, maintainability scoring, malformed input, highlighter escaping, motif stability, chord progressions, section timing, pitch validity across all five styles, mute and solo semantics, MIDI file structure, server rendering, optimized local illustration assets, lexical scope, repeated unresolved identifiers, synchronized glitch composition, humanization, Diff Mode behavior, and MCP tool discovery.

Current boundaries

• JavaScript and TypeScript receive the deepest AST and lexical-scope analysis. Other recognized languages can still be composed through structural token mapping.
• OpenAI semantic review requires an API key; deterministic analysis and the complete included demo do not.
• CodeConcerto performs static review—it does not execute pasted source code.
• Browser playback requires a user gesture because of standard Web Audio autoplay protections.

What comes next

• Language-native parsers for Python, Rust, Go, and Java
• Pull-request annotations with shareable musical review links
• Collaborative playback sessions for pair and accessibility-focused review
• MPE-aware MIDI export for preserving microtonal glitch gestures
• Custom mapping profiles for teams, codebases, and learning environments

Ending Notes

CodeConcerto brings a new React interface, sampled and dynamically mixed browser audio, hybrid deterministic and OpenAI semantic review, exact source-line risk localization, fix-to-resolution playback, musical Git diffs, language key signatures, five MCP tools, responsive accessibility work, and deployment support.

See THIRD_PARTY_NOTICES.md for required license notices and audio attributions.