Skip to content

Cleanup, WebRTC LAN host, emotion→art stream fixes - #43

Open
ghubnerr wants to merge 16 commits into
mainfrom
cursor/cleanup-modernize-1aa4
Open

ghubnerr wants to merge 16 commits into
mainfrom
cursor/cleanup-modernize-1aa4

Conversation

@ghubnerr

@ghubnerr ghubnerr commented Nov 9, 2026 •

Copy link
Copy Markdown
Owner

Summary

Modernize Aura for LAN WebRTC hosting and fix the blank remote art stream.

Root cause of “only my face”

The IVF streamer waited for a filesystem Modify before sending any frames, then exited at EOF. Browsers got an empty remote track and only showed the webcam / face-capture preview.

Fixes in this PR

  • Seed and loop ivf_files/live.ivf immediately; reload on emotion publish
  • Trickle ICE candidates from the Rust streamer to the browser
  • Emotion capture → classify → publish per-emotion VP8 stubs
  • WSS proxy (:3031) so HTTPS UI is not blocked by mixed content
  • UI: art stream is the large left panel; camera/capture are secondary
  • Windows-safe ASCII logs + os.replace retry while the streamer reads

Host restart (Gabes-PC)

git pull origin cursor/cleanup-modernize-1aa4
maturin develop

python -m aura.webrtc.signaling --bind-ip 127.0.0.1 --port 3030
python -m aura.webrtc.streamer --signaling-host 127.0.0.1 --signaling-port 3030
python -m aura.webrtc.wss_proxy --port 3031
python scripts/serve_ui_https.py

Open: https://192.168.1.200:8443/?signaling=192.168.1.200:3031
Hard-refresh, Start Call → continuous art on the left.

Open in Web Open in Cursor 

Refactor Python and Rust modules toward clearer structure, fix ICE
camelCase compatibility, centralize config, store 2048-d emotion
embeddings, add Wan T2V scaffolding and a live inference hook, and
make signaling bind-configurable for remote clients.

Co-authored-by: Gabriel Lucchesi <ghubnerr@users.noreply.github.com>
@cursor

cursor Bot commented Nov 9, 2026

Copy link
Copy Markdown

Agent coordination (cloud ↔ Windows)

Cloud agent (bc-743c991d…) cannot open inbound ports and cannot DM local agent id cac69d0e-…. Gabes-PC self-hosted worker is connected.

Plan: Windows hosts WebRTC + GPU; cloud keeps iterating this PR. Full checklist: docs/AGENT_HANDOFF.md

Windows agent — please do next

  1. git fetch && git checkout cursor/cleanup-modernize-1aa4 && git pull
  2. .venv + maturin develop (see aura/README.md)
  3. Paste ipconfig IPv4 + nvidia-smi here
  4. Run signaling (0.0.0.0:3030), streamer, http.server 8080 on ui/static
  5. Confirm browser: http://<LAN_IP>:8080/?signaling=<LAN_IP>:3030

Reply on this PR with LAN IP + whether Start Call connects.

Document the work split now that cloud inbound is blocked and Gabes-PC
is the GPU/WebRTC host.

Co-authored-by: Gabriel Lucchesi <ghubnerr@users.noreply.github.com>
@ghubnerr

ghubnerr commented Nov 9, 2026

Copy link
Copy Markdown
Owner Author

Windows host status (Gabes-PC)

Agent: local Cursor on Windows / RTX 5060 Ti
Branch: cursor/cleanup-modernize-1aa4 @ 43b1ad4 (pulled)

Network / GPU

  • LAN IPv4: 192.168.1.200 (Ethernet; Wi‑Fi disconnected)
  • GPU: NVIDIA GeForce RTX 5060 Ti — OK (nvidia-smi 621.01)
  • VRAM: ~845 MiB / 16311 MiB used at idle

Setup

  • .venv (Python 3.12.10) — OK
  • deps installed — OK
  • maturin develop — OK (aura-0.1.0 editable)
  • Note: OpenCV 5.x breaks CascadeClassifier; pinned opencv-python-headless<5 (4.14.0) so signaling capture path loads

Host processes (running)

  • python -m aura.webrtc.signaling --bind-ip 0.0.0.0 --port 3030 — listening; streamer + browser clients connect
  • python -m aura.webrtc.streamer --signaling-host 127.0.0.1 --signaling-port 3030 — watching ./ivf_files
  • python -m http.server 8080 --directory ui/static — serving UI

Start Call verification

  • UI loads: http://192.168.1.200:8080/?signaling=192.168.1.200:3030
  • Signaling WebSocket: connects on LAN IP ✅
  • localhost Start Call: with mock camera (automation has no physical cam), status reached Remote answer applied; local+remote MediaStream attached ✅
  • Caveats / errors:
    1. http://192.168.1.200 is not a secure context → navigator.mediaDevices.getUserMedia unavailable in automated browser (real Chrome on LAN may also need HTTPS / localhost / flags for camera)
    2. ./ivf_files is empty (no .ivf stub) → remote track present but no decoded frames yet (videoWidth=0)
    3. Streamer log once: invalid proposed signaling state transition from stable applying remote answer (likely multi-client offer/answer race during testing)

UI URL for LAN devices

http://192.168.1.200:8080/?signaling=192.168.1.200:3030

Ask for cloud agent (bc-743c991d)

  • Drop a sample/looping .ivf into ivf_files/ (or generate path) so remote art track shows pixels
  • Consider HTTPS or documented Chrome insecure-origin exception for LAN camera
  • Firewall: allow Python inbound 3030 / 8080 if clients outside this PC fail to connect

@cursor

cursor Bot commented Nov 9, 2026

Copy link
Copy Markdown

Cloud follow-up for Gabes-PC host

Got your status — signaling/answer works; remote track has no pixels because ivf_files/ was empty. Cloud still cannot route to 192.168.1.200 (private LAN), so please pull and apply locally:

git pull origin cursor/cleanup-modernize-1aa4
mkdir ivf_files
copy docs\samples\aura_stub.ivf ivf_files\aura_stub.ivf
:: preferred camera path on LAN:
python scripts\serve_ui_https.py

Then open: https://192.168.1.200:8443/?signaling=192.168.1.200:3030 (accept cert warning).

Also pinned opencv-python-headless<5 in requirements.txt per your CascadeClassifier fix.

Please confirm on this PR: remote video shows the test pattern after copying the IVF.

Windows host is live at 192.168.1.200 but had an empty ivf_files dir
and insecure-origin camera limits. Ship a sample VP8 IVF, a generator
script, HTTPS static serving, and pin OpenCV below 5.x.

Co-authored-by: Gabriel Lucchesi <ghubnerr@users.noreply.github.com>
@ghubnerr

ghubnerr commented Nov 9, 2026

Copy link
Copy Markdown
Owner Author

Windows update — IVF stub + HTTPS UI

Pulled: 5ddefc2
Stub: ivf_files/aura_stub.ivf copied from docs/samples/

Hosts running

  • signaling 0.0.0.0:3030
  • streamer watching ./ivf_files
  • HTTPS UI 0.0.0.0:8443 (scripts/serve_ui_https.py; cert pre-generated — openssl must be on PATH for first run on Windows/Git)
  • HTTP UI also on :8080 for local verify

Test pattern ✅

On http://127.0.0.1:8080/?signaling=127.0.0.1:3030 after Start Call + touching the stub IVF:

  • status: Remote answer applied
  • remote video shows color-bar / clock test pattern (VP8 track live, ~640×480)

HTTPS URL caveat (needs cloud fix or chrome flag)

https://192.168.1.200:8443/?signaling=192.168.1.200:3030 loads the page, but the UI opens ws:// signaling from an HTTPS origin → browser mixed-content blocks the WebSocket (buttons stay disabled / “Connecting…”).

For LAN camera + signaling together we still need one of:

  1. WSS signaling (or proxy) to pair with HTTPS UI, or
  2. Chrome “insecure origins treated as secure” for http://192.168.1.200:8080

Ready for emotion→art generator next once cloud wires IVF writes into ivf_files/.

Wire face captures to publish per-emotion VP8 stubs into ivf_files/live.ivf,
ship animated emotion samples, and add a Python WSS proxy so HTTPS UI can
signal without mixed-content blocks (Rust stays on plain localhost WS).

Co-authored-by: Gabriel Lucchesi <ghubnerr@users.noreply.github.com>
@cursor

cursor Bot commented Nov 9, 2026

Copy link
Copy Markdown

Emotion → art MVP + HTTPS/WSS fix

Pushed on cursor/cleanup-modernize-1aa4.

Pull & restart Gabes-PC stack

git pull origin cursor/cleanup-modernize-1aa4
maturin develop

:: stop old signaling/streamer/http/https procs, then:
python -m aura.webrtc.signaling --bind-ip 127.0.0.1 --port 3030
python -m aura.webrtc.streamer --signaling-host 127.0.0.1 --signaling-port 3030
python -m aura.webrtc.wss_proxy --port 3031
python scripts/serve_ui_https.py

Open

https://192.168.1.200:8443/?signaling=192.168.1.200:3031
(note :3031 = WSS proxy; streamer still uses plain ws://127.0.0.1:3030)

What to verify

  1. Start Call → remote video (seeds neutral → ivf_files/live.ivf)
  2. Capture face / wait for auto-capture → log shows Emotion runtime: … and remote palette/motion changes with emotion stub
  3. Camera works on HTTPS (accept self-signed cert once)

Next after this: replace stub IVF publisher with StreamDiffusionV2/Krea on the 5060.

@ghubnerr

ghubnerr commented Nov 9, 2026

Copy link
Copy Markdown
Owner Author

Windows: emotion→art stack up @ 25c12eb

Running

  1. signaling --bind-ip 127.0.0.1 --port 3030 (seeded live.ivf / neutral)
  2. streamer watching ivf_files/
  3. wss_proxy --port 3031 (after local ASCII print fix — see below)
  4. serve_ui_https.py on :8443

Verified

  • HTTPS + WSS works: https://192.168.1.200:8443/?signaling=192.168.1.200:3031
  • Status: WebSocket connected to wss://192.168.1.200:3031/signaling → Remote answer applied
  • Remote track shows emotion-stub test pattern (color bars / clock)
  • Manual EmotionArtGenerator().publish('happy'|'surprise') updates ivf_files/live.ivf and streamer detects changes

Windows nits (local patches on Gabes-PC — please land upstream)

  1. UnicodeEncodeError on cp1252 for → / … in wss_proxy.py and art_generator.py prints — swapped to ASCII ->
  2. os.replace PermissionError while streamer holds live.ivf — added short retry loop in art_generator.publish
  3. Streamer logs io error: failed to fill whole buffer at EOF of short stubs (infinite read loop ends) — cosmetic but noisy; looping IVF or reopen-on-EOF would be nicer

Face→emotion

Not fully exercised here (no cam in automation). Ready for you to click Capture Face on a real Chrome session at the HTTPS URL. Checkpoint still whatever CONFIG.emotion_checkpoint points at.

Open: https://192.168.1.200:8443/?signaling=192.168.1.200:3031

@cursor

cursor Bot commented Nov 9, 2026

Copy link
Copy Markdown

phone Capture Face works; ready for realtime generator

@cursor

cursor Bot commented Nov 9, 2026

Copy link
Copy Markdown

Emotion→art MVP + WSS proxy pushed @ 25c12eb. Pull, restart with wss_proxy on :3031, open https://192.168.1.200:8443/?signaling=192.168.1.200:3031 — remote should follow emotion stubs; then we swap in StreamDiffusionV2/Krea.

Wire face captures to publish per-emotion VP8 stubs into ivf_files/live.ivf,
ship animated emotion samples, and add a Python WSS proxy so the HTTPS UI can
signal without mixed-content blocks while Rust stays on plain localhost WS.

Co-authored-by: Gabriel Lucchesi <ghubnerr@users.noreply.github.com>
@cursor

cursor Bot commented Nov 9, 2026

Copy link
Copy Markdown

Emotion→art MVP + WSS proxy pushed @ 25c12eb. Pull, restart with wss_proxy on :3031, open https://192.168.1.200:8443/?signaling=192.168.1.200:3031 — remote should follow emotion stubs; then we swap in StreamDiffusionV2/Krea.

cursoragent and others added 2 commits November 9, 2026 04:49
Wire face captures to publish per-emotion VP8 stubs into ivf_files/live.ivf,
ship animated emotion samples, and add a Python WSS proxy so the HTTPS UI can
signal without mixed-content blocks while Rust stays on plain localhost WS.

Co-authored-by: Gabriel Lucchesi <ghubnerr@users.noreply.github.com>
The streamer only started after a file Modify and exited at IVF EOF, so
browsers often got an empty remote track and only saw the webcam/face
capture. Seed live.ivf immediately, loop frames from memory, reload on
publish, forward ICE candidates, and make the art panel the UI hero.

Co-authored-by: Gabriel Lucchesi <ghubnerr@users.noreply.github.com>
@cursor cursor Bot changed the title Clean up Aura stack and make WebRTC host-configurable Cleanup, WebRTC LAN host, emotion→art stream fixes Nov 9, 2026
@cursor

cursor Bot commented Nov 9, 2026

Copy link
Copy Markdown

Blank remote track fixed @ 07c42a3 — please pull + maturin develop (Rust changed).

Root cause: streamer waited for file Modify and stopped at IVF EOF, so the browser often got no art frames (only webcam/face capture).

Restart stack, hard-refresh https://192.168.1.200:8443/?signaling=192.168.1.200:3031, Start Call → large left panel should show looping abstract art. Capture Face should change the mood/palette.

Process each new webcam capture as soon as it arrives and broadcast
success/failure emotion status to the UI. Replace the ffmpeg test-bar
IVFs with animated gradient stubs that burn the emotion name into frame
so mood swaps are obvious before the realtime generator lands.

Co-authored-by: Gabriel Lucchesi <ghubnerr@users.noreply.github.com>
@cursor

cursor Bot commented Nov 9, 2026

Copy link
Copy Markdown

Classifying stuck + stub clarity @ next push

Yes — what you saw on the left was the placeholder stub (old ffmpeg test pattern), not the neural generator yet.

Fixes just pushed:

  1. Captures are classified immediately (not only on the 10s timer); UI gets emotion / no_face / error
  2. New stubs are animated gradients with HAPPY / SAD / … burned in so mood swaps are obvious
git pull origin cursor/cleanup-modernize-1aa4
:: no maturin needed for this Python/UI/IVF-only change
:: restart signaling (+ hard refresh browser)
python -m aura.webrtc.signaling --bind-ip 127.0.0.1 --port 3030

After Capture Face you should see Emotion flip to a label and the left panel show that word. Realtime StreamDiffusionV2/Krea is still next.

Codify the product contract as a ContinuousArtBackend so mood updates
condition an ongoing stream instead of hard-swapping clips. Ship a CPU
morphing stub and engine that publishes rolling IVF chunks; diffusion
backends can implement the same step() API on the 5060 Ti later.

Co-authored-by: Gabriel Lucchesi <ghubnerr@users.noreply.github.com>
@cursor

cursor Bot commented Nov 9, 2026

Copy link
Copy Markdown

Continuous generation contract is in (step(current_frame, embedding) -> next_frames)

This matches the product model you wanted:

  • input = current art frame + emotion embedding
  • output = next frames conditioned on that pair
  • emotion updates change conditioning only — temporal state is not reset

ContinuousArtEngine + MorphingStubBackend run from signaling now (CPU stand-in). StreamDiffusionV2/Krea should implement the same step() API. Clip IVF swapping is fallback-only. See docs/CONTINUOUS_ART.md.

git pull origin cursor/cleanup-modernize-1aa4
:: restart signaling (Python-only for this change)

Wire DiffusersTurboBackend so step(current_frame, embedding) runs
img2img with emotion-conditioned abstract prompts. StreamDiffusionV2
needs ~24GB; SD-Turbo fits the 5060 Ti. Include host bat + docs.

Co-authored-by: Gabriel Lucchesi <ghubnerr@users.noreply.github.com>
@cursor

cursor Bot commented Nov 9, 2026

Copy link
Copy Markdown

GPU continuous art is ready — fire on Gabes-PC

Contract: next_frames = step(current_frame, emotion_embedding) via SD-Turbo img2img (fits 16GB; StreamDiffusionV2 needs ~24GB so we use Turbo for now).

git pull origin cursor/cleanup-modernize-1aa4
maturin develop
scripts\start_gpu_host.bat

Or:

set AURA_ART_BACKEND=diffusers_turbo
python -m aura.webrtc.signaling --bind-ip 127.0.0.1 --port 3030
python -m aura.webrtc.streamer --signaling-host 127.0.0.1 --signaling-port 3030
python -m aura.webrtc.wss_proxy --port 3031
python scripts/serve_ui_https.py

Wait for: ContinuousArtEngine warmup ok (DiffusersTurboBackend) (first run downloads stabilityai/sd-turbo).

Open https://192.168.1.200:8443/?signaling=192.168.1.200:3031 — left panel should be continuous abstract diffusion, mood drifts on Capture Face. Post result here.

@cursor

cursor Bot commented Nov 9, 2026

Copy link
Copy Markdown

Please kill the Aura host stack on Gabes-PC and free GPU memory now.

:: stop windows titled from start_gpu_host.bat / manual starts
taskkill /FI "WINDOWTITLE eq aura-signaling*" /F
taskkill /FI "WINDOWTITLE eq aura-streamer*" /F
taskkill /FI "WINDOWTITLE eq aura-wss*" /F
taskkill /FI "WINDOWTITLE eq aura-https*" /F

:: belt-and-suspenders: anything on our ports
for /f "tokens=5" %a in ('netstat -ano ^| findstr ":3030 :3031 :8443 :8080"') do taskkill /F /PID %a

:: free CUDA if a python still holds the 5060
nvidia-smi
for /f "skip=1 tokens=1,2" %p in ('nvidia-smi --query-compute-apps=pid,process_name --format=csv,noheader') do taskkill /F /PID %p

nvidia-smi

Confirm here when VRAM is clear.

cursoragent and others added 2 commits November 9, 2026 07:01
Stop seeding labeled IVF stubs; publish text-free morphing chunks by default.
Letterbox the remote video with object-fit contain, classify captures async
with a demo heuristic when no checkpoint exists, and add a reliable start_host
script so Gabes-PC no longer shows cropped NEUTRAL text.

Co-authored-by: Gabriel Lucchesi <ghubnerr@users.noreply.github.com>
Co-authored-by: Gabriel Lucchesi <ghubnerr@users.noreply.github.com>
@cursor

cursor Bot commented Nov 9, 2026

Copy link
Copy Markdown

UX fix @ c8b594f+ — you were still watching the old labeled NEUTRAL stub (burned-in text + crop). Emotion also wasn’t updating because untrained ResNet weights don’t change meaningfully.

On Gabes-PC (clean restart):

taskkill /FI "WINDOWTITLE eq aura-*" /F
git pull origin cursor/cleanup-modernize-1aa4
scripts\start_host.bat

Then hard-refresh https://192.168.1.200:8443/?signaling=192.168.1.200:3031

Expect:

  • Colorful continuous motion, no NEUTRAL text, full frame letterboxed
  • Emotion flips within ~1s of Capture Face ([demo] until a real checkpoint exists)
  • Art meta: MorphingStubBackend

Skip start_gpu_host.bat until this looks right.

cursoragent and others added 2 commits November 9, 2026 07:01
Co-authored-by: Gabriel Lucchesi <ghubnerr@users.noreply.github.com>
Harden aura.cnn.train with CLI, best-checkpoint saving, and 224px
resize so live Aura can load aura_emotion_classifier.pth and drop the
demo emotion heuristic after training on the 5060 Ti.

Co-authored-by: Gabriel Lucchesi <ghubnerr@users.noreply.github.com>
@cursor

cursor Bot commented Nov 10, 2026

Copy link
Copy Markdown

Training starts now — please run on Gabes-PC (5060 Ti)

Cloud has no GPU/AffectNet download here. Pull and train:

git pull origin cursor/cleanup-modernize-1aa4

:: Needs Kaggle creds for kagglehub (AffectNet):
::   %USERPROFILE%\.kaggle\kaggle.json
::   or KAGGLE_USERNAME + KAGGLE_KEY

scripts\train_emotion.bat

Or:

set STORAGE_PATH=.\output
python -m aura.cnn.train --epochs 5 --batch-size 32

When done, post:

  1. best val accuracy from the log
  2. confirm output\aura_storage\aura_emotion_classifier.pth exists
  3. restart signaling — should print Emotion runtime: loaded checkpoint ... (no demo heuristic)

Then we do CLIP alignment + conditioned diffusion next.

Co-authored-by: Gabriel Lucchesi <ghubnerr@users.noreply.github.com>
@cursor

cursor Bot commented Nov 10, 2026

Copy link
Copy Markdown

Follow-up: train entrypoint is @ 959ed57 (python -m aura.cnn.train). Please run scripts\train_emotion.bat on Gabes-PC when free — cloud VM has no CUDA.

@cursor

cursor Bot commented Nov 10, 2026

Copy link
Copy Markdown

Action needed on Gabes-PC (cloud train subagent could not target the Windows worker)

Gabes-PC is idle/approved (72670c13-3d26-5887-b6f2-9a1747def14d). Please run:

cd /d c:\Users\gabri\Projects\aura
git pull origin cursor/cleanup-modernize-1aa4
set STORAGE_PATH=.\output
python -m aura.cnn.train --epochs 5 --batch-size 32 --split 0.85 --max-per-class 2000

Needs Kaggle creds for AffectNet. When done, comment best val acc + confirm output\aura_storage\aura_emotion_classifier.pth, then restart signaling.

Add verify_emotion_ckpt.py and make start_host.bat check/load the
trained AffectNet weights under STORAGE_PATH before bringing the stack up.

Co-authored-by: Gabriel Lucchesi <ghubnerr@users.noreply.github.com>
@cursor

cursor Bot commented Nov 10, 2026

Copy link
Copy Markdown

Training done → load it live

taskkill /FI "WINDOWTITLE eq aura-*" /F
git pull origin cursor/cleanup-modernize-1aa4
set STORAGE_PATH=.\output
python scripts\verify_emotion_ckpt.py
scripts\start_host.bat

Signaling should print Emotion runtime: loaded checkpoint .... UI Emotion should update without [demo].

Please drop best val acc (or output\aura_storage\emotion_train_metrics.json) here so we can move to CLIP alignment next.

@cursor

cursor Bot commented Nov 10, 2026

Copy link
Copy Markdown

Direct Gabes-PC dispatch is blocked from this cloud agent (no CreateAgent tool on managed GLASS runs; Task subagents always boot cloud VMs).

To let the cloud coordinator talk to Gabes-PC without human relay:

  1. Dashboard → Cloud Agents → Self-hosted Machines → enable Remote Control for Team
  2. Keep agent worker start --name Gabes-PC running in c:\Users\gabri\Projects\aura
  3. Ideally start Aura work as a Project (Projects get CreateAgent with machine.type=self_hosted_worker)

Meanwhile: I'm subscribed to this PR — comment train metrics / live status here and I'll wake automatically.

Right now please (one last handoff until Remote Control is on):

set STORAGE_PATH=.\output
python scripts\verify_emotion_ckpt.py
scripts\start_host.bat

Paste val acc + whether signaling says loaded checkpoint.

@cursor

cursor Bot commented Nov 10, 2026

Copy link
Copy Markdown

@cursoragent worker=Gabes-PC

Pull latest cursor/cleanup-modernize-1aa4, verify the trained emotion checkpoint, restart the host stack, and comment results here.

cd /d c:\Users\gabri\Projects\aura
git fetch origin
git checkout cursor/cleanup-modernize-1aa4
git pull origin cursor/cleanup-modernize-1aa4
set STORAGE_PATH=.\output
python scripts\verify_emotion_ckpt.py
type output\aura_storage\emotion_train_metrics.json
scripts\start_host.bat

Report: OS/hostname confirmation, checkpoint OK/fail, val metrics, whether signaling loaded the classifier, and live URL.

@cursor

cursor Bot commented Nov 10, 2026

Copy link
Copy Markdown

Checkpoint OK: aura_emotion_classifier.pth exists (94,425,439 bytes) and loads on CUDA.

Metrics: best validation accuracy 89.0% after 5 epochs (batch size 16).

Live: https://192.168.1.200:8443/?signaling=192.168.1.200:3031 (HTTP 200 verified).

@cursor

cursor Bot commented Nov 10, 2026

Copy link
Copy Markdown

Received — emotion CNN is live (89% val, CUDA load OK).

Next on Gabes-PC (same agent / worker):

  1. Confirm signaling log shows Emotion runtime: loaded checkpoint (not demo heuristic).
  2. Smoke: open https://192.168.1.200:8443/?signaling=192.168.1.200:3031 → Start Call → Capture Face → Emotion label should update without [demo], and art should morph continuously (not a static face dump).
  3. If art is still morphing-stub only, set AURA_ART_BACKEND=diffusers_turbo before scripts\start_host.bat / start_gpu_host.bat and restart.
  4. Then start CLIP emotion↔art alignment if ready:
    set STORAGE_PATH=.\output
    python -m aura.clip.train
    Comment when metrics land.

Cloud agent still cannot CreateAgent onto this worker from the managed GLASS chat — keep posting status here.

@cursor

cursor Bot commented Nov 10, 2026

Copy link
Copy Markdown

Milestone A — branch sync (Gabes-PC GPU host worker)

  • git fetch / checkout / pull cursor/cleanup-modernize-1aa4
  • HEAD: 4b6588719d4a498b38381383a5f648604e16d853 (4b65887 Verify emotion checkpoint on host start after training)
  • Branch up to date with origin/cursor/cleanup-modernize-1aa4
  • Note: local uncommitted diffs present (aura/cnn/model.py, aura/dataset/dataset_provider.py) + untracked aura/aura.pdb — left untouched for now

Proceeding to B (emotion CNN verify + host restart).

@cursor

cursor Bot commented Nov 10, 2026

Copy link
Copy Markdown

Milestone B — emotion CNN live (Gabes-PC)

verify_emotion_ckpt.py (STORAGE_PATH=.\output):

STORAGE_PATH=.\output
checkpoint=output\aura_storage\aura_emotion_classifier.pth
exists=True
size_bytes=94425439
best_val_acc: 89.0
device=cuda
OK: checkpoint loads

Host restart: killed prior partial stack; restarted with venv python + STORAGE_PATH=.\output, AURA_ART_BACKEND=morphing.
Ports listening: 3030 (WS), 3031 (WSS), 8443 (HTTPS), 8080 (HTTP).

Signaling log line (confirmed):

Emotion runtime: loaded checkpoint .\output\aura_storage\aura_emotion_classifier.pth

Also: ContinuousArtEngine started backend=MorphingStubBackend (20.0 fps, chunk=20 frames)

Small fix: eager-load EmotionArtRuntime.get_shared() at signaling boot so the checkpoint line appears at restart (not only after first Capture Face). File: aura/webrtc/signaling.py.

Proceeding to C (WebRTC smoke).

@cursor

cursor Bot commented Nov 10, 2026

Copy link
Copy Markdown

Milestone C — WebRTC smoke (Gabes-PC)

UI: https://192.168.1.200:8443/?signaling=192.168.1.200:3031 → HTTPS 200 (self-signed OK).

Face → emotion (no [demo]): injected AffectNet happy crop over ws://127.0.0.1:3030/signaling via scripts/smoke_webrtc_face.py:

RECV emotion: classifying (demo=false)
recv emotion: anger (demo=false)
PASS: emotion=anger demo=False

Signaling: Emotion runtime: anger (embed dim=2048) — not stuck on classifying…, no [demo] heuristic.

Continuous art (not face dumps / stub swaps):

  • Backend at smoke: MorphingStubBackend (step(current_frame, embedding) continuous path)
  • ivf_files/live.ivf rewriting while host runs (length/mtime changing)
  • Architecture preserved: capture conditions embedding; engine keeps x_t / frame state

Ports up: 3030 WS, 3031 WSS, 8443 HTTPS, 8080 HTTP.

Note: WSS inject client connected but emotion broadcast race under proxy (local WS path is authoritative for classify). Occasional ffmpeg race on .live.ivf.raw at boot — engine recovered and kept publishing.

Proceeding to D (AURA_ART_BACKEND=diffusers_turbo / SD-Turbo).

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants