An open research handoff for affordable PICO Swift body tracking on Steam Frame.
Research prototype · Community continuation welcome · Full body tracking is not working yet
Start here · Current status · Research map · Continue the work · Credits
We have already communicated with real PICO trackers through Steam Frame's onboard radio, received real motion data and processed the headset's camera streams. The next challenge is turning those pieces into reliable body tracking.
Some measured milestones from the research:
| Achievement | Recorded result |
|---|---|
| Service radio replies | 99.30% retained replies over 150 seconds, with 7,187 validated replies from 7,238 completed transmissions. A stable 118 second window reached 99.98%. |
| Game radio transport | One historical run reached 99.53% CRC valid replies while receiving 6,756 unique IMU samples. |
| Real motion data | A later run received 9,027 accepted IMU samples. Gaps remained too large for continuous inertial fusion. |
| Headset camera access | Captured and processed real streams from the headset's upper cameras. |
| Camera timing | 1,618 measured frames, with a median frame age of 66.8 ms and a maximum of 85.1 ms in the recorded camera timing test. |
| Remote mode control | A bounded 150 second Game test verified fresh replies showing the requested tracker mode actually applied. |
These results come from different experiments and configurations. They are milestones, not a claim that everything works together. See the current status for the conditions and remaining work.
The goal is affordable body tracking on Steam Frame using inexpensive PICO trackers and the radio already inside the headset. More people should be able to bring their whole body into VR without an expensive tracking setup. If that is something you want to help build, there is real work here to continue.
You do not have to start from zero. This repository includes 136 source files, 24 firmware diffs, an English research history and a continuation guide, including the failed approaches and the questions still open.
PICO Swift trackers are relatively inexpensive. I wanted to make them useful for body tracking on Steam Frame through its onboard Roy radio, without an external receiver or a PICO headset acting as a live bridge. The eventual target is waist and both legs, together with the normal controllers, for applications such as VRChat.
This repository brings together the implementation, the research history, negative results, exact firmware changes and instructions that another developer needs to pick up the work. It is a starting point for developers, not something you can install today to get working full body tracking.
Publication snapshot: October 15, 2026. These are saved results, not a new check of the devices on publication day.
| Area | What the evidence supports |
|---|---|
| Frame app and diagnostics | Native dashboard, Game and Service choices, diagnostic API and guarded session control are implemented. |
| Onboard Roy radio | A bounded 150 second test confirmed fresh Game mode replies from one tracker. The test ended with all four recorded restoration checks passing. |
| Tracker firmware work | The protected diagnostic10 candidate has an exact stock reference. This repository distributes changes as diffs, never a stock or patched image. |
| Roy RAM modules | Game and service source variants are included. They use temporary SRAM and depend on a specific stock Roy build. Their validation status differs. |
| IMU and optical research | Earlier pilots received real IMU data and published sparse optical measurements under their own conditions. That is useful evidence, but it is not stable current tracking. |
| Latest passive optical probe | 900 stereo pairs produced zero accepted fits. Too few consistent markers were detected in the short exposure data. |
| Full body tracking | Not achieved. Stable current optical identity, calibrated optical and IMU fusion, three tracker roles and reliable controller coexistence remain open. |
There is no honest completion percentage. Several essential steps still need measured proof. This is beyond a protocol sketch, but it is still several major milestones away from a usable full body tracking system. See the detailed status and milestones.
Game and Service: Service supports tracker maintenance and OTA with the controllers powered off, reducing competition for the shared Roy radio. Game remotely requests mode1 on compatible custom tracker firmware and checks the actual applied state. Mode1 changes operating cadence; it does not by itself prove LED and camera phase synchronization or working controller coexistence. Exact mode contracts.
-
Read QUICK_SETUP.md. It separates what is included from the source, assets and configuration you still need.
-
Read CURRENT_STATE.md, RESEARCH_TREE.md and SUPERSESSION.md. Later summaries do not erase earlier experiments.
-
Check the package offline with Python 3.12 or newer:
python tools/verify_package.py python tools/query_catalog.py knowledge --contains optics --limit 20 -
Read the source supplement guide and copy
source/into a separate development checkout. Obtain dependencies and your own calibration as described in ACQUISITION.md. -
Pick one blocker from CONTRIBUTION_TASKS.md. Keep your observations and limitations in the same change.
If you are using an AI assistant, start with AI_START_PROMPT.md and AGENTS.md. Use the catalogs to retrieve relevant history instead of guessing from one summary.
| Path | Contents |
|---|---|
source/pc/frame_app/ |
Our native dashboard, project authored OpenVR/EGL overlay and device monitors. |
source/pc/radio/ |
Protocol, diagnostics, session control and tracker firmware tooling. |
source/pc/radio/ram/ |
Roy game and service modules, loader, runtime and recovery logic. |
source/historical-original210/ |
The protected historical Roy source, separate from newer copies. |
source/pc/optical/ |
Camera timing, marker association, geometry, calibration interfaces and fusion research. |
source/pc/driver/ |
Pose transport and native SteamVR driver source. |
docs/ |
Current evidence, interfaces, acquisition instructions and continuation plans. |
research/ |
English history ledger and source cards covering 768 discovered Markdown versions. |
catalog/ |
Searchable source, asset, document and research graph metadata. |
firmware-diffs/ |
24 exact stock relative differences and their provenance. |
tools/ |
Offline package checks, catalog queries and diff verification. |
verification/ |
Hash manifests, review findings, exclusions and validation limits. |
There are 136 selected source files. Private calibration values were removed, personal device identifiers were replaced, and three legacy C candidates deliberately refuse compilation until their identity configuration is reviewed. Historical tests with unreviewed recorded fixtures are listed exclusions. This is not a fully reproducible build environment or a turnkey release.
No PICO or Valve firmware images, SDKs, native vendor libraries, decompilations, private calibration, raw captures or credentials are included. You obtain permitted copies of the necessary external inputs yourself. Firmware diffs require the exact matching stock image. Instructions and old experiments do not authorize a new hardware operation.
I cannot continue the work right now. My health has become much worse, and I currently do not have access to the medication I need.
I am sharing what I have so the research does not disappear with my ability to work on it. I hope the community can take it further and make this technology useful to people who cannot afford an expensive tracking setup. These trackers are cheap enough to make that worth trying.
If you continue it, please keep the history and credit, be honest about what works, and leave the next person better notes than you started with. I may not be able to reply quickly or provide ongoing support.
Thanks to Hinata Reira for encouragement and the packet reference pico_tracker_packets_en.md. It helped with cross checking IMU packet layout, conversion scales and timing behavior. This was a focused reference contribution; the Frame implementation and its experimental results are separate work. Specific credit and evidence boundaries.
Project authored code and original documentation are released under GNU GPL version 3 or later, with scope and exceptions in NOTICE.md. Keep copyright and license notices, credit existing contributors and identify your changes. The GPL preserves those notices and keeps distributed derivative code under its terms. It cannot physically prevent someone from misrepresenting authorship.
The avatar screenshot and third party material are outside that license grant. Product names identify compatibility targets and imply no endorsement. The banner is an AI generated illustration, not a photograph or evidence of a completed device.