Thomas Fransolet 0310d28b5e Canal VR Meta Quest : projet Unity, viewer web et documentation

Premier commit du cinquième front. Trois parties :

- `unity/MyInfoMateVR/` : le projet Unity (6000.0.83f1, URP, Meta XR SDK 205),
  un APK unique pour tous les clients. Menu flottant à sélection au regard,
  appairage, chargement de scène GLB, POI, cache de contenu, télémétrie.
- `unity-overlay/` : les mêmes scripts à recopier sur un projet Unity neuf, avec
  les pièges rencontrés consignés dans son README.
- `viewer/` : viewer et éditeur de scène web autonome (Vite, TypeScript,
  three.js), partagé avec les autres fronts.
- `docs/` : état des lieux, setup Unity, décisions d'architecture et plan
  d'exécution en 9 étapes.

La scène est décrite par un `scene.json` poussé par `adb push` : l'app le
préfère à celui embarqué dans l'APK.

Les binaires (GLB, textures de l'échantillon Sponza, DLL Meta XR) passent par
Git LFS dès ce premier commit — les y faire entrer après coup demanderait de
réécrire l'historique. Les artefacts régénérés par l'éditeur et par CMake
(`Library/`, `.utmp/`, Burst debug) sont ignorés.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

2026-09-16 15:26:07 +02:00

3.3 KiB

Raw Blame History

name, description, allowed-tools, tags

name

description

allowed-tools

Meta XR Operator Usage

Meta XR Operator is an OpenXR API Layer that gives AI agents the ability to perceive, understand, and manipulate VR/AR applications at runtime via MCP tools.

Key constraint: Meta XR Operator tools are ONLY available when the VR/XR application is actively running. Tools will return errors if the app is not active.

Setup

Meta XR Operator is available as an MCP server. To setup:

If HZDB v1.3.0+ is installed as an MCP server but the Meta XR Operator tools (e.g., openxr_*) are not available, run the "install meta xr operator mcp proxy" tool exposed by HZDB.
Otherwise, prompt the user to follow steps within the Meta XR Core SDK's AI Tools window in Unity to set up Meta XR Operator.

When to Use

Use Meta XR Operator to interact with a running XR application at runtime:

Iteration — Launch the app, verify features, interact, then exit to make changes.
Debugging — Reproduce and investigate runtime bugs by inspecting poses, scene state, and visuals.
Test Automation — Programmatically verify features. See the hz-meta-xr-operator-unity-test-mechanics skill for more details.

Start by orienting yourself. Use openxr_get_head_pose + openxr_capture_composited_image to see where you are.
Use data before vision. Query scene data (get_scene_root_objects, get_children, get_world_pose) to understand layout via coordinates, then confirm visually with screenshots.
Don't guess positions — always use get_world_pose for both target and rig. Check positions relative to the camera rig, not world origin.
If you can't find something visually, step back to get a wider field of view. If still not found after 2-3 screenshots, use data tools to get coordinates and navigate there directly.

For detailed coordinate math and controller positioning, see hz-meta-xr-operator-coordinates. For aiming grabbed objects (guns, tools), see hz-meta-xr-operator-grabbed-objects.

Verification

Verify both via data AND visually. Confirm results using coordinate/scene-data tools AND compositor screenshots.
Take screenshots after key interactions to verify visual state matches expectations.
Check for visual artifacts (aliasing, noise) by capturing screenshots from multiple angles.

Input Simulation

Smooth movements: Use the duration parameter on openxr_set_head_pose and openxr_set_controller_pose for realistic motion.
Controller input values: Buttons are 0/1, Trigger/Grip are 0.0–1.0, Thumbstick is -1.0 to 1.0 on each axis.
Hold inputs across frames — setting an input to 1 and immediately to 0 may be missed by Update().

Common Pitfalls

Tools only work when the app is running. Connection errors mean the app may have stopped.
Coordinate systems matter. OpenXR is right-handed (-Z forward). Unity is left-handed (Z forward). See the hz-meta-xr-operator-coordinates skill for conversion.

3.3 KiB Raw Blame History Unescape Escape