vr-app/unity/MyInfoMateVR/.claude/skills/meta-xr-operator-unity-workflow/SKILL.md

name, description, allowed-tools, tags

name	description	allowed-tools	tags
hz-meta-xr-operator-unity-workflow	Iterates on Unity projects targeting Meta Quest and Horizon OS via the Meta XR Operator MCP server, covering Play Mode gating, scene editing safety, scene hierarchy navigation, UI interaction, and HZDB documentation lookup.	Bash(hzdb:*)	agentic-xr unity openxr mcp vr development iteration

Meta XR Operator + Unity Development

Using Meta XR Operator within a Unity + XR Simulator development workflow. Follow the hz-meta-xr-operator skill for general runtime interaction guidance.

The Iteration Loop

1. Plan feature/fix (use HZDB for Oculus VR docs if available)
2. Modify Unity project (use Unity MCP to edit scripts, scenes, etc.)
3. Validate scene in Editor (take editor screenshot for layout understanding)
4. Enter Play Mode (app starts, Meta XR Operator becomes available)
5. Interact & verify at runtime (use Meta XR Operator MCP tools)
6. Exit Play Mode (Meta XR Operator disconnects)
7. Repeat from step 2

Critical Rules

• Unity MUST be in Play Mode for Meta XR Operator tools to work. They return errors otherwise.
• Exit Play Mode before making persistent changes to scripts, scenes, or project settings via Unity MCP.
• If Unity MCP becomes unresponsive, Unity is likely compiling code — wait and retry.

Before Entering Play Mode

• Take an editor screenshot to understand the initial layout.
• Query the scene hierarchy via Unity MCP to understand what GameObjects and components exist.

Navigating the Scene Hierarchy

1. get_scene_root_objects — see top-level objects
2. get_children("path") — drill down (e.g. "Canvas/Panel/StartButton")
3. get_world_pose("path") — get coordinates, then convert to OpenXR (see coordinates skill)

Interacting with UI

1. find_canvases — locate UI canvases
2. find_interactables("CanvasPath") — discover buttons, sliders, etc.
3. get_world_pose on the interactable → convert to OpenXR coordinates
4. openxr_set_controller_pose — move controller to that position
5. openxr_set_controller_input(Trigger, 1) then (Trigger, 0) — simulate click
6. openxr_capture_composited_image — verify

General Unity MCP Tips

• Script types aren't available immediately — after creating C# scripts, Unity needs to compile. AssetDatabase.Refresh() triggers compilation but disconnects MCP. Wait, then use Type.GetType("ClassName, Assembly-CSharp") to add components.
• RunCommand can't reference Assembly-CSharp types directly — use Type.GetType() + AddComponent(type) and SerializedObject for wiring references.
• Update serialized values in the editor, not just code defaults — public fields are serialized in the scene. Use Unity_RunCommand with EditorUtility.SetDirty() to update live values.
• Reimport shaders/scripts after editing — AssetDatabase.ImportAsset(..., ImportAssetOptions.ForceUpdate) and SceneView.RepaintAll(). Shader.Find() returns null for unimported shaders.
• Reassign shader to material after adding/removing properties to avoid stale bindings.

Colliders & Physics

• Check isTrigger — OnCollisionEnter only fires on non-trigger colliders; triggers require OnTriggerEnter. When in doubt, implement both.
• Use ContinuousDynamic for fast projectiles — default Discrete mode lets fast objects tunnel through colliders.
• Never spawn GameObjects from OnDestroy — Unity calls it during scene teardown. Move spawn logic to the caller.

Using HZDB (if available)

Query HZDB for Oculus VR documentation on VR-specific features (hand tracking, passthrough, spatial anchors, etc.). If unavailable, proceed with general VR knowledge.