SPATIAL INTERACTION · EMBODIED AI · HUMAN–ROBOT INTERACTION
Open worlds, 3D and robotics
Designing experiences when people and agents share a space.
Interactive 3D worlds can support navigation, exploration and rehearsal. Robotics adds physical bodies, sensing and consequences. Their connection creates UX questions about orientation, intention, boundaries and control—not simply the realism of the graphics.
3 peer-reviewed background papers · Sources checked 18 October 2026
Related areas, different claims
Open worlds and interactive 3D
Environments offering multiple routes and possible actions, including games and exploratory virtual spaces. An open-world game still has designed boundaries; it is not an unrestricted model of reality.
Simulation and embodied AI
Virtual agents perceive and act within an environment. Simulation supports repeatable trials, but visual realism alone does not establish realistic physics, sensing or human behaviour.
Robotics and shared physical spaces
People must interpret a machine’s movements and intentions, sometimes without a screen. Bystanders, personal space, accessibility and physical constraints become part of the experience.
World models
Models that represent or predict aspects of an environment and how it changes. These are distinct from a rendered 3D scene; generated predictions are not guaranteed physical outcomes.
These reading routes connect people, research and design questions. They are editorial pathways, not claims that the organisations collaborate.
From a researcher to a paper:Fei-Fei Li is a co-author of BEHAVIOR. Follow that connection to examine how household activities become simulated tasks, then ask which human needs the benchmark does—and does not—capture.
From language to action:RT-2 connects vision, language and robotic control. Read it alongside UX for agentic AI to consider permission, intervention and recovery when an instruction leads to action.
Orientation and agency: can people recognise where they are, what is actionable and how to leave or regain control?
Readable behaviour: can someone infer an agent’s destination or intention? An efficient route may not be the easiest movement to interpret.
Shared space: whose route, attention or personal space is affected? Include people who did not choose to use the system.
Accessible alternatives: provide ways to participate without a headset, precise gestures, stereoscopic vision or standing. Offer a text or 2D representation of essential state.
Evidence of transfer: distinguish simulation performance, laboratory robot trials and field studies. Evaluate understanding and recovery alongside task completion.
Editorial research prompts, not a robotics safety standard or a claim of verified accessibility.
Research papers 3 items
Start with these foundations. All are older background readings, not current news; review depth is stated separately for each.
Anca D. Dragan, Kenton C. T. Lee and Siddhartha S. Srinivasa · ACM/IEEE HRI 2013 Open author manuscript · full text reviewed
Distinguishes motion that helps an observer infer a goal from motion that matches expectations when the goal is already known. The paper combines a formal model with experiments involving different kinds of actors.
Limitations: The tasks and trajectories studied are bounded. Findings do not establish that a robot will communicate clearly in crowded, unfamiliar or culturally varied settings, or that legible motion is physically safe.
Consider: Can a person tell where the robot is going before its movement is complete?
Sanjana Srivastava and colleagues · CoRL 2021 · proceedings published in PMLR 164, 2022 Open paper · full text reviewed
Introduces 100 simulated household activities, task-state definitions and evaluation metrics, with 500 VR demonstrations as human reference data. The evaluated learning approaches struggled with the benchmark’s complex activities.
Limitations: These are simulation results, not evidence of reliable assistance in occupied homes. Activity definitions draw on the American Time Use Survey; their cultural and household assumptions should be examined. VR demonstration performance is not a general measure of usability.
Consider: Whose definition of a successfully completed household task does the benchmark encode?
Brianna Zitkovich and colleagues · CoRL 2023 · PMLR 229 Open publication · abstract-only review; full PDF could not be retrieved for this review
The abstract describes training vision-language models with robot trajectories and web-derived vision-language tasks, representing robot actions as tokens. This connects language-based instructions with physical robot control.
Limitations: This summary is restricted to the publisher’s abstract. Detailed experimental conditions and limitations were not independently reviewed here. Reported generalisation should not be read as unrestricted competence or evidence of safe everyday deployment.
Consider: How should an interface reveal uncertainty when an instruction could produce a physical action?
A starting directory across the US, UK and Singapore, selected for relevance rather than ranked prominence. Institutional pages and roles checked 18 October 2026.
Stanford HAI lists Li as its founding director and senior fellow, Sequoia Professor of Computer Science, and an advisory council co-chair. Her work connects visual intelligence with embodied AI; she is a co-author of the BEHAVIOR paper in this collection.
Follow her publications to connect perception, simulation and robot learning. This is a research profile, not a claim that she practises as a UX designer.
Connects AI research with human needs, education and policy. Its scope extends beyond robotics to the broader social and institutional conditions in which AI is developed and used.
Useful for human-centred framing, research reports and public discussions. Assess the methods of each publication separately; institutional standing is not an evidence label.
University of Oxford’s robotics research institute within the Department of Engineering Science.
A source of robotics projects and publications relevant to machines operating in physical environments. Examine the task, environment and evaluation before drawing UX conclusions.
Brings together robotics, AI, HCI, design and other disciplines to study social and physical interaction between people and robots.
Particularly relevant to readable behaviour, assistance and collaboration. Follow individual studies rather than treating a laboratory demonstration as evidence of general readiness.
NUS’s AI group directory includes Learning for Embodied Minds and Agents (LEMA) and the Manipulation and General Intelligence Control (MAGIC) Lab.
Useful for following perception, learning and action research. This is a directory of groups, not a single centre or a ranking; check each project’s publications and evaluation.
Portfolio exercise: a robot guide in a virtual gallery
Editorial learning exercise · begin with a storyboard or desktop prototype; a headset or physical robot is not required.
Create a small gallery with alternative routes, an artwork stop and a quiet area. A virtual guide must communicate its intended route and let a visitor pause, redirect or dismiss it. Think about how pacing, proximity and attention affect the experience of viewing art.
Map the space and responsibilities. Show accessible routes, personal-space boundaries and what the guide may do. Separate visitor preferences from assumptions about what they want.
Prototype two behaviours. Compare a direct movement with one that makes the destination easier to infer. Add a blocked route and an ambiguous instruction. Provide visible controls and a keyboard-accessible 2D alternative.
Evaluate understanding. After any required ethics approval, ask people to predict the destination, redirect the guide and recover from the blocked route. Record errors, intervention effort and explanations of intent. Describe what a simulated prototype cannot establish about physical safety.
Portfolio output: a spatial map, annotated interaction sequence, accessible alternative, findings and one evidence-led revision. Any later physical-robot work needs specialist supervision and an appropriate risk assessment.