Spatial UI design refers to the discipline of creating user interfaces (UI) that exist within a three-dimensional space, often in virtual or augmented reality environments. Unlike traditional two-dimensional interfaces, spatial UI design considers the user's physical presence and movement within the digital space. It focuses on enhancing user interaction and experience by leveraging spatial awareness, depth perception, and natural gestures.
In spatial UI design, elements are positioned and arranged in a way that mimics real-world environments, allowing users to navigate and interact intuitively. This approach opens up new possibilities for immersive experiences across various domains, including gaming, education, architecture, and healthcare.
Overall, spatial UI design represents an exciting frontier in the field of user experience (UX) design, offering endless possibilities for creating immersive and interactive digital environments that blur the line between the physical and virtual worlds.
Let's summarize Frank's guidelines for spatial design in AR.
Have your program sense and respond to the environment. Use space to trigger new content automatically when the user gets near.
Affordances are the best tool to make UIs intuitive, especially 3D ones. Don't force users to interpret what things mean. If something looks intractable, make it intractable. Work with real physics and the environment. Use real physics and respect the boundaries of physical space; don't put an intractable object beyond a wall or window. Avoid "secret UIs" or hidden features that users need to use.
Have objects behave and interact in a way that mimics their real-world counterparts. A rubber ball should bounce, and the door handle should open a door.
Use natural interactions—for example, simple hand gestures like pressing a button, rotating, or grabbing an object. Use eye tracking or triggers based on the user's proximity. AR is not a suitable medium to flip through menus in, so make interactions as simple and intuitive as possible.
AR is highly interruptible. Don't make experiences with uninterruptible content like long videos or animations, especially if they cannot be paused. Allow the user to drop in and out of the experience quickly. It s very similar to how mobile apps work.
Don t overwhelm users with multiple interactions simultaneously. Instead, use contextual tutorials, which tell the user what they need to know in the moment.
Spatial layout techniques help you take advantage of the infinite canvas of Apple Vision Pro and present your content in engaging, comfortable ways.
A person’s field of view is the space they can see without moving their head. The dimensions of an individual’s field of view while wearing Apple Vision Pro vary based on factors like the way people configure the Light Seal and the extent of their peripheral acuity.
The system doesn’t provide information about a person’s field of view.
Center important content within the field of view. By default, visionOS launches an app directly in front of people, placing it within their field of view. In an immersive experience, you can help people keep their attention on important content by keeping it centered and not displaying distracting motion or bright, high-contrast objects in the periphery.
Avoid anchoring content to the wearer’s head. Although you generally want your app to stay within the field of view, anchoring content so that it remains statically in front of someone can make them feel stuck, confined, and uncomfortable, especially if the content obscures a lot of passthrough and decreases the apparent stability of their surroundings. Instead, anchor content in people’s space, giving them the freedom to look around naturally and view different objects in different locations.
People rely on visual cues like distance, occlusion, and shadow to perceive depth and make sense of their surroundings. On Apple Vision Pro, the system automatically uses visual effects like color temperature, reflections, and shadow to help people perceive the depth of virtual content. When people move a virtual object in space — or when they change their position relative to that object — the visual effects change the object’s apparent depth, making the experience feel more lifelike.
Because people can view your content from any angle, incorporating small amounts of depth throughout your interface — even in standard windows — can help it look more natural. When you use SwiftUI, the system adds visual effects to views in a 2D window, making them appear to have depth.
Provide visual cues that accurately communicate the depth of your content. If visual cues are missing or they conflict with a person’s real-world experience, people can experience visual discomfort.
VisionOS defines two types of scale to preserve the appearance of depth while optimizing usability.
Dynamic scale helps content remain comfortably legible and interactive regardless of its proximity to people. Specifically, visionOS automatically increases a window’s scale as it moves away from the wearer and decreases it as the window moves closer, making the window appear to maintain the same size at all distances.
Fixed scale means that an object maintains the same scale regardless of its proximity to people. A fixed-scale object appears smaller when it moves farther from the viewer along the z-axis, similar to the way an object in a person’s physical surroundings looks smaller when it’s far away than it does when it’s close up.
These elements now exist in a whole new dimension, adding depth to the digital world and making it more immersive. It's not just about looks; it's about making users feel at ease in this new 3D environment.
In spatial design, keeping things familiar is crucial. Designers achieve this by incorporating recognizable elements from 2D interfaces, like sidebars, tabs, and search fields, into the 3D space.
Windows play a fascinating role in spatial design, forming a unique visual language. They mimic real glass, bringing a cool contrast to the surroundings and making users more aware of their environment, adapting to different lighting conditions.
In this new spatial setting, users won't feel lost because they're already familiar with windows from 2D design. They can easily move, close, and resize windows, placing them wherever they like. Plus, the windows automatically adjust to face users, making everything readable and accessible.
If you have experience in other kinds of design, you're probably already familiar with points. They adjust as windows or elements move closer or farther from users, safeguarding legibility and functionality. For instance, if an object is designed to be 10 points at one meter, it will appear twice as large at half a meter, and half as large at two meters. This principle is essential in preserving a consistent visual experience within a 3D space.
Another key principle of spatial design is being human-centered. The aim here is to put users in the driver's seat, making the user experience a vital part of the design itself. With devices like Vision Pro, for instance, users' eyes, hands, and movements become an integral part of the interaction with the digital world. This approach ensures a deeply personalized and immersive experience.
When crafting apps for spatial computing, it's crucial to consider how users will see and move within this digital 3D environment. It's not just about ease of use — it's about crafting an experience that feels like a natural extension of the user's world.
Designing for spatial computing, like with Apple's Vision Pro, demands a deep understanding of ergonomics, or how people physically interact with their environment. Why does ergonomics matter in spatial design? Simply put, the placement of your content directly affects how users react physically to your design. That's why we want to position objects comfortably in all dimensions.
The dimensional design principle focuses on creating an amazing experience in spatial computing. By adding depth to your app, you make it feel more real and immersive, going beyond the limits of a regular screen. It's all about utilizing the space around users to make them feel like they're part of the digital world.
When designing apps for spatial environments, the goal is to create a world that can either blend in with the surroundings of users or totally change it, depending on the app's purpose. Take the Keynote app as an example. It starts off in a shared space with other apps, but when it's time to rehearse a presentation, it expands to a full-sized theater, taking over the whole space and giving users a truly immersive experience.
Yet, even when an app takes over the full space, it doesn't always mean a total transformation of the environment. It can still make users feel like they're in their own space, just with some added dimensions. For example, imagine a digital element that casts a shadow on a real-life table.
