B4. Audio & User Interface

F01

Learning Outcomes

  • Add and configure audio sources in Unity, distinguishing between diegetic and non-diegetic sound. Ahead of class, open Unity, attach an Audio Source to a GameObject, enable looping, adjust volume, and play a short sound effect while considering how sound placement impacts XR immersion.
  • Implement spatial audio effects for immersive XR soundscapes. As preparation, set an audio source’s Spatial Blend to 3D, configure distance falloff, then move the player in Play mode to hear changes in spread and directionality.
  • Create and position UI elements for both overlay and world-space contexts. Before the session, add a Canvas with a TextMeshPro label and a Button, then switch the Canvas render mode between Overlay and World Space to compare presentation styles in XR.
  • Use anchors to make UI elements responsive to screen size changes. In advance, place a button in the top-right corner using Anchor Presets, then test different Game view resolutions to confirm the position remains consistent.
  • Configure interactive buttons that trigger actions in Unity. Prior to class, create a button that toggles a GameObject’s visibility via OnClick() and adjust its hover and pressed states in the Color Tint settings.
  • Implement toggles to control GameObject properties. For your prep work, add a toggle that switches an Audio Source on or off, replacing the default label with a TextMeshPro label that reads “Music.”
  • Build sliders to dynamically adjust audio properties. Before coming in, create a slider that modifies an Audio Source’s volume, label it “Volume” using TextMeshPro, and set its starting value to 0.5.

Audio

Audio is a crucial element in immersive experiences, yet it is often overlooked. In interactive games or XR applications, sound plays a significant role in shaping user perception, guiding interactions, and reinforcing realism. A well-designed audio system enhances engagement, while poor implementation can break immersion. Unity provides a powerful built-in audio system that allows developers to create realistic soundscapes.

F02

In XFactory, for example, audio can add critical depth to the simulation—such as the subtle humming of CNC machines, robotic arms whirring, or the clanking of parts being assembled—improving realism and training effectiveness.

Core Audio Components

Unity’s audio system is based on two fundamental components:

  • Audio Listener: This component acts as the ears of the user within the Unity environment. It captures and interprets sound based on spatial relationships between sound sources and the user’s position. It is typically attached to the Main Camera. Therefore, only one active Audio Listener should exist in a scene. It determines how sounds are heard depending on user location and orientation.

  • Audio Source: This is the component that plays a sound. It’s attached to a GameObject and emits an audio clip with various customizable parameters:

    • Audio Clip: The sound file to be played.
    • Volume: Controls loudness.
    • Loop: Repeats the audio clip continuously.
    • Play on Awake: Automatically plays sound when the scene starts.
    • Spatial Blend: Switches between 2D (non-directional) and 3D (spatialized) sound.
    • Pitch: Alters sound frequency.

F03

In XFactory, the listener follows the user’s head in a VR headset, adjusting the volume and direction of sounds as they explore the factory. As a user walks into the welding station, the sizzling of arc welders becomes more prominent, while the drone beeping from the logistics station fades away.

Sound Types

  • Diegetic Sound (In-World Sound): These are sounds that originate from within the scene and are heard by both users and virtual characters. In XFactory, examples of diegetic sounds include forklift engines revving in the logistics station or 3D printer layering noises in the production zone.

  • Non-Diegetic Sound (External Sound): These are sounds that don’t originate in the scene itself but enhance atmosphere or guide user experience. In XFactory, ambient background music during the onboarding menu, notification tones when an objective is completed, or dramatic stingers when transitioning between factory zones are all examples of non-diegetic sounds.

Common audio file formats in Unity are MP3 (compressed format that is suitable for background music), WAV (uncompressed format best for sharp and high-fidelity sound effects), OGG (compressed, open-source format that offers a good balance of quality and file size), and AIFF (high-quality format ideal for looping or ambient tracks).

Adding Audio

  1. Creating Sound-Emitting GameObjects:
    • Select a GameObject (e.g., CNC_Mill_Set in the manufacturing station).
    • In the Inspector panel, click Add Component > Audio Source.
    • Drag an audio file into the Audio Resource field (e.g., Assets > XFactory > Audio > Factory Environment Sounds > machine.ogg).
    • Enable Loop for continuous operation sounds like machining.
    • Adjust Volume and Pitch to simulate different machine loads or tool types.
    • Test in Play mode to verify how the sound behaves as you move around the machine.

    01

  2. Playing Background Music:
    • Right-click in Hierarchy > Create Empty.
    • Name it Ambient Factory Sound.
    • Add an Audio Source component.
    • Add an ambient track to the Audio Resource (e.g., Assets > XFactory > Audio > Factory Environment Sounds > ambiance.ogg).
    • Enable Loop and set Volume to a subtle level.

    02

To ensure critical sounds are not drowned out in busy scenes, adjust the Priority slider in the Audio Source. Lower values mean higher importance (default is 128).

Accessibility Considerations

To ensure inclusive audio experiences, integrate the following into auditory interactions with the user:

  • Closed Captions: Provide on-screen text descriptions (e.g., [robot arm clicks], [machine hums]).
  • Visual Feedback: Pair sounds with icons, flashes, or animations.
  • Custom Audio Settings: Let users adjust or mute background music vs. sound effects separately.

In XFactory, users with hearing impairments can be provided with a “3D Printer Operating” notification when the audio cue would normally play. Visual waveform overlays or light flashes signal machine activity.

Spatial Audio

Spatial audio is a crucial component of immersive experiences in XR, enabling developers to create realistic soundscapes that dynamically respond to user movement and environmental factors. Unity provides a robust 3D audio engine that simulates real-world sound behavior—attenuating with distance, shifting between stereo channels, and interacting with surfaces and obstacles. In XR applications, spatial audio is essential to enhance the user’s spatial awareness and comprehension in use cases such as:

  • Training and Safety: Simulating industrial environments where audio cues—like beeping of forklifts, alarms from malfunctioning machines, or alerts from PLC HMIs—signal hazards and guide user attention.
  • Prototyping and Design: Enabling real-time walkthroughs of virtual factories with contextual machine sounds, making it easier to evaluate layout and acoustic behavior.
  • Data Sonification: Turning complex sensor data (like pressure or torque values) into audio signals for easier real-time interpretation.

F04

In XFactory, spatial audio can greatly enhance realism and engagement. The humming of CNC machines, the whirring of drone propellers, or the clank of metal parts being assembled by robots not only provide atmospheric immersion but also convey important contextual information about what’s happening in different areas of the factory.

Key Concepts

  • Spatial Blend: A slider (0 to 1) that controls the balance between 2D (non-directional) and 3D (directional) audio. Set to 1 for spatial realism.
  • Volume Rolloff: Dictates how sound diminishes as the listener moves away.
  • Spread: Controls how widely a sound is panned across stereo speakers. Affects how naturally a sound appears to wrap around the user.
  • Doppler Effect: Simulates changes in pitch due to motion—useful for moving robots, drones, or AGVs (Automated Guided Vehicles).

Adding Spatial Audio

Let’s revisit the existing setup by converting the CNC machine’s Audio Source to spatial audio for realistic 3D sound behavior in Unity using the XFactory environment.

  1. Convert Existing Audio Source to Spatial:
    • Select the CNC machine GameObject in the Hierarchy (CNC_Mill_Set).
    • In the Inspector, locate the Audio Source component.
    • Verify that an appropriate Audio Resource is assigned (e.g., machine.ogg) and Loop is enabled.
    • Set the Spatial Blend slider fully to 3D to make the sound position-based.
  2. Adjust 3D Sound Settings:
    • Min Distance: Set to 2–3 for CNC machines. This is the distance where the sound plays at full volume.
    • Max Distance: Set to 30–50 to define how far the sound can be heard.
    • Spread: Set between 90–150 to simulate how sound fills the space and transitions between ears as you move.
    • Doppler Level: (optional) Set to 0 for stationary machines, or around 1.0 if the sound source may move (e.g., on mobile platforms).

    03

  3. Choose a Volume Rolloff Curve:
    • Expand the 3D Sound Settings section in the Audio Source.
    • Choose a Rolloff Mode.
    • Logarithmic is best for realistic audio behavior in large open areas (e.g., factory floor).
    • Linear is used for enclosed environments where you want quicker drop-offs (e.g., welding booths).
    • Custom allows to design your own curve for special materials or dampened acoustic zones.
  4. Test the Setup:
    • Press Play.
    • Move near the CNC machine and observe how the sound grows louder as you approach.
    • Rotate around it to check directional audio behavior.
    • Move away and confirm the sound fades based on the Max Distance and rolloff curve.
  5. Optimize Listener Position:
    • Proper listener placement ensures immersive audio response.
    • If using a player or VR rig, move the Audio Listener from the Main Camera to the player’s root object (e.g., XR Rig or robot controller).
    • Walk through the environment in Play mode and listen to how audio shifts across logistics, assembly, and production zones.

With spatial audio applied, your CNC machine now produces immersive sound feedback that responds to distance, orientation, and player movement—ideal for XR training, robotics simulations, or factory walkthroughs.

Audio Reverb Zones

Sound waves interact with the environment similar to light waves. In soft, irregular spaces, sound is absorbed quickly. However, in enclosed, hard-surfaced environments, sound waves bounce off surfaces, creating an echo or reverberation. This effect is essential in areas such as caves, tunnels, large halls, and narrow corridors. Since Unity doesn’t automatically apply these effects, developers must manually define where reverberation should occur using Reverb Zones.

In XFactory, the production station where CNC machines and robotic arms operate in close proximity might benefit from a “Factory Hall” or “Concrete Hall” preset to reflect sound bouncing off metallic and concrete surfaces.

  1. Create a Reverb Zone:
    • In the Hierarchy, right-click and select Audio > Audio Reverb Zone.
    • Rename it appropriately (e.g., XFactory Hall Reverb).
  2. Position the Reverb Zone
    • Move the reverb zone to the center of the echoing environment.
    • Adjust the Min Distance to cover most of the enclosed space.
    • Adjust the Max Distance so that it slightly extends beyond the area.
  3. Select a Reverb Preset:
    • In the Inspector, locate the ReverbPreset dropdown.
    • Choose an appropriate preset (e.g., Hallway).
    • Playtest and tweak settings as needed.

    04

Unity provides multiple ReverbPreset types that simulate real-world environments. Experimenting with these helps fine-tune the desired effect. In advanced implementations, reverb zones can be scripted to activate when a player enters the space and deactivate when they leave.

Audio Mixer

The Audio Mixer allows precise control over multiple sounds, enabling volume balancing between different sound groups, applying effects (e.g., low-pass filters for underwater or muffled communication), and dynamic sound mixing based on game events. In XFactory, for example, you can use the mixer to balance sound levels between CNC machines, robot movement, and PLC beeps. Simulate headset noise cancellation by reducing machine sounds when the user picks up an AR/VR headset in the exhibit station.

  1. Open the Audio Mixer:
    • Navigate to Window > Audio > Audio Mixer.
    • Click the + button to create a new mixer.
    • Rename it (e.g., XFactory Audio Mixer).
  2. Create an Audio Group for Machine Sounds:
    • Click + in the Groups section.
    • Rename it Machinery.
  3. Assign Sounds to the Mixer Group:
    • Select an Audio Source (e.g., CNC machine hum).
    • Locate the Output property in the Inspector.
    • Select the Machinery group from the dropdown.

    05

  4. Fine-tune Audio Levels and Effects:
    • In Play mode, adjust volume sliders for real-time changes.
    • Add EQ or reverb effects to the Machinery group to reflect distance or occlusion (e.g., sounds muffled by walls or partitions).

Consider creating a Voice Instructions group for human guidance audio, and duck the volume of all other groups when the instructor is speaking.

User Interface

A user interface (UI) is a system that enables users to interact with a computer system. In Unity, UI elements are essential for game development, engineering simulations, and interactive applications, allowing users to navigate menus, configure settings, and interact with 3D environments. UI elements in Unity are primarily created using the Unity UI system, which allows developers to build interfaces visually using a hierarchical structure of UI components.

F05

In XFactory, UIs enable intuitive user interaction. Large screens at the exhibit, assembly, and production stations provide immersive visualizations and task guidance. Users can also interact with simulated CNC machine HMIs, Siemens PLC HMIs, and UR10e teach pendants to control machines or robots, mirroring real-world interfaces for realistic engineering simulation.

Canvas

The Canvas is the foundation of Unity’s UI system. It acts as a container for all UI elements and determines how they are rendered on the screen. Every UI element must be a child of a Canvas to be visible. Different types of Canvas render modes include:

  • Screen Space—Overlay: UI elements are rendered on top of everything in the scene. The UI size is independent of the camera position. Best for HUDs (Heads-Up Displays) and fixed UI menus. Example: A persistent overlay showing production metrics while navigating the welding station in XFactory.

  • Screen Space—Camera: UI elements are rendered relative to a specific camera. Allows for depth-based effects, making the UI feel integrated with the scene. Best for 3D UI menus that need to scale with perspective. Example: Displaying contextual information on parts or machines when viewed through a mobile device or headset camera in AR mode.

  • World Space: UI elements exist as objects within the 3D scene. These UI elements can be moved, rotated, and interacted with in 3D space. Best for in-world UI, such as control panels, floating indicators, or VR interfaces. Example: Interactive screens at the assembly station providing step-by-step task guidance, or a floating UI for controlling the quadruped robot in the exhibit station.

RectTransform

The RectTransform component is the UI-specific version of the Transform component used for GameObjects. It allows for precise control over UI element positioning, size, and alignment. Key RectTransform properties include:

  • Anchors: Define how the UI element is positioned relative to its parent. In XR, this helps keep controls fixed to device screens or scene anchors.
  • Pivot: The reference point for rotation and scaling.
  • Offset: Control padding and spacing relative to the parent object.
  • Stretching: Enables dynamic UI layouts that adapt to different screen sizes, such as varying screen dimensions in AR headsets or tablets used in the logistics station.

UI Components

  • Text (TextMeshPro): Displays static or dynamic text. TextMeshPro is the recommended text rendering system as it provides sharper fonts, better formatting, and more customization. Example: Status messages or sensor readouts on a CNC machine interface in the manufacturing station.

  • Image: Displays 2D sprites or textures in the UI. Can be used for backgrounds, icons, and decorative elements. Example: Machine logos or safety icons in a digital replica of a PLC HMI.

  • Button: A clickable UI element that triggers an action. Can contain text and images to enhance visual representation. Example: Start/Stop buttons on an interactive robotic cell interface in the welding station.

  • Toggle: A switch between two states (on/off). Useful for enabling/disabling settings. Example: Toggling between different camera views of the production station.

  • Slider: Allows users to adjust a value within a defined range (e.g., volume, brightness). The handle can be moved to different positions. Example: Adjusting welding parameters or robot speeds in a robotics training module.

  • Dropdown: A list of selectable options. Expands when clicked, allowing users to choose from multiple predefined values. Example: Selecting different product configurations on the assembly station UI.

  • Scrollbar: Provides scrolling functionality for UI elements that contain more content than can fit in the visible area. Example: Viewing a long instruction set or part list on a touchscreen in the exhibit station.

  • Panel: A container for grouping UI elements together. Helps organize complex UI layouts. Example: A full-screen production dashboard in the exhibit station showcasing real-time KPIs.

F06

UI Layout Components

Unity provides several layout components to help structure UI dynamically:

  • Horizontal Layout Group: Aligns child UI elements horizontally with customizable spacing. Example: Displaying multiple control buttons on a teach pendant** simulation.

F07

  • Vertical Layout Group: Aligns child UI elements vertically, often used for menus and lists. Example: A step-by-step checklist for assembling components in the assembly station.

F08

  • Grid Layout Group: Arranges child elements in a grid format, ideal for inventories and icon-based menus. Example: A grid of parts/components shown on the logistics station inventory screen.

F09

  • Content Size Fitter: Automatically resizes UI elements based on their content.
  • Aspect Ratio Fitter: Ensures an element maintains a specific aspect ratio, regardless of screen size.

UI Event System

Unity’s Event System handles user interactions with UI components. It detects and processes inputs from mouse clicks, keyboard inputs, touch gestures (for mobile or headset applications), or controller inputs (for gaming applications). Unity’s UI system uses a Raycaster to detect which UI element a user is interacting with. The most common raycasters are:

  • Graphic Raycaster: Used for standard UI interactions.
  • Physics Raycaster: Used when UI elements need to interact with 3D objects in the scene, like pressing a button on a virtual PLC panel in the production station.

UI Fundamentals

When developing UI elements in Unity, ensuring they remain correctly positioned and proportionally sized across different screen sizes and aspect ratios is crucial. Unity provides tools like the Canvas, Anchor Points, and Canvas Scaler to help with responsive UI design. Let’s work on a running example to turn the large display in the assembly station (Display GT) into an interactive UI.

Canvas

The Canvas is a GameObject that acts as a container for all UI elements in Unity. It represents the screen where UI components such as buttons, text, and images are displayed. UI elements must be placed inside a Canvas to be rendered properly. To add and attach a Canvas to a large screen prefab in Unity:

  1. Add a Canvas:
    • In the Hierarchy, right-click on your large screen GameObject (e.g., Display GT) and select UI > Canvas.
    • This creates a new Canvas as a child of the display.
    • Unity may also add an EventSystem GameObject if it doesn’t already exist — this is required for UI interactions.
  2. Setup the Canvas in the Inspector:
    • Change the Render Mode to World Space (required for placing UI on 3D surfaces).
    • Adjust the RectTransform scale to something small (e.g., 0.002, 0.002, 0.002) so it fits realistically on the screen surface.
    • Adjust Width and Height as appropriate.
    • Resize and reposition the Canvas so that it aligns with the visible screen area on your mesh.

    06

  3. Edit in 2D (optional):
    • Enable 2D mode in the Scene view.
    • Make sure Gizmos are enabled to see the Canvas outline.
    • Use the Rect Tool (T) to position and scale the Canvas.
    • Double-click the Canvas in the Hierarchy to focus on it in the Scene view.

In XFactory, Canvas can be used for a variety of applications. For example, a large interactive touch screen in the assembly station displays guides and instructions for assembling components like the V8 engine and the tire. This UI is built on a World Space Canvas, parented to the screen mesh, and aligned to its surface for immersive in-scene interaction using VR controllers.

Anchors

Anchors define the UI element’s positioning relative to its parent, ensuring proper alignment when the Canvas scales or changes resolution. Each UI element on a Canvas has an anchor point, which determines how it moves and scales within the Canvas. To view and adjust anchors, select a UI element in the Hierarchy, press T or use the Rect Tool to access its Rect Transform, and look for blue circles (element bounds) and small triangles (anchor points). Here is an example of positioning UI with anchors:

  1. Create a Button by right clicking the Canvas and selecting UI > Button - TextMeshPro.
  2. Rename it Engine Assembly Guide and edit its label.
  3. Adjust the Anchor by:
    • Clicking the Anchor widget and dragging it to the bottom left (manual), or
    • Clicking the Anchor Presets dropdown in Rect Transform and choosing a bottom left preset to snap the button into place.

    07

In XFactory, this ensures the V8 engine guide button always stays in the top-right corner of the assembly station touchscreen, no matter the headset resolution or viewing angle.

Images

In Unity UI, images play a crucial role in designing user interfaces, serving as backgrounds, icons, buttons, and decorative elements. In XFactory, images and backgrounds enhance realism for XR-based UIs. For example, on the assembly station touchscreen, image elements help visualize parts of the V8 engine or tire, and replicate the look of physical control panels or instructional dashboards.

Adding a Background Image

A background image is commonly used in UI menus to enhance the visual appeal and structure of the interface. This is especially useful for building realistic in-world control screens or product guides. To add a background image:

  1. Create an Image Object:
    • In the Hierarchy, right-click on the World Space Canvas and select UI > Image.
    • Rename it to Interactive Engine Diagram (or a relevant name).
    • This adds a resizable, white square anchored to the Canvas.
    • Use the Rect Tool (T) to resize and position the image to fill part of the screen or align with a product display area.
  2. Changing the Background Color & Transparency:
    • In the Inspector, locate the Image Component.
    • Use the Color Picker to assign a background color.
    • Assign an image of the V8 engine model (or something similar) to Source Image.

    09

To import an image into Unity for UI, drag the image into the Assets folder, select it, set the Texture Type to Sprite (2D and UI) in the Inspector, and click Apply.

Adding Text to UI

A title or label helps users understand the purpose of an interface panel or section. To add a title to your in-world touchscreen UI:

  1. Create a Text Object:
    • In the Hierarchy, right-click on the background image or panel.
    • Select UI > Text - TextMeshPro.
  2. Customize the Text:
    • Rename it to something like V8 Engine Diagram.
    • In the TextMeshPro - Text (UI) component, enter a title (e.g., V8 Engine Diagram).
    • Adjust the font size, color, and alignment for readability.

    10

Creating an Exit Button

Every interface panel benefits from an Exit or Close button to allow users to return to the main screen or dismiss overlays. To add one:

  1. Create a Button Object
    • In the Hierarchy, right-click within the background image panel.
    • Choose UI > Button - TextMeshPro and rename it to Exit Button.
  2. Modify the Button Text:
    • Expand the button in the Hierarchy.
    • Select the child Text (TMP) and change it to X.
  3. Adjust Button Size and Position:
    • Use the Rect Tool to make the button a small square.
    • Move it to the upper-right corner of the background panel.

    11

Buttons

A button is a core UI element, enabling user interactions such as navigation, toggling settings, or triggering actions. Buttons are widely used across applications, from mobile keyboards to web search filters and game menus. In Unity, buttons are part of the UI system and are managed through the Button component.

In XFactory, buttons might be used to start the robotic welding cycle, toggle between machine states on a CNC virtual HMI, or interact with task guidance menus in the assembly station.

Button Interactions

A well-designed UI provides clear feedback to users, ensuring they understand what can be interacted with and whether their actions were successful. Common button interactions include:

  1. Hover Effects: Buttons often change appearance when hovered over (e.g., color tint changes or slight enlargements).
  2. Click Effects: A more noticeable change occurs when the button is pressed, confirming interaction.
  3. Selection Effects: Some buttons stay highlighted after selection, indicating an active state.

Button Transition Colors

Unity provides various ways to visually indicate button interactions. By default, the Button component uses the Color Tint` transition. To customize button colors:

  1. Select the Button GameObject in the Hierarchy.

  2. In the Inspector, locate the Button component.

  3. Find the Transition property (default: Color Tint).

  4. Customize the following color properties:
    • Normal Color: Default color of the button.
    • Highlighted Color: Color when hovered over.
    • Pressed Color: Color when clicked.
    • Disabled Color: Color when the button is inactive.
    • Color Multiplier: Increases the tint effect (useful for dark or semi-transparent buttons).

    17

  5. Test button colors.
    • Run the Unity scene and hover over the button.
    • Adjust the Highlighted and Pressed colors for better visibility.
    • Experiment with different color combinations to improve UI clarity.

If UI clicks are not registering during in-editor testing on macOS, try switching the EventSystem to a Standalone Input Module, as Unity’s new Input System may not detect trackpad taps.

On Click Event

A button in Unity becomes functional when an On Click event is assigned. This is managed using UnityEvents, which allow buttons to trigger actions when clicked. Let’s use On Click to toggle visibility of the V8 Engine Diagram on the assembly station touchscreen in XFactory.

  1. Set Initial Visibility:
    • Deactivate the Interactive Engine Diagram (which includes the image, title text, and exit button) in the Inspector by unchecking the checkbox next to the GameObject’s name.
    • Activate the Engine Guide Button, which users will click to bring up the guide.
  2. Assign On Click to Show the Guide:
    • Select the Engine Guide Button in the Hierarchy.
    • In the Inspector, locate the Button component.
    • In the On Click () section, click the + button to add a new event.
    • Drag the Interactive Engine Diagram GameObject into the Object field.
    • From the dropdown, choose GameObject > SetActive (bool). Make sure the checkbox is enabled — this means the panel will be shown.
    • Now, clicking the Engine Guide Button activates the instructional overlay showing the Interactive Engine Diagram and related UI.

    12

  3. Assign On Click to Hide the Guide:
    • Select the Exit Button GameObject in the Hierarchy.
    • In the On Click () section of its Button component, click the + button.
    • Drag the Interactive Engine Diagram into the Object field.
    • From the dropdown, choose GameObject > SetActive (bool). Uncheck the box — this will hide the panel when the button is clicked.
    • Now, pressing the Exit Button on the touchscreen will close the guide and return the user to the main UI.

    13

Toggles

A Toggle in Unity is a checkbox-like UI element that allows users to switch a setting between on (true) and off (false). In this tutorial, we’ll use a Toggle to show or hide an Assembly Instruction Text Window on the assembly station’s touchscreen.

  1. Create the Toggle Element:
    • Right-click on the Canvas in the Hierarchy.
    • Navigate to UI > Toggle.
    • Rename it to Instructions Toggle.
  2. Replace the Default Label:
    • Expand Instructions Toggle in the Hierarchy.
    • Delete the default Label child.
    • Right-click on Instructions Toggle and select UI > Text - TextMeshPro.
    • Rename the new text to Toggle Label and set it to Show Instructions.
    • Use the Rect Tool (T) to place the toggle somewhere visible on the large touchscreen (e.g., button right).

    14

  3. Create the Instruction Text Window:
    • Right-click the Canvas and select UI > Text - TextMeshPro.
    • Rename it to Instruction Text Window.
    • Enter multi-line instructional content (e.g., Step 1: Assemble the crankshaft...\nStep 2: Attach pistons to connecting rods...).
    • Resize the RectTransform to form a readable text box.
    • Set Text Overflow to Truncate or Ellipsis in the Inspector.
    • Disable this GameObject in the Inspector for now — it will be toggled on/off by the Toggle.

    15

  4. Make the Toggle Functional:
    • Select the Instructions Toggle.
    • In the Inspector, locate the Toggle component.
    • Find the On Value Changed (Boolean) event.
    • Click the + button.
    • Drag the Instruction Text Window into the Object field.
    • From the dropdown, choose GameObject > SetActive (bool).
    • When the toggle is on, the text window will be shown. When it’s off, it will be hidden.

    16

Scrollbars

A Scrollbar in Unity allows users to scroll through content that exceeds the visible area of a container. If you already have a long Text - TextMeshPro object (like assembly instructions), you can make it scrollable by embedding it in a custom Scroll View using Unity’s Scroll Rect component. Let’s wrap the existing Instruction Text Window in a scrollable container and add a vertical scrollbar to the XFactory touchscreen UI.

  1. Create the Scrollbar:
    • Right-click on the Canvas and select UI > Scrollbar.
    • Rename it to Instruction Scrollbar.
    • Move and resize it to align with the left edge of the text you want to scroll.
    • In the Scrollbar component, set Direction to Top To Bottom.
  2. Create a Masked Viewport:
    • Right-click on the Canvas and select UI > Image.
    • Rename it to Instruction Viewport.
    • In the Inspector, add a Mask component via Add Component.
    • Optionally set Image > Source Image to None for transparency.
    • Resize it to the visible area you want (e.g., same size and position as Instruction Text Window).
    • Adjust the A value under Color to make it transparent.
    • This defines the clipping region — only content inside this frame will be visible.

    18

  3. Move the Existing Text Under the Viewport:
    • Drag your existing Instruction Text Window (Text - TextMeshPro) into the Instruction Viewport GameObject (make it a child).
    • Make sure that the Instruction Text Window is larger (taller) than the Instruction Viewport.

    19

  4. Add a Scroll Rect Component:
    • Select the Instruction Viewport.
    • Click Add Component and choose Scroll Rect.
    • In the Scroll Rect component, drag the Instruction Text Window into the Content field.
    • Set Horizontal to disabled.
    • Set Vertical to enabled.
    • Leave Movement Type as Elastic (or change to Clamped if preferred).
    • The Scroll Rect now links the visible viewport with the scrollable text content.

    20

  5. Link the Scrollbar to the Scroll Rect:
    • Select the Instruction Viewport (with the Scroll Rect component).
    • In the Scroll Rect Inspector, drag the Instruction Scrollbar into the Vertical Scrollbar field.
    • The Scrollbar now controls the visible portion of the instructional text.

    21

    If it doesn’t scroll, ensure (a) the Instruction Text Window height is taller than the Instruction Viewport, (b) the text alignment starts at the top of the content area (top-left pivot), and (c) the Scroll Rect and Scrollbar are correctly assigned.

  6. Play and Test:
    • Click the “Engine Assembly Guide” button to display the engine diagrams.
    • Press the red “X” button to close the engine assembly diagram.
    • Scroll through the assembly instructions using the vertical scrollbar.
    • Toggle the “Show Instructions” checkbox to show or hide the assembly instruction panel.
    • Confirm that the scrollbar and instruction text are hidden when toggled off, and reappear correctly when toggled on.
    • Verify that all UI elements remain responsive during scene interaction.

F10

Key Takeaways

Designing effective audio and user interfaces in Unity involves more than just adding sound or placing buttons—it is about crafting an immersive, intuitive experience that responds to the user and environment. By distinguishing between diegetic and non-diegetic sound, leveraging spatial audio settings, and applying effects like reverb, developers can create rich XR soundscapes that enhance realism and guide attention. Meanwhile, thoughtful UI design—using world-space canvases, responsive anchors, interactive elements like buttons, toggles, sliders, and scrollbars—ensures controls are accessible and adaptable across devices. Together, well-implemented audio and UI elevate both the functionality and engagement of XR applications, making virtual interactions feel natural, informative, and compelling.