Frequently Asked Questions about Immersive Learning with AR, VR, & AI

Your guide to hardware, use cases, research, AI-powered coaching, and more

As L&D innovators explore opportunities to leverage immersive learning for just-in-time, flow-of-work solutions, they’ve got more questions than ever about the design, development, implementation, and effectiveness of these solutions. 

Below are some of the most frequently asked questions we’ve been hearing from L&D leaders in a wide range of industries and organizations. We’ve mixed in audience questions from our live Tech Talk with Training Industry on January 13th and a few FAQs from our innovative client-partners.


1. How might we overcome hygiene and cost concerns about headsets? 

Let’s first level-set and clarify that when we’re talking about headsets, we mean a virtual reality (VR) headset that immerses the wearer in a fully digital, 360-degree 3D environment that they experience instead of the real world. 

Some ways to help with hygiene concerns include:

  • Disposable covers, similar to face masks, which create a barrier between the face and the headset.
  • Hand sanitizer stations, which are often offered at conferences and events.
  • Sanitization stations for larger deployments, allowing for quick and easy disinfection of headsets.
  • The use of augmented reality (AR) as an alternative, where participants scan a code on their smartphones or tablets to experience the solution without utilizing a shared device. They then experience digital content as a layer on top of their real environment. (The “view in room” feature on many shopping websites is a great example of AR!) 

Because participants use their own devices to enter the experience, an AR solution can also help alleviate cost concerns associated with VR hardware. So can WebXR, a browser-based immersive technology that can be experienced via headset or via browsers on a laptop, tablet, or smartphone. WebXR can be a great option for any organization to alleviate cost or hygiene concerns—or simply to offer their people options for how to consume the experience.

 

2. How do you create a 3D environment, and what is the experience like for learners? 

Here’s an answer you’ll hear a lot: It depends on the client’s needs! 

We have a skilled team of 3D technical artists who create realistic 3D environments by collaborating closely with clients to accurately depict machinery, equipment, and workspaces. 

For clients who need a hyperrealistic, extremely precise 3D digital twin of their workspace, our team visits the space with specialized equipment to measure each item and the space itself. 

For client projects that don’t need a fully custom environment, we can leverage stock 3D assets that can be adjusted, edited, and customized. This option helps us increase efficiency while reducing cost. 

The results? A fully interactive 3D environment where learners can:

  • Manipulate objects: Open valves, pick up items, and interact with the environment.
  • Experience full-body immersion: Practice skills in an authentic, immediate 3D environment.
  • Enjoy a safe practice space: For skills that learners can’t afford to get wrong—yet need to practice—a 3D environment provides a safe environment to try, receive feedback, and try again. 

WebXR, the browser-based immersive technology we mentioned in Question 1, can be a great way to scale these benefits to a geographically distributed audience. If learners enter via headsets, they can enjoy the fully immersive digital experience; if they enter via their laptop or tablet, they experience a deeply engaging, realistic experience much like a first-person video game.  

 

3. Are there any efficacy studies showing that VR training outcomes meet or exceed those of job shadowing, classroom training, or traditional eLearning?

There are a few! Below are some findings that have caught our attention. 

A comprehensive study of VR training outcomes by PwC found that: 

  • Learners trained in VR develop soft skills 4x faster than learners trained in the classroom and feel 275% more confident applying what they learned. 
  • Learners trained in VR are 3.75 times more emotionally connected to the content than classroom learners and 4x more focused than eLearning participants. 

The Imperial College of London used VR to train surgeons and discovered that 83% of learners were ready to perform procedures with little guidance after training—compared with an astounding 0% using traditional methods. 

A randomized clinical trial focusing on pathology recognition and complex procedural skills found that orthopedic surgeons trained in VR demonstrated “superior learning efficiency, knowledge, and skill transfer” compared with those trained via traditional methods. 

 The Miami Children’s Health System found that medical staff who learned new procedures in an immersive environment had an 80% retention rate one year later, compared to 20% one week later using traditional training methods.

Why is VR so effective for learning?

  • Consistency: VR provides a standardized learning experience, unlike shadowing, where individual mentors may have different teaching styles and levels of experience.
  • Repetition: VR allows for repeated practice, which is often impractical with shadowing due to time constraints and mentor availability.
  • Safety: VR allows for training on dangerous or rare situations that cannot be replicated through shadowing.
  • Scalability: VR enables a global workforce to be trained quickly and efficiently without scheduling conflicts or mentor availability issues. (Browser-based immersive technology WebXR, which learners can access via either a headset or their laptop, tablet, or smartphone, extends the benefits of scalability even further.)

True, an immersive learning strategy is an investment, but it’s a sound one: VR training has been shown to be more cost-effective at scale than traditional classroom or eLearning experiences.

  • At 375 learners, VR training achieves cost parity with classroom learning.
  • At 1,950 learners, VR training achieves cost parity with eLearning.
  • At 3,000 learners, VR training becomes 52% more cost-effective than classroom learning.

 

4. How do bandwidth and internet connectivity impact the usability of an immersive learning experience?

Native VR applications are pre-downloaded and installed on a headset, making them great for offline access. Of course, you need connectivity for the initial download and any updates but, after that, the experience can run without any internet connectivity. 

Because it’s stored and accessed from a server, AR content depends on continuous cellular or internet connectivity. The same goes for WebXR content: Because it’s accessed either via headset or web browser, it relies heavily on internet connection and speed.

Striking a balance between scalability and bandwidth can be complex—and we consult closely with our client-partners to understand their audience, their technical constraints, and their performance objectives. 

If you’re curious what an immersive learning solution could look like for your organization, we’ll work with you to design a strategy that works for you. We have extensive experience creating native VR, AR, and WebXR experiences, and even dual-delivery solutions that offer the best of both worlds.  


5. What tools do you use to build your immersive simulations?

We’re technology-agnostic, which means that we recommend solutions built with the software, game engines, hardware, and platforms that best fit your needs and technology ecosystems. For example, we leverage game engines like Unity to enable interactions with 3D objects and environments in VR and AR experiences; we often use PlayCanvas for WebXR solutions. These tools offer us the flexibility to create highly immersive, game-like simulations rather than linear eLearning-style experiences. 


6. Can accessibility features be built into immersive learning experiences to accommodate learners with different visual, auditory, and mobility abilities? What are some best practices for accessible immersive learning?

We often hear questions about designing XR experiences to meet or exceed WCAG standards; however, these standards speak more to webpages and eLearning. By prioritizing inclusive design and integrating assistive technologies, XR can better serve users with diverse abilities, ensuring a more accessible and beneficial experience for neurodiverse and differently abled learners.

To ensure that we’re designing an XR program or experience that welcomes all learners, we consider a range of accessibility needs across three main areas:  

Visual Accessibility: XR typically relies heavily on visual elements, but incorporating audio cues, haptic feedback (such as vibrating hand controls), and robust voice control—which allows learners to talk their way through the experience—can make an immersive experience more inclusive.

Mobility and Interaction Accessibility: Customized controllers, gesture recognition, spatial mapping, and more accessible interactions—such as single-click actions instead of double-clicks—can facilitate smoother interaction and movement.

Cognitive and Linguistic Accessibility: Simplified interfaces, customizable settings (for speed, difficulty, and content complexity), gesture recognition, haptic feedback to offer more guidance on what to do next, and guided tutorials can help improve understanding and engagement with immersive content.


7. How would you incorporate hand movements into an immersive learning experience?

Adding hand movements helps to create a more immersive narrative experience. Hand agency spans a wide range of gestures, including pointing, swiping, pinching, and other refined motor movements that require learners to use their hands instead of controllers. 

From simpler tools, such as paintbrushes or wrenches, to highly sophisticated surgical or mechanical tools, hand agency is a great element to leverage for highly specialized, finely detailed interactions. We’ve leveraged hand movements to help learners practice installing and repairing sophisticated machinery and performing specialized medical procedures (See Question 3 for more on why this is effective), to name a few. 

That said, hand agency isn’t limited to highly technical use cases: It’s also great in soft skills training to help learners shake hands with an avatar, hold documents, and de-escalate emotionally charged situations.    


8. What are some of the most common industries and use cases you see for immersive learning experiences? 

Once upon a time, technical training was the dominant use case for immersive learning. Imagine a digital twin of a manufacturing environment where learners could get the hang of specialized equipment, machinery, and procedures—all without harming people or property. 

This safe, immersive practice environment speeds learners’ time to proficiency, reduces errors, and enhances retention.

But as we’ve already touched on, as workplaces and job roles continue to evolve and increase in complexity, L&D leaders are embracing XR solutions for a wider spectrum of use cases, including nuanced interpersonal skills like leadership, empathy, and de-escalation. In fact, we’re now seeing a 50-50 split between hands-on technical training and soft skills training use cases. 

What are some of the most common industries and use cases you see for immersive learning experiences?

9. I’ve been hearing a lot about incorporating AI into immersive scenarios. Could you share more about how that works and what some use cases might be?  

Incorporating generative AI (genAI) into any and all of the immersive learning use cases (see Question 8) helps to increase the emotional immediacy of the learning experience, on-the-job skill transfer, and long-term skill retention…while decreasing learners’ time to competence. 

As learners interact with genAI-powered characters, coaches, and scenarios, situations evolve in response to what they say and do. There’s no “script”: Any scenario or interaction can—and does!—unfold in myriad possible ways. 

Caveat: Like any team member, genAI works best when it has a clear understanding of its role and purpose.

It’s not a substitute for humans, but it can, with careful oversight, help us extend human expertise and intelligence. (Think of it as a bright, eager, but extremely literal robot assistant that needs constant training and supervision.)

Thanks to its coachability and bottomless energy, genAI can be an effective and engaging addition to our immersive learning solutions. To help learners practice the skills they need, we can train it to serve in any of the following roles:

I’ve been hearing a lot about incorporating AI into immersive scenarios. Could you share more about how that works and what some use cases might be?

Just like our human team members, genAI also needs to be trained: Context, role, and goals are key. Equally, it needs a personality, backstory, and communication style that feels authentic, not mechanical. Getting our genAI evaluator, coach, assimilator, curator, or simulator to perform well in its role requires plenty of training—complete with trial, error, feedback, and refinement. 

GenAI should never stand alone: It’s part of a much larger digital strategy that comprises data security, ethics, privacy, value, authenticity, and commitment to unwavering human oversight. (Discover the foundations and blueprinting of a sound genAI strategy with our comprehensive playbook.) 

(Wondering what a genAI-powered coaching experience might look, sound, and feel like? Check out Hilton’s genAI-powered, immersive guest service training experience for its global family of hotel team members.)


10. I’d love to add immersive experiences to my learning portfolio, but I need to work within my organization’s LMS. What options do I have?

This is a very common constraint, and we’ve got a fix for it! It’s called the WebXR LMS Integration Tool, or LIT. Its superpower is integrating easily with your LMS platform to allow WebXR experiences to launch directly from your LMS—helping you create a one-stop shop for all of your learning solutions. 

Thanks to LIT, WebXR experiences can launch directly from existing eLearning courses. At launch, LIT cues learners to choose their device: headset or computer.

I need to work within my organization’s LMS. What options do I have?

After selecting their devices, learners proceed seamlessly into the WebXR experience. While they’re in the immersive experience, LIT “recognizes” them and brings their performance metrics back to your LMS, creating a continuous, seamless experience on the front and back ends.

LMS Learning System

Which data does it collect? That depends on your needs! We work with you to determine the learner performance data that matter most to you—including complex performance data that your LMS may not be able to process. Using our LIT Analytics Dashboard, we can capture and display such detailed learner metrics as tone-of-voice analytics, gaze tracking, and reaction time. 

If you’re wondering whether LIT is compatible with your organization’s LMS, our working answer would be “yes”! We’ve deployed WebXR content on hundreds of LMSs, all with their own complexities, quirks, and security protocols. 

Got an LMS challenge, immersive or otherwise? Our in-house Director of Systemic Solutioning, John Cleave, would love to hear more about your current learning and IT ecosystem and how we might help you build, in the words of one client-partner, “with, through, over, around, and under” your existing environment.

 

11. Questions about immersive learning and AI strategy?

Wondering how immersive learning solutions for a variety of use cases might look, sound, and feel? Check out the webisode featuring no fewer than eight real-life immersive simulations we’ve created with some of our most innovative client-partners. (Yes, AI is involved!)

If you’ve got a question of your own about immersive learning and AI strategies, technologies, or scaling, we’re all ears! We’re incredibly excited about fresh opportunities to bring these effective, engaging, and high-quality experiences to new learner audiences, and we’d love to chat about the possibilities.

Training Industry Honors SweetRush for 3 Consecutive Years: A Leader in Experiential Learning

San Francisco, Calif., September 11, 2024 — Good things happen in threes for SweetRush’s Emerging Tech team, who are currently celebrating their third consecutive year on Training Industry’s list of Top Experiential Learning Technologies Companies

Experiential learning technologies include modalities such as virtual reality (VR), extended reality (XR), simulations, and serious games, which can be leveraged to meet such diverse learning needs as medical, compliance, manufacturing, and soft skills training.

Training Industry uses the following criteria to evaluate the Top 20 Experiential Learning Technologies Companies:

  • Breadth, quality, and advancement of features, capabilities, and analytics
  • Industry visibility, innovation, and impact in the learning technologies training market
  • Client and user representation
  • Business performance and growth

SweetRush Solution Architect Danielle Silver characterizes the honor as “a testament to the brilliant and forward-thinking client-partners who inspire us to push the boundaries of what’s possible. Crafting immersive and AI-powered learning experiences is not just our job, it’s our passion.”

Indeed, 2024 has been a year of innovation for SweetRush and its world-class client-partners. Leveraging the power duo of generative AI (genAI) and web-based extended reality (WebXR) technologies, the SweetRush XR team has helped L&D innovators in a wide range of industries bring high-impact learning, coaching, and skilling experiences to a wider audience than ever before. 

Director Adrián Soto and team have also been engaged in developing custom enterprise AI training programs to help forward-thinking organizations future-prep their people. These programs model responsible, ethical AI usage via AI coaching, gamified branching simulations, and other experiential learning modalities. 

In addition to earning the interest of learners, these solutions have also garnered attention from an industry audience. The SweetRush Emerging Tech team is enthusiastic about the AI opportunity and sharing their innovations in AI-powered and immersive learning via webinars, presentations, and publications. (Visit the SweetRush YouTube Channel for recordings of these live events.) 

Most recently, the team authored a comprehensive eBook, titled The GenAI Playbook: An L&D Innovator’s Guide Leading-Edge Learning Transformation. “The Playbook doesn’t just talk about possibilities, it gives you a roadmap. The use cases and project lookbook show you exactly how genAI can transform L&D, from creating better learning experiences to getting your teams ready for the AI-driven workplace”, describes Annie Hodson, Chief Client Solutions and Marketing Officer.

SweetRush’s consultative approach to custom experiential learning design, which one client describes as “second to none,” addresses organization-wide business and learning needs through beginning-to-end strategy mapping. Leading organizations such as Hilton, Coursera, Bridgestone, and the American Academy of Family Physicians (AAFP) are pioneering leading-edge experiential learning solutions that engage diverse, globally distributed learner audiences in skill acquisition and practice. 

These partnerships result in creative, compelling solutions that make learning an experience to remember—and have been recognized by 157 Gold Brandon Hall Awards and 10 Gold CLO Learning in Practice Awards to date.

If you’d like to create rich, resonant, and award-winning experiential learning of your own, please reach out to be connected with an expert learning consultant.

 

About SweetRush

SweetRush is trusted by many of the world’s most successful companies to help them improve the performance of their employees and extended enterprise. SweetRush is known for exceptionally creative and effective solutions that combine the best of learning experience design with highly engaging delivery. 

SweetRush services include custom L&D solution design and development, high-performing staff-augmentation talent, certification development, and innovative learning technologies such as VR, AR, and AI. SweetRush’s work has earned a long list of awards and accolades in collaboration with its world-class clients. Discover more at www.sweetrush.com

 

Generative AI + Immersive Training: SweetRush’s Power Duo for Effective, Engaging Guest Service Training at Hilton

An insider’s look at leading-edge learning innovation at Hilton

We’re entering our second year of generative AI buzz, but organizations are just beginning to scratch the surface, with most using the technology to create first drafts, identify data patterns, and summarize lengthy texts. Meanwhile, they’re leaving nuanced human pursuits like critical thinking, creativity, relationship building, ethics, and navigating complex and ambiguous situations to…well, their humans. 

Training is another pursuit that’s been left to humans: None of the 1,600+ respondents to a global survey on the state of generative AI called out training as a use case. 

This vast, virtually untapped opportunity is what makes generative AI so exciting for L&D innovators—like our friends at Hilton. Read on to discover how the Hilton-SweetRush team set out on an exploratory mission into the heart of generative AI… and emerged with a leading-edge immersive learning experience.

Generative AI: Hilton’s L&D Use Case

With a robust portfolio of effective, engaging WebXR and native VR learning solutions, the Hilton-SweetRush team were already veteran immersive learning creators. Amazing experiences such as Hotel Immersion and Exceed with Empathy (see below) place learners in someone else’s shoes—and drop them into interactive, day-in-the-life scenarios that help them gain insight and empathy.

 

Generative AI: Hilton’s L&D Use Case by SweetRush

Learner Audience: Newly hired team members 

Purpose: Help learners gain empathy for hotel operations team members.

The Experience: The Hilton-SweetRush team wanted newly hired team members to experience a day in the life of the hotel operations team members who drive results via brand standards. In this energetic simulation, learners land in a Hilton hotel—and work against the clock to assemble room-service trays, clean a guest room, and check guests in at the front desk. Learners access this experience via a headset, which gives them the freedom to move around and interact with objects in these 3D spaces.

Generative AI: Hilton’s L&D Use Case by SweetRush

Image: 94% of learners reported increased empathy for hotel operations team members as a result of their Hotel Business Immersion experience

Learner Audience: Hotel operations team members

Purpose: Help learners develop empathy for guests. 

The Experience: Learners step into the shoes of a guest waiting in a long line to check in, after hours of travel; a guest who’s served the wrong order at the restaurant; and another who finds that the conference room they’ve booked is improperly set up. After each experience, they reflect on their mood, gain greater insight into the guest experience, and discover how to exceed guest expectations.

Image: 95% of learners agree they had more empathy for hotel guests as a result of their Exceed with Empathy experience.

In spring 2023, the team began to wonder how they might leverage generative AI to take Hilton’s immersive learning game up a notch. The timing was right, and so was the learning need: Create an engaging, effective, and scalable immersive experience to help hotel operations staff practice their guest service skills. 

Read on for background about the business and learning needs—and how the team met them with creativity, ingenuity, and industry-first technology. 

Hilton’s Business Need: Deliver a Reliable and Friendly Stay

Guest feedback shared the wish for a reliable, consistent, and friendly stay, where team members treat every guest like a friend. Thus the powerful impetus for Hilton’s leading-edge guest service skills training was born. 

This impactful learning journey, called Make It Right, features a variety of bite-sized eLearning modules, pre-shift team meetings, videos and more.  

One part of Make It Right focuses on service recovery with HEART, Hilton’s five-step problem-resolution model (see below). 

Service recovery offers Hilton team members a powerful opportunity to show guests how much they care-and can even elicit more guest loyalty than delivering a perfect stay in the first place (a phenomenon known as the Service Recovery

Generative AI: Hilton’s L&D Use Case by SweetRush

At this stage in the learning journey, Hilton team members are equipped with a good understanding of HEART. But knowing the five steps and their purpose and putting them into practice—especially under pressure—are two different things. 

Like so many interpersonal skills, service recovery requires plenty of trial, error, and feedback before learners achieve competency. While role-plays are one way to achieve this, Hilton chose a virtual environment to build confidence with their teams during their training process.  

Would this catch-22 derail the opportunity for hands-on practice of the HEART model? Not for the Hilton-SweetRush team!


WebXR and Generative AI: Scalable Immersive Learning for Hilton’s Global Audience

The Hilton-SweetRush team rose to the challenge by creating a capstone experience that brought hands-on HEART practice to team members across all its hotels

How? By leveraging WebXR, a super-scalable virtual reality learning technology. 

Because WebXR content lives on a web browser and can be accessed by learners with or without a headset, it’s an ideal modality to bring the benefits of virtual learning to a distributed global audience like Hilton’s. 

Generative AI: Hilton’s L&D Use Case by SweetRush

Helping Hilton team members speak from the heart was the perfect learning use case for WebXR. To practice these vital service recovery skills, they created the Delivering on Our Customer Promise experience. This WebXR- and AI-powered simulation drops learners into three different scenarios with concerned guests, each of whom explains their problem and waits expectantly for a response.

 

Artificial Inteligence Training by SweetRush

These scenarios make learners sweat a little, just as real service recovery situations do. Yet rather than risk real relationships with Hilton guests, they find their feet—and voices—within a safe, authentic virtual practice space.

 

Generative AI: Hilton’s High-Tech Human Skills Coach

Hilton’s Delivering on Our Customer Promise WebXR adventure was already a leading-edge learning experience—but the team’s addition of generative AI lands it among L&D industry firsts. 

Here’s how generative AI helps learners make it right: After listening to the guest explain their problem, learners speak their response into their device’s microphone, using the HEART steps they’ve learned.

The simulation records their response, converts it into text, and feeds the text of the response into a Large Language Model (LLM). On the back end, the LLM, which has been carefully “trained” by SweetRush SMEs, creates a response to the learner’s attempt. (Discover how SweetRush’s exacting SMEs systematically trained the LLM to evaluate learner input and deliver relevant, actionable feedback.)

Artificial Inteligence Training by SweetRush
Learners receive feedback and a grade of pass or fail on each step of Hilton’s HEART model. Though most learners can tackle the HART, most find the E, or Empathize, to be a challenge.

After completing each scenario, learners can then reattempt it as many times as they like, receiving targeted feedback each time.

Learner Experience (LX): Designed by Experts, Hosted by VIC

To create the greatest impact, the team focused the WebXR experience on three different scenarios, all based on top guest problems: 

  • Room climate controls
  • Food and beverage options
  • Special room requests not met

Using their best storytelling and dialogue-writing practices, the team built branching scenarios featuring a guest interaction in different areas of the hotel. These in-room, front-desk, and breakfast-area conversations could end with positive, negative, or neutral results based upon the learner’s input. 

The instructional designer worked closely with Hilton SMEs to craft realistic guest scenarios—and responses that matched Hilton’s high standards and signature service recovery methods.

 

Artificial Inteligence Training by SweetRush
This guest visits the front desk to express frustration with the air conditioning unit in her room. It rattled all night, and she didn’t get any sleep. The learner’s challenge: Respond with HEART.

 

Artificial Inteligence Training by SweetRush
This guest has been working on a presentation all morning and missed breakfast. Learners think on their feet to offer a heartfelt resolution.

 

Artificial Inteligence Training by SweetRush
This guest hasn’t received the in-room amenities he requested when he made his reservation. The learner has only moments to Make It Right.

Making It Look, Sound, and Feel Right: Designing in 3D

Appearance matters for a 360° immersive experience, and a realistic setting is an important part of creating an authentic immersive scenario. The team custom-created these lifelike settings by taking photographs at real Hilton properties—and the SweetRush extended reality (XR) team built depth and detail into each scene. 

The guests’ avatars needed to be as diverse and expressive as Hilton’s guests are in real life, with the tone of voice, gestures, facial expressions, and posture that reflects their mood and level of satisfaction. 

Why this painstaking care? To truly immerse learners, they need to feel that they’re face to face with a real guest in a real Hilton hotel.

Because of the complexity of the scenarios and their different potential outcomes, the creative director, instructional designer, and XR team worked closely together to ensure that the content, audio, and visual treatment informed one another. 

Artificial Inteligence Training by SweetRush


Like any great guest experience, the HEART model training
experience needed a great host who would encourage learners and serve as their guide throughout all three scenarios. VIC, Hilton’s friendly robot emcee (and breakout star of previous VR experiences), was up to the task, hovering in the scene with helpful instructions, a HEART walkthrough, and learner feedback. (Join us for a demo of this leading-edge immersive AI coaching experience at Training Industry’s Leader Talk: Using AI to Transform Content Development.) 


Leading-Edge Learning at Your Organization: Leverage Generative AI Coaching and Immersive Training for Your Unique Needs

Feeling inspired by Hilton’s generative AI-powered, immersive learning VICtory? We’re with you!

We’ve loved sharing the story of how one innovative client-partner leveraged the leading-edge powers of WebXR and generative AI to meet Hilton’s deeply human business need: a warm welcome and friendly stay for every guest. 

If you’re curious about how to apply WebXR and generative AI to your organization’s unique needs and learners, we’ve got you covered!