12 Nov 2026
|
Mike Boland
AWE Talks: XR and AI at Work
AWE USA 2026
Welcome back to AWE Talks, our series that revisits the best AWE conference sessions. With AWE USA recently concluded, there's a fresh batch of stage footage to keep us occupied for weeks.
We continue the action and insights this week with a look at the convergence of XR and AI in the enterprise. Where are they coming together for maximum impact? Real practitioners tell us about their deployments.
See the summarized takeaways below, along with the full session video. Stay tuned for more video highlights each week and check out the full library of conference sessions on AWE's YouTube Channel.
Speakers
Logan Lawler, Dell Pro Precision AI Solutions, Dell Technologies
Greg Barbone, Sr. Manager, XR Product and Partner Management, NVIDIA
Lisa Bell-Cabrera, Director XR Business Development, NVIDIA
Le Cong, Associate Professor | Co-Founder, Stanford University | AutoBio
Jacob Biehl, Associate Professor, University of Pittsburgh, School of Computing and Information
Key Takeaways & Analysis
- NVIDIA is building the foundation for the next generation of agentic XR experiences with its newly announced NVIDIA XR AI open-source library.
- The library provides developers with a blueprint for building multimodal AI agents designed for frontline workflows.
- It brings together NVIDIA's AI models, agents, tools, and runtime infrastructure into an accessible, integrated development framework.
- Developers can leverage voice, vision, and retrieval-augmented generation (RAG) capabilities while customizing models for specific applications.
- Designed as a companion to NVIDIA CloudXR, the library connects XR devices to powerful GPU computing resources, enabling demanding AI and rendering workloads.
- The future of XR isn't about choosing between AR, VR, mixed reality, or AI-powered wearables. It's about bringing all of these technologies together.
- NVIDIA sees different XR form factors serving different purposes, from immersive training and robotics applications to hands-free industrial workflows.
- AI-powered smart glasses are promising because they can deliver relevant, contextual information without requiring users to look down at a screen.
- Connecting these devices to more powerful compute resources will be essential to enabling increasingly sophisticated experiences.
- The University of Pittsburgh is demonstrating how mixed reality can fundamentally improve the surgical experience.
- The lab is developing technologies that combine AR, AI, and surgical expertise to augment—not replace—the surgeon.
- One groundbreaking application brings live endoscopic video, patient imaging, and real-time surgical guidance directly into a mixed reality headset during endonasal skull-base procedures.
- Rather than forcing surgeons to divide their attention among the 25–32 displays commonly found in modern operating rooms, the technology places critical information directly within their field of view.
- The team has moved beyond cadaveric studies and has completed nearly a dozen live surgeries using its mixed reality system.
- One of the most compelling benefits of surgical XR is its potential to reduce cognitive load.
- Using the NASA Task Load Index (NASA-TLX), a standardized measure of perceived workload, the research team has found encouraging results showing reduced cognitive demands when information is presented directly where surgeons need it.
- This helps address the split-attention problem created by traditional operating room displays.
- Head-mounted displays also offer surgeons a way to interact with digital information while maintaining sterile conditions, something that traditional keyboards, mice, and touchscreens cannot easily accommodate.
- Stanford University is meanwhile applying XR and AI to another incredibly complex physical environment: the scientific laboratory.
- Professor Le Cong introduced LabOS, an AI-native operating system designed to bring multimodal AI agents into physical laboratories.
- The motivation is to address one of the biggest challenges in scientific research: reproducibility.
- Experiments often involve intricate physical processes that are difficult to document, repeat, and transfer between researchers.
- LabOS aims to capture what happens in the laboratory, understand the context of experiments, and provide researchers with real-time AI assistance.
- Researchers wearing smart glasses can stream their activities directly to AI agents that interpret what's happening and provide contextual guidance.
- These agents can connect to specialized models, including Stanford's CRISPR-GPT, which supports gene-editing workflows.
- In one demonstration, a visiting student was able to perform a complex genetic experiment during her first week at Stanford, with AI guidance and results reportedly comparable to those achieved by trained researchers.
- The platform can also connect to NVIDIA's simulation technologies and robotic systems, creating a pathway toward more automated and intelligent laboratories.
- Local computing will remain critical for real-world XR and AI applications.
- Both research teams highlighted the importance of low-latency processing, particularly when AI systems need to respond to live video and physical activity.
- In surgical environments, keeping sensitive patient information local can also help address privacy, security, and regulatory requirements.
- Powerful systems such as Dell's GB300-based platform provide the computing capacity needed to run increasingly sophisticated AI models closer to where the work happens.
- As models become more efficient, the opportunity to move additional processing to local and edge devices will continue to grow.
For more context and color, watch the full session below.
Want more XR insights and multimedia? ARtillery Intelligence offers an indexed and searchable library of XR intelligence known as ARtillery Pro. Learn more here.
