Mixed reality and reverse shoulder arthroplasty

Correct glenoid component positioning is the cornerstone of reverse shoulder arthroplasty (RSA), as it dictates the implant longevity and functional outcome. However, the surgical exposure of the glenoid is a complex undertaking faced by surgeons, especially when glenoid morphology is deformed. The more limited the intraoperative view of the glenoid becomes, the greater the risk of implant malposition and early implant failure. But what if mixed reality (MR), a technology blending the digital and physical worlds, could help the surgeon to see beyond the wall? Could a technology derived from the gaming industry reduce the gap between preoperative planning and surgical execution and, in turn, improve the accuracy of implant placement? In this article, Ashish Gupta, Professor at Queensland University of Technology, Brisbane, Australia, shares his tips and tricks about how he has integrated MR into his practice—from preoperative planning to the operating room—and how it is transforming revision RSA into a 1-stage procedure and offers an excellent training tool for surgery residents .

Ashish Gupta

Ashish Gupta

Queensland Orthopedic Clinic
Brisbane, Australia

Mixed reality in the operating room

The first total shoulder arthroplasty combined with MR was performed in 2018 by Gregory et al [1], who concluded that the real-time visualization of the operation keyframe (ie, muscles, blood vessels, and bones) by three-dimensional (3D) holograms improved the safety and accuracy of the procedure. Contrary to virtual reality, where the user is fully immersed in a digital world, MR overlays a 3D reconstruction of a patient's scapula onto the surgical field, providing surgeons with access to the hidden part of the bones [2]. Alongside a computer for 3D rendering, the required equipment is as simple as an optical see-through head-mounted display (OST-HDM) with visual and auditory cues, such as HoloLens 2 (Microsoft, Redmond, Washington, USA). This self-contained and wireless device allows the surgeon to gain access intraoperatively to patient-specific information or preoperative plans via the headset with oral commands or a simple hand gesture. To this aim, a 3D model of the patient's scapula is generated based on CT scans, which is then downloaded to the headset before the operation begins [1]. In the operating room, navigation is based on Quick Response (QR) codes, which brings the advantage of aligning the hologram and the surgical field in the same line of sight. [3]. Moreover, the visualization of 3D holograms in real time allows a step-by-step comparison of the surgical procedure with the preoperative plan while providing guidance in implant positioning [1]. Despite the many benefits associated with this technique (Table 1), it also has its limitations. According to Ashish Gupta, "the main limitation of this technology comes with the lenses. Originally designed as a videogame device, the HoloLens 2 camera requires further development to be adapted to clinical medicine". A better intraoperative registration could be achieved by reducing the size of the QR codes used for navigation. However, as he adds, "we can create QR codes as small as 5 mm, but the HoloLens' camera cannot perceive them". This opinion is also shared by Schlueter-Brust et al [4], who highlighted the lack of calibration accuracy between the virtual hologram and the real scene as the main limitation, hindering MR from becoming a scalable tool for surgery. An additional problem underlying the use of MR in the settings of RSA arises from the lighting conditions of the operating room. As Gupta explains, “surgical illumination is designed to maximize visibility at the surgical site and prevent eye fatigue. But how to project a 3D hologram in such bright conditions without disrupting the visibility of the surgical field?”. While he has been working with his multidisciplinary team to overcome the main caveats of MR in the operating room, he is convinced that the advantages of introducing MR in shoulder arthroplasty greatly outshines the drawbacks associated with this technique. As he explains in the following sections, “MR not only contributes to improved implant accuracy, but it is also an excellent surgical training tool that helps me enhance preoperative planning while being unexpensive”.

Read the full article with your AO login

  • Benefits and limitations of mixed reality
  • Implant component positioning and mixed reality
  • Mixed reality and surgical training: a technique with a 45-minute learning curve
  • Can mixed reality enhance preoperative planning?
  • A cost-effective technique
  • Conclusions and future perspectives

Part 1 | Adopting lateralization techniques

Part 3 | Preventing infection

Additional AO resources

Access videos, tools, and other assets.

Contributing experts

This series of articles was created with the support of the following specialists (in alphabetical order):
Ashish Gupta

Ashish Gupta

Queensland Orthopedic Clinic
Brisbane, Australia

Simon Lambert

Simon Lambert

AO Recon Shoulder Curriculum Taskforce
University College London Hospital
London, United Kingdom

Markus Scheibel

AO Recon Shoulder Curriculum Taskforce
Schulthess Clinic
Zurich, Switzerland

Jan-Philipp Imiolcyk

Jan-Philipp Imiolczyk

Charité—University Medicine
Berlin, Germany

This issue was written by Antia Rodriguez-Villalon, AO Innovation Translation Center, Clinical Science, Switzerland.


  1. Gregory TM, Gregory J, Sledge J, et al. Surgery guided by mixed reality: presentation of a proof of concept. Acta Orthop. 2018 Oct;89(5):480–483.
  2. Lohre R, Warner JJP, Athwal GS, et al. The evolution of virtual reality in shoulder and elbow surgery. JSES Int. 2020 Jun;4(2):215–223.
  3. Kriechling P, Roner S, Liebmann F, et al. Augmented reality for base plate component placement in reverse total shoulder arthroplasty: a feasibility study. Arch Orthop Trauma Surg. 2021 Sep;141(9):1447–1453.
  4. Schlueter-Brust K, Henckel J, Katinakis F, et al. Augmented-Reality-Assisted K-Wire Placement for Glenoid Component Positioning in Reversed Shoulder Arthroplasty: A Proof-of-Concept Study. J Pers Med. 2021 Aug 10;11(8).
  5. Kriechling P, Loucas R, Loucas M, et al. Augmented reality through head-mounted display for navigation of baseplate component placement in reverse total shoulder arthroplasty: a cadaveric study. Arch Orthop Trauma Surg. 2023 Jan;143(1):169–175.
  6. Rojas JT, Ladermann A, Ho SWL, et al. Glenoid Component Placement Assisted by Augmented Reality Through a Head-Mounted Display During Reverse Shoulder Arthroplasty. Arthrosc Tech. 2022 May;11(5):e863–e874.
  7. Erickson J, Batko BD, Schneider G, et al. eds. Mixed-reality holographic-assisted placement of glenoid guidewire in shoulder arthroplasty: preliminary comparison to patient-specific instrumentation in B2 glenoid model. Seminars in Arthroplasty: JSES. 2022: Elsevier.
  8. Gupta A, Launay M, Lane A, et al. eds. Validation of Mixed Reality Technology as a Guidance System in Shoulder Arthroplasty. 30th SECEC-ESSSE Congress; 2022 September 7–9; Dublin, Ireland.
  9. Lohre R, Bois AJ, Athwal GS, et al. Improved Complex Skill Acquisition by Immersive Virtual Reality Training: A Randomized Controlled Trial. J Bone Joint Surg Am. 2020 Mar 18;102(6):e26.
  10. Chalmers PN, Boileau P, Romeo AA, et al. Revision Reverse Shoulder Arthroplasty. J Am Acad Orthop Surg. 2019 Jun 15;27(12):426–436.
  11. Boileau P. Complications and revision of reverse total shoulder arthroplasty. Orthop Traumatol Surg Res. 2016 Feb;102(1 Suppl):S33–43.
  12. Italia KR, Green N, Maharaj J, et al. Computed tomographic evaluation of glenoid joint line restoration with glenoid bone grafting and reverse shoulder arthroplasty in patients with significant glenoid bone loss. J Shoulder Elbow Surg. 2021 Mar;30(3):599–608.
  13. Gill DRJ, Page RS, Graves SE, et al. The rate of 2nd revision for shoulder arthroplasty as analyzed by the Australian Orthopaedic Association National Joint Replacement Registry (AOANJRR). Acta Orthop. 2021 Jun;92(3):258–263.
  14. Italia K, Launay M, Gilliland L, et al. Single-Stage Revision Reverse Shoulder Arthroplasty: Preoperative Planning, Surgical Technique, and Mixed Reality Execution. J Clin Med. 2022 Dec 14;11(24).