Positioning holograms during surgery is a major challenge when using mixed reality as the primary navigation tool. This article provides an overview of the developed adjustable frame that acts as a marker holder for mixed reality. This frame allows the technology to be applied in large quantities without the need to manufacture individual systems for each patient.
Keywords: Mixed Reality; Augmented Reality; Surgery; Hololens
To position three-dimensional holograms to a strictly defined point in space, it is necessary to use special markers, which can be represented in the form of images , QR codes  or geometric objects . In the case of using mixed reality technology in surgery, these markers must be rigidly linked to the patient’s anatomy in order to accurately position the 3D model of the anatomical structures. This can be achieved through the use of special frameholders of the marker , which are based on the individual anatomy of the patient and are made using the 3D printing method. The main disadvantage of such marker-holders is that for each patient it is necessary to design and manufacture a new marker-holder, which is time-consuming and expensive. To solve this problem, we have developed an adjustable frame (Figure 1), which is intended for performing operations on the head using mixed reality glasses . This frame fits over the patient’s head and adjusts to his individual parameters. This device is entirely made of polyamide, which allows it to be sterilized before each procedure and used repeatedly in various operations related to neurosurgery and maxillofacial surgery .
Design and Basic Principle of Use
The design of the frame is designed in such a way that it rests on the fixed parts of the patient’s anatomy, namely the bridge of the nose and ear canals. All adjusting elements are near the ears. As a result of their adjustment, the frame is firmly adhered to the patient’s head due to the tension created between the support on the bridge of the nose and the ears. The original fitting position of the frame is designed neurosurgery, however it can be placed upside down in order to open access to the face for maxillofacial surgery (Figure 2). The frame also contains radiopaque markers, which can be used to compare the position of the frame relative to the CT scan and thus calculate the exact position and orientation of the hologram relative to the marker when using mixed reality glasses. The marker itself is inserted into a special slot in the frame, which allows you to set markers of various configurations depending on the surgical access and the position of the patient during surgery.
Iterative Design Approach
At the moment, the design of the frame has undergone 2 major development iterations. On each of them, various design changes were made to improve the ergonomics and quality of positioning of the holograms (Figure 3). The first version used plastic adjusting clips with metal rods. Despite the small length and size of the threads, they gave very strong interference in CT scans; as a result, the retainers were replaced with polyamide screws with nuts from the same material. In the second version, in addition to screws, polyamide plugs were made, which were put on over the screws and increased convenience in the process of adjusting the position of this frame. Also, the frame was reinforced with stiffening ribs to reduce possible deformation as a result of tightening the screws in the ears. In addition, the design of the installation of radiopaque markers has been revised. Now they are represented by small 2x2 mm set screws. This made it possible to significantly improve the quality of calibration and the positioning accuracy of the holograms. The result of using the frame during surgery with mixed reality glasses can be seen on the Figure 4.
The adjustable frame allows surgeons to perform multiple surgeries using the same rig without creating custom system for each procedure. The latest version of the design is not final and requires some improvements. In particular, it is planned to revise the regulation mechanism to increase compactness in order to fit portable dental CT scans. We are also considering an option in which has an additional emphasis on the forehead to increase the rigidity of fixation.
- Saša Ćuković , Michele Gattullo , Frieder Pankratz, Goran Devedžić , Ernesto Carrabba, et al. (2015) Marker based vs. natural feature tracking augmented reality visualization of the 3D foot phantom. Proceedings of the International Conference on Electrical and Bio-Medical Engineering, Clean Energy and Green Computing, p. 24-31.
- Kan, Tai Wei, Chin Hung Teng, Wen Shou Chou (2009) Applying QR code in augmented reality applications. Proceedings of the 8th International Conference on Virtual Reality Continuum and its Applications in Industry.
- Adam N, D Purnamasari, Ali Ibrahim (2019) Implementation of Object Tracking Augmented Reality Markerless using FAST Corner Detection on User Defined-Extended Target Tracking in Multivarious Intensities. Journal of Physics: Conference Series 1201 (1).
- Ivanov V, Krivtsov A, Strelkov S, Gulyaev D, Godanyuk D, et al. (2021) Surgical navigation systems based on augmented reality technologies.
- Ivanov VM, Klygach AS, Strelkov SV (2021) Marker holder used for head surgery based on mixed reality. RF patent no. 202367.
- Ivanov VM, Krivtsov AM, Strelkov SV, Kalakutskiy NV, Yaremenko AI, et al. (2021) Intraoperative Use of Mixed Reality Technology in Median Neck and Branchial Cyst Excision. Future Internet 13(8): 214.