Metaverse, a convergence of virtual and physical space, is changing medicine.

Seoul National University Bundang Hospital (SNUBH) built the metaverse platform in a smart operating room.

The word metaverse is made up of the prefix “meta” meaning virtual and “universe,” meaning the real world. It is a more advanced concept than the existing virtual reality (VR). The core technology of the metaverse is extended reality (XR), which encompasses VR, augmented reality (AR), and mixed reality (MR).

On May 29, the Asian Society for Cardiovascular and Thoracic Surgery (ASCVTS) provided lung cancer surgery training through a metaverse platform at an online conference. The surgery took place at the SNUBH’s smart operating room, and over 200 Asian thoracic surgeons received the training in a virtual environment. All the equipment they needed was a head-mounted display (HMD).

Recently, the upgraded platform does not even require HMD. Surgeons can see the operating room with a 360-degree camera through a laptop.

Professor Jheon Sang-hoon of the Thoracic and Cardiovascular Surgery Department at the SNUBHl, who is president of ASCVTS, pushed to install the smart operating room when he was president of the SNUBH.

The smart operating room made it possible for ASCVTS to offer such training.

The hospital completed the smart operating room to produce 4K or 3D video and 8K VR video using 360-degree cameras in 2019.

In the smart operating room, surgeons can perform image-guided surgery (IGS) using near-infrared rays and displays that simultaneously support 4K and 3D surgical endoscopes.

They can also share opinions in real-time using a “Tele-Pathology” system.

After experiencing the “reality” of VR realized at the SNUBH’s smart operating room through ASCVTS, the Singapore General Hospital decided to apply the system to create a similar operating room.

In other words, the SNUBH exported the smart operating room to Singapore.

The SNUBH plans to use the metaverse in local medical training – not only for clinical training for medical school students but for surgical training for interns, residents, and fellows. As early as the second half of the year, the SNU College of Medicine will provide the metaverse clinical training.

In an interview with Korea Biomedical Review, Jheon said the metaverse applied to various clinical settings and medical training.

However, the nation’s outdated health insurance program is not catching up with rapid healthcare technology changes, which could hamper advancement in clinical care, he said.

Question: You provided lung cancer surgery training in the VR realized by the metaverse in the online conference of the ASCVTS. The training was organized by the Automotive Technical Education Program (ATEP), which you established.

Answer: I established the ATEP in 2011 to teach video-assisted thoracic surgery to Asian surgeons. We held workshops once or twice a year, mainly for those from developing countries. We rented education centers in Korea or Japan to provide surgical training for two days free of charge in the early days. Forty people in groups of two received surgical training using animals. However, this kind of training costs quite a lot.

Q: Did the ATEP change the training method from offline to online due to the Covid-19 pandemic?

A: After I took office as president of ASCVTS in 2019, I started ATEP’s surgical training in the lung field. We introduced VR in surgical training because offline surgical training costs a lot. Then, the outbreak of Covid-19 pushed us to offer timely VR education.

Trainees needed only HMD, so ATEP purchased 40 sets of HMD and sent them to Vietnam once. In late June, we will give one more training session in Moscow.

The technology has advanced compared to the early days. So, surgeons can experience the VR 360-degree through a laptop without HMD.

Q: Live broadcasting of the surgical procedure has been around for a long time. What is different about VR training realized by the metaverse?

A: It shows the surgical scene perfectly. Instead of offering one scene unilaterally, it can show you everything happening in the operating room if you put on HMD. Trainees can see what anesthesiologists and nurses are doing in more detail than in the operating room. This is possible because of XR technology, and it helps trainees to concentrate more.

Q: Is the metaverse used for training for specialists or residents?

A: Residents do not have many chances to operate these days. The thoracic surgery department normally uses animal hearts or lungs to educate specialists, but this is also not easy. So, we provide surgical training virtually.

Technology has been so advanced that it can even reproduce tactile sensations in virtual environments. We are conducting a pilot study to apply the metaverse to the clinical practice education for students at SNU College of Medicine.

In the second half of this year, we plan to apply it to clinical training. Under the MOU between SNU College of Medicine and the National University of Singapore and between SNUH and the Singapore General Hospital, the National University of Singapore will use the metaverse in medical education. We are also making VR for intensive care units where clinical training opportunities are scarce.

Q: What other healthcare areas can you use the metaverse except for medical training?

A: There are many. For example, patients on chemotherapy play with their mobile phones or read magazines while getting the injection. But their minds will be somewhere else because they are anxious. So, we’re preparing a trial to realize a virtual environment where cancer patients can receive chemotherapy comfortably. During a trial, clinicians will identify and solve problems and find a good technology while doing so.

Q: Some said the medical community’s opposition to telemedicine could hamper Korea’s technology advancement. What do you think?

A: Providing education in a virtual environment through VR or XR technology is one thing, and telehealth is another. But we need to introduce telemedicine. So, we have to discuss sufficiently what merits telemedicine could bring and what the potential risks are. We can’t oppose it indefinitely.

The medical community and the government, and citizens should make efforts to solve problems one by one. If you think doctors oppose telemedicine to keep their vested interests, there will be more misunderstandings. The times are changing.

Q: Are there any institutional areas that need to be improved?

A: The government says it will ease regulations, but it makes another regulation to do so. When it comes to national health insurance, it’s more frustrating. Rather than deciding on something and announcing it first, the government should listen to what clinicians have to say. If the government says doctors oppose telemedicine “to keep vested interests of doctors’ group,” the damage will be inflicted on the public. It is the same with the controversy of installing mandatory CCTV surveillance cameras in operating rooms.

In Korea, medical care is viewed as a public good. So, the government controls reimbursement rates. If doctors provide new technology for patients, the government should guarantee it with the health insurance program. There should be a method to introduce new technology into the existing system so that medical institutions can use it more actively. Otherwise, medical institutions would find it hard to invest in developing new technology and applying it to clinical care. National university hospitals may do so out of commitment, but other private hospitals may not.