This article was first published on Computer Vision News of May 2022.
Sharib Ali is a postdoctoral researcher at the University of Oxford and co-organizer of the MICCAI Preoperative to Intraoperative Laparoscopy Fusion (P2ILF) challenge, which aims to improve the complex task of fusing preoperative 3D CT/MRI scans with intraoperative 2D images in laparoscopic liver surgery. He speaks to us about the first edition of this exciting event.
Augmented reality-assisted laparoscopic liver surgery uses key landmark detection from intraoperative 2D video frames registered to a preoperative 3D liver model from CT/MRI data for tumor localization.
“Sometimes tumors are not visible in laparoscopy images because they’re embedded inside the liver,” Sharib tells us.
“However, these tumors can be visible in preoperative CT/MRI scans, so the idea is to fuse the two in real-time to give the precise location of the tumor during surgery. This allows the surgeon to resect it completely, reducing the risk of recurrence and ultimately saving lives.”
The P2ILF challenge asks participants to use machine learning methods for two tasks. They must first segment five liver anatomical curves, including silhouette, falciform ligament, left and right ridges, and liver boundary, from the 2D intraoperative images and preoperative 3D liver model. The second task is to register those segmented curves in 2D laparoscopy to the corresponding landmarks in the 3D model.
Most current methods use traditional computer vision methodologies to do this. Sharib approached Adrien Bartoli, Professor of Computer Science at Clermont Auvergne University, and discussed designing tasks using deep learning rather than a mathematical model.
The challenge uses single image frames, but he does not rule out using video in future editions. The plan is first to assess where the community is at in solving this kind of complex problem in the era of deep learning.
“I don’t want to scare people by saying it’s complex,” Sharib interjects.
“The only way to approach it is to truly understand the problem and what you’re trying to achieve. There are two tasks, but the registration task is particularly important and can be challenging with liver views at different positions and angles. People must take great care when finding the landmarks and understand which landmarks are critical for registration and which are not. If they always keep that in mind, they’ll be successful!”
Compared to other organs, the liver does not move much, which is one of the reasons the team picked it for this first edition of the challenge. However, preoperative to intraoperative laparoscopy fusion can be performed with other organs, and the plan is to explore this in future editions.
Sharib and his fellow organizers – Adrien, Yueming Jin, Yamid Espinel López, and Lena Maier-Hein – have put a great deal of thought into designing this challenge and curating the data. They are still working on collecting additional data from their clinical collaborators, which is all manually annotated to get the ground truth and provide a clear picture of the metric that will be used to evaluate the challenge.
The team recognizes the potential for metric failure and has introduced a 2% tolerance for the predicted anatomical curves with respect to the ground truth.
“We’re doing our best to pose a great challenge with a metric that will be clinically valuable, but at the same time, we don’t want participants to be too disheartened if it’s not working,” Sharib points out.
“You just have to be close enough, and we’ll compensate for the rest!”
The final results of the challenge will be announced at MICCAI in Singapore in September, which will be its first in-person event since Shenzhen, China, in 2019 and the first MICCAI conference hosted in Southeast Asia.
Sharib adds that we have a responsibility to tackle these complex problems together as a community.
“I’d like to invite the whole community working in this domain to contribute,” he declares.
“As a community, we can try to solve it, or at least understand where we’re at. Independently, we can do our own research going forward too, but collectively, I think we’ll have a much bigger impact on society. That’s the best way to use our time as researchers.”
Keep reading the Medical Imaging News section of our magazine.
Read about RSIP Vision’s R&D work in Laparoscopic and Robotic-Assisted Surgery.