This process ensures the laser point is visible on the tissue surface despite the ambient lighting conditions. “Our network includes two branches,” she explains. “For the first branch, the images fed to the network were the ‘laser off’ stereo RGB images, but crucially, the intersection points for these images were known a priori from the paired ‘laser on’ images. Then, we use the PCA, the Principal Component Analysis, to extract the central axis of the probe on the 2D. Then, we want to feed this information to the second branch. We sampled 50 points along this axis as an extra input dimension.” The network employed ResNet and Vision Transformer as backbones, and the principal points were learned through either a multi-layer perceptron (MLP) or a long short-term memory (LSTM) network. These features from both branches were then concatenated for regressing the intersection point, with the network being trained end-to-end using the mean square error loss. “Since it’s important to report the errors in 3D and millimeters, we also recorded the ground truth depth data, just for evaluation, for all frames,” Baoru adds. “We used a custom-developed structured lighting system and the corresponding algorithms developed by us. With 23 different patterns projected onto one frame, we can get the depth map for this frame. We’ve released this dataset to the community.” Overall, what makes this work truly special is its innovative use of the gamma probe to detect gamma signals and locate cancerous tissue, enhancing the accuracy of resection and diagnosis. Moreover, its ability to enhancing 10 DAILY MICCAI Monday Oral Presentation
RkJQdWJsaXNoZXIy NTc3NzU=