Yet, Stereo Transformer can only extract sparse information of geometries. Thus, Max introduces Neuralangelo (Li et al , CVPR 2023) to recover dense and accurate geometry from casually captured videos. Neuralangelo enables high-fidelity surface reconstruction, achieving sub-millimeter accuracy for surgical procedures. It also allows users to reconstruct largescale scenes, such as creating virtual models of JHU campus (Fig. 1). Neuralangelo represents a critical step towards building virtual representations of real objects. Max’s research also explores tracking objects in 3D space over time. He proposes the TAToo algorithm (Li et al , IPCAI 2023) to track the surgical tool and patient simultaneously from video input. TAToo employs an end-to-end differentiable pipeline with geometric optimization to achieve millimeterlevel tracking accuracy. TAToo has the potential to replace commercial motion capturing devices like optical trackers to make motion tracking more accessible. With the advancements mentioned above, Max has pioneered the concept of digital twins, where virtual models fully mimic real-world processes. In collaboration with others, Max co-developed the Twin-S system (Shu et al , IPCAI 2023), which models surgical processes with great accuracy (Fig. 2). This serves as a foundation to enable next generation of surgical guidance systems, where users are provided with computational analysis that is otherwise hard to obtain. Looking ahead, Max is eager to explore more advanced systems that not only recover the motion and structures from video input, but also understand 3D geometries. Such a system can truly democratize the creation of metaverse and autonomous systems. For more information, seeMax’swebsite. 29 Max Zhaoshuo Li Computer Vision News
RkJQdWJsaXNoZXIy NTc3NzU=