Computer Vision News - November 2022

29 Hermann Blum classes. To counter this, Hermann worked on SynBoost, a wrapper method that is still the SOTA for detecting anomalies from given segmentation networks without messing with their training. The second part of the thesis follows the question how robots can improve their semantic segmentation autonomously by combining recent advancements in self-supervision and continual learning. Through self-supervision, perception systems can be trained without human input. That makes it possible to adapt these systems to the robot’s environment during autonomous deployment. On the other hand, continual learning methods make it possible to integrate all the knowledge a robot gathers when deployed to different environments or executing different tasks sequentially. By combining the two, the thesis studies a robot that from experience alone builds up a perception system that works well in all explored environments. The advantage of continual learning shows up in those results where a robot that has already seen some other environments will adapt better to a target than an inexperienced robot. It all comes together in a final system that can incrementally learn those parts of a scene that were identified as unknown by an anomaly detector. On hermannblum.net you can find many videos of real-world robotic experiments related to these works. Figure 1: Overview of Hermann’s approach to self-improving robotic perception. In this example, the segmentation model has never seen TVs before. Consequently, they are detected as anomalies and a self-supervised pseudo label is created that adds TVs as a new category.