ECCV 2020 Daily - Thursday

2 Poster Presentation 16 Tony Ng is a PhD student at Imperial College London, supervised by Dr KrystianMikolajczyk. His work called SOLAR, which stands for Second- Order Loss and Attention for Image Retrieval, explores two second- order components in deep learning. He speaks to us ahead of his poster presentation today. Tony’s PhD research topic is visual- based localization . The first step to do this in many pipelines is image retrieval. In image retrieval, database images are sorted according to how similar they are to a query image. A way to do that is to transform the image into an embedding through a convolutional neural network , such as ResNet or VGG , so it can do the sorting. The final product after the transformation from the image to the embedding is called the global descriptor . This work focuses on landmark retrieval . Landmarks present some specific challenges. They can all look very similar, or very different, so it can be hard to tell which class they belong to. Sometimes you can have more than one landmark in a single image. Also, the image could have very different conditions, extreme zooms, or viewpoint changes. The method described in this paper and shown in the video below uses the self-attention mechanism from Natural Language Processing (NLP) , which calculates the second-order correlation between two feature maps. It was recently introduced to the computer vision community through the non-local block. Tony explains how this could help resolve some of the challenges mentioned above: “In landmark retrieval, there are two major camps for calculating the global descriptor. One camp uses local features detected on an image and they do aggregations. That method is more accurate than the method I’m trying to tackle – which uses one single CNN and then pooling on the last feature map to get the global descriptor – but it’s slower and requires more memory. In visual localization problems , speed SOLAR: Second-Order Loss and Attention for Image Retrieval DAILY T h u r s d a y