Daily CVPR - Wednesday

CVPR Daily: What algorithms do you use in order to solve this problem? Kevin: We used convolutional neural network along with an optimization that is reminiscent of coordinate ascent and uses logistic regression as well. CVPR Daily: What was particularly difficult in this work? Kuan Fang: The traditional methods to tackle this problem are using vanishing point detectors. So they first have these geometric constraints and gather those vanishing points, lines, and edges. Then they get the 3D room layout. We tackled this using fully- connected neural networks. The output of the neural networks doesn’t have a geometric constraint. So we found ways to shrink a good neural network to give us a per-pixel probability of the semantic labels of the room and also have a corresponding refinement staff to observe geometric constraints to gather the final layout. CVPR Daily: What is the next step in this work? Kuan Fang: There are several steps. One thing is to estimate the layout from a single image, but we are considering using other techniques like structure from motion to estimate a more accurate layout. We are also trying to make it faster. So currently it will take 15-16 seconds for a single image, but we want to make it faster or even real-time... So that you can just turn on your camera, have a video of your room, and you generate a 3D layout in real-time That would be really cool and useful for a lot of things. So potential application could be that you could use this 3D layout and 3D models for let’s say IKEA or another furniture company, so that they can provide you with some 3D models. You can just augment reality using this 3D layout and 3D models. You don’t want to buy furniture you don’t like, then try it out in your room, and return it, but you can simulate it to get an Augmented Reality effect. Presentations 7 CVPR Daily: Wednesday “We found ways to shrink a good neural network to give us a per-pixel probability of the semantic labels of the room ”