ICCV Daily 2021 - Wednesday

When they started the project, Viktor and Vlad realized there was not much work on 3D reconstruction of deformable or non-rigid objects from event streams . Previously, there was work on tracking of human bodies from an event camera, and hands represented an additional challenge because there are more severe self-occlusions, and the degree of articulation is higher. From a scientific point of view, this was a challenge which required creating a new dataset to look at the problem of domain gap when training on the synthetic event stream and wanting to generalize to the real event stream. “ It was very interesting for me to start a computer vision project because I was mostly working in theory before, ” Viktor reveals. “ This is actually my first attempt at a computer vision paper. It seemed to be a nice topic because event cameras had existed for a long time, but there had not been much work on learning-based methods . Then there was a very nice question that since the data format is so abstract, would it be able to generalize to the real data? It turned out that there’s not much of a domain gap after all between the synthetic data and the real data. I was so excited to make a live demo to see our results and play around with the model without having to retrain it every time or produce the results offline. ” In terms of next steps, both Viktor and Vlad are in agreement that they would like to add a second hand to this work. Also, robustness to the background motion because it is rare that you have a hand floating in the air without a body behind it. 16 DAILY ICCV Wednesday “The main challenge is that conventional computer vision methods can’t be applied to this new data modality…” Poster Presentation