ECCV 2018 Daily - Wednesday

Daily Tuesday 11 Jiawei He is a PhD student at Simon Fraser University. He speaks to us about yesterday’s poster session, where he presented Probabilistic Video Generation using Holistic Attribute Control, a collaboration with Disney Research, where he worked for 9 months last year. Jiawei tells us that the work is about having a controlling model to cast some control on video generation. To do that, at each time-step they have a VAE model to capture local information and they link the local information using a temporal model. He points out that video generation is still a very tough topic , but they have managed to get some very good videos. He thinks it is the first work to try to combine VAE with a temporal model. Usually when you generate content for a very large space, you have to take into consideration that the video must be consistent regarding some kind of content. For example, if you want to generate one person performing a motion, then the person should be the same in the entire video. Jiawei explains how they solved this: “ That is one of the novelties in this paper. We have something called a structured latent space. Having this structured latent space, we are able to have this holistic control inside. Having the holistic control, we can cast control on the entire sequence. By doing that, we are able to generate videos with the same person performing the motions we like .” Jiawei tells us there are plenty of possibilities for future work . One is about combining this work with natural language input. Currently, it needs some labels to have the control, but what if he just tells a person he wants to generate a sequence of them walking? They need to have this natural language transformed into the label. He thinks that would be an interesting direction. Finally, he adds: “ What I really like about this work is that we have a full page of mathematical equations to show that what we did actually makes sense. That is not what computer vision people mostly do and I’m very happy about that .” Poster Presentation Jiawei He 10 “… we have a full page of mathematical equations to show that what we did actually makes sense .”