Computer Vision News - September 2018

13 Application: ZOOX Computer Vision News Application Here, it’s different. You always have to do the right thing. You can’t fail. You just want to get to the point where you leverage all availabilities as good as you can for a system that is both high on recall in detecting an object and doesn’t make any mistakes. ” Currently, Sarah works more with the vision team while David spends more time focusing on lidar. Both methods use similar techniques, and at the end of the day, they work as one big perception team . How do they do it? Well, therein lies the real challenge. David expands on how they leverage each modality to the fullest so that they can compensate for the others’ limitations. For example, lidar has great depth, but sometimes struggles with classifying objects, particularly ones that are far away. During detection, cameras can then figure out the object's type. Then lidar assists further in providing a better understanding of depth. At the same time, radar establishes the velocity and improves tracking of moving objects. “ We try to leverage different sensors in terms of their strengths to enable more robust detection and tracking ” David explains. In terms of the challenges in vision, Sarah adds: “ I think that what might be challenging for all of us is long-tail events. We’re using a lot of neural networks. These are obviously very good at things that we’ve seen and understood well. They work well in detecting most normal situations. It’s really planning for these long-tail things that will inevitably happen and that happen so rarely that it’s hard to collect data for or plan models for them. But we also have ways to detect whether there's an object present at all, even if we don't know exactly what it is, and that is very important for our safety case. " Sarah tells about her early years at Zoox. “ I’ve been here almost three and a half years. When I came, the robots we were driving were not on public roads. We had very limited 3D perception and no computer vision at all. Even that was so amazing! To see this little golf-cart-looking thing , this very different looking vehicle that could navigate through these experiments -- for example, a person would jump in front of it and it would stop. We still do that sometimes. It’s amazing that it can recognize and stop for all sorts of different situations, people, things, animals… it’s very cool! Now we’ve moved to San Francisco . We go out very often. All the software engineers, especially those writing code that runs on the vehicle, are encouraged to and do go out very often to be able to see and feel what the vehicle is doing . We are very intimately connected with what the problems are .” David goes on to talk about the challenges in his work. He explains the difficulty in building a vehicle that can Sarah Tariq