Reflections on CVPR 2015
One of our colleagues, Dr. Micha Feigin, presents his thoughts on this year’s Computer Vision & Pattern Recognition Conference:
Coming back from CVPR 2015, the main conclusion is that everyone (including us) is doing neural networks, preferably deep learning. Sparsity seems to have lost its glory in the imaging and vision community. It feels like at least 75% of the papers either utilized or were on the subject of neural networks. Quite often even if it seemed otherwise, a neural network ended up hiding somewhere in the bushes.
The interesting question that immediately comes to mind, how long until the next big thing and what is it going to be? We have seen the hype around SVMs, Bayesian methods, random forests, just to name a few, come and go. For the moment, it seems that neural networks are the new thrill and cannot be trumped, although history has shown otherwise time and time again. As usual, only time will tell.
The big change that finally gave neural networks the lead is the amount of data used to learn the network. The jump from tens of thousands to tens of millions of samples is what brought along the performance land slide. One notable example of this is of course face.com.
This would probably not be possible without the processing power provided by modern GPUs. No computer science researcher is complete today without a small super computer with a big GPU warming up the space under the desk.
This unfortunately also marks the downside of neural networks. Other than the fact that there are no clear techniques as to how to design a new network topology, aspiring scientists have to compete with Google who dedicated about 16,000 processors to the problem of learning new networks. Learning the weights for a new topology with large data sets can take days even on such a system. The end results is that most people tweak and relearn on top of existing networks rather than designing new ones. This leads us to the main problem. Neural networks can be fragile to changes and worse than that, it’s difficult to analyze what was learned and how decisions are made, they can essentially be a black box.
Finally, we’ll end with a bit of interesting history. Neural networks are actually the predecessor to probably all other machine learning techniques, dating back to the 1940s to the work of McCulloch and Pitts. This is work that actually led to von Neumann’s proposal for a programmable computer. A piece worth reading on the subject here.
How did you find CVPR 2015? What do you think of Micha’s view?
Coming back from CVPR 2015, the main conclusion is that everyone (including us) is doing neural networks, preferably deep learning. Sparsity seems to have lost its glory in the imaging and vision community. It feels like at least 75% of the papers either utilized or were on the subject of neural networks. Quite often even if it seemed otherwise, a neural network ended up hiding somewhere in the bushes.
The interesting question that immediately comes to mind, how long until the next big thing and what is it going to be? We have seen the hype around SVMs, Bayesian methods, random forests, just to name a few, come and go. For the moment, it seems that neural networks are the new thrill and cannot be trumped, although history has shown otherwise time and time again. As usual, only time will tell.
The big change that finally gave neural networks the lead is the amount of data used to learn the network. The jump from tens of thousands to tens of millions of samples is what brought along the performance land slide. One notable example of this is of course face.com.
This would probably not be possible without the processing power provided by modern GPUs. No computer science researcher is complete today without a small super computer with a big GPU warming up the space under the desk.
This unfortunately also marks the downside of neural networks. Other than the fact that there are no clear techniques as to how to design a new network topology, aspiring scientists have to compete with Google who dedicated about 16,000 processors to the problem of learning new networks. Learning the weights for a new topology with large data sets can take days even on such a system. The end results is that most people tweak and relearn on top of existing networks rather than designing new ones. This leads us to the main problem. Neural networks can be fragile to changes and worse than that, it’s difficult to analyze what was learned and how decisions are made, they can essentially be a black box.
Finally, we’ll end with a bit of interesting history. Neural networks are actually the predecessor to probably all other machine learning techniques, dating back to the 1940s to the work of McCulloch and Pitts. This is work that actually led to von Neumann’s proposal for a programmable computer. A piece worth reading on the subject here.
How did you find CVPR 2015? What do you think of Micha’s view?
