Computer Vision News - April 2016

Research Generating Notifications for Missing Actions Every month, Computer Vision News reviews a research which has been published in our field. We do our best to choose one which you might have not read yet and you will find worth reading about. This month we have chosen to review Generating Notifications for Missing Actions , a model which segments habitual actions to alert us and remind us of anything that we might forget to do in the process. The research’s paper was presented at ICCV 2015 and you can read it here . We thank the authors ( Bilge Soran , Linda Shapiro and Ali Farhadi ) for authorizing the use of their images. Ever forgot to do a single action while performing repetitive tasks? Like setting the alarm clock before going to sleep at night? Failures to do so might be embarrassing and sometimes devastating. Domestic accidents sometimes happen as a result of one single inadvertence. The proposed model proof of concept demonstrates that computer vision algorithms can solve the problem of issuing notifications on actions that may be missed . In order to do so, the model studies interdependencies between actions recorded on video, with the purpose of predicting the order of the actions while segmenting the input video stream. The example taken to illustrate the model is people preparing latte (a coffee drink made with espresso and steamed milk). COMPUTER VISION NEWS 16 Sample frames from the collected egocentric dataset of latte making activity An algorithm for action reminders requires solving segmentation, recognition, and prediction at the same time . The researchers propose a solution to this problem that couples these tasks at a higher level. The proposed method has three main components: (i) a graphical model that segment and predict the underlying sequences of action in the video; (ii) the Flexible Ordered Graph - a graph that model the temporal dependencies between actions; (iii) the notification decision mechanism - that associated costs for missing actions and when it is necessary issue a notification about this missing action. We will briefly review those three components: For a proof of concept, the researchers have collected a latte making dataset using an egocentric camera: subjects wore a head- mounted camera to record their actions during the latte making activity in their own style.