Computer Vision News - January 2018

Computer Vision News Challenge 33 towards the end of the challenge. The participants in this challenge were very diverse, with expertise in reinforcement learning, biomechanical systems, etc.; and only a small percent of the participants were actually well versed in both the major subdomains this problem belonged to. A setting like this, usually ends up with participants assuming one component or other to be a black box. In the case of this challenge, it was great to see participants with expertise in reinforcement learning developing a decent familiarity with the internal workings of OpenSim, and also around the basic workings of the musculoskeletal models, to come up with relevant adaptations to their approaches. For instance, a common local minima was one where the model learnt to walk with just one leg while only dragging the other leg; to account for the same, some participants exploited the inherent symmetry in a human musculoskeletal model to do action replays of every episode by swapping the actions for the muscles associated with the left and the right legs. There were many other such examples where participants used enrichment of the observation vector , reward shaping, etc. exploiting the domain specific information they already had about the environment. This helped collectively reiterate the fact that blind application of blackbox approaches from domain A to a problem of domain B does not go a long way ; and a decent understanding of problems and the constraints of the approaches are necessary to make good progress. In conclusion, running this challenge was a great experience, from trying to explore the feasibility of the problem, to helping shape the final problem with considerable input and support from the community of participants. By the end of the challenge, we collectively demonstrated that many reinforcement learning algorithms have reached a stage where we can start exploring their use on a range of high dimensional real world problems in more expensive real world simulation settings; and in the process, also learn to look at the classical problems in different domains in a new light. “Blind application of blackbox approaches from domain A to a problem of domain B does not go a long way…” Challenge S.P. Mohanty “Towards the end of the challenge, DDPG ended up dominating the leaderboard…”