Computer Vision News - July 2020

3 Summary A map of object sp ce ... 7 To test the fourth network hypothesis (in the ‘no man’s land’ region) the primates were shown objects belonging to this category while using fMRI to monitor their responses. Electrophysiology targeted to two of these patches which revealed cells that were strongly selective for stubby objects, as originally suspected. Each quadrant represented by a network of multiple regions and earlier experiments showed that different face patches throughout IT cortex encode an increasingly abstract representation of faces. The newly discovered networks showed this same property: cells in more anterior regions of the brain responded to objects across different angles, while cells in more posterior regions responded to objects only at specific angles. The temporal lobe contains multiple copies of the map of object space, each more abstract than the preceding. To further assess the anatomical layout of the face, body, NML, and stubby networks the brain activity from each of those was measured as the primates were looking at those images of objects. The output of the four networks onto coronal slices was overlaid and flattened to a cortical map that showed an ordered progression. This pattern was consistent across animals and quantitative analysis of the linear fit between the patch-ordered label and cortical location of patch peak confirmed it as well. To quantify the object information object identity was decoded using the responses of cells from these networks. Leave-one-object-out cross-validation was used to learn the linear transform that maps responses to features in the deep learning network architecture. The general objects were reconstructed using neural activity. Those decoded feature vectors were passed through a generative adversarial network (GAN) trained to invert the 6th fully connected (FC) layer of the AlexNet and generate fake images. The very accurate reconstructions are shown in Fig. 2. Deep dream images were also generated using MATLAB’s Deep Learning Toolbox which projected onto the four quadrants of the object space. "This is the first time that a deep network has predicted a feature of the brain - unknown before - and turned out to be true."