Computer Vision News - June 2019

is used to estimate the density. Given n data points in high dimension we write: where the summation in the denominator is over all data point except the i'th. We arbitrarily set p i|I to zero. The width of the gaussian is manipulated by what is called perplexity, which influences the variance of the distribution 2) Similarity measure between low dimensional points: as described in the previous section, we now measure similarity between the low dimensional data point; however, instead of using gaussian kernel, we now use student-t distribution with one degree of freedom (also called Cauchy distribution). This distribution is similar to normal distribution but its heavier tails allows dissimilar object to be located far apart. The resulting conditional distribution is: where again we define q i|i =0 . 3) Distance optimization: let us remind ourselves that the y i are the unknown representation in the low dimensional space. So, our objective now is to find the low dimensional representation that enables the pairwise similarity measures in both spaces to be as close as possible. Since we are dealing with probability distributions, the immediate solution is to use Kullback-Leibler (KL) divergence . KL divergence is a measure for similarity between two probability distributions P,Q. It is asymmetric in nature, which explains why we call it a measure and not a metric. Given our distribution P,Q defined in the two previous sections, we optimize KL(P||Q) using gradient descent where our optimization variables are 1 , . . , Note that we do not have a guarantee that the above embedding converges; however, as we will see next, in well-behaved data sets t-SNE demonstrates very nice results. Now that we understand what t-SNE does, we are ready to use it and explore different hyperparameters and different visualization styles of the model. Implementation We implemented for you a code that embeds the MNIST dataset into a 23 Focus on Computer Vision News t-SNE | = exp − − 2 2 2 exp − − 2 2 2 | = exp − − 2 exp − − 2