K-Means

Shrinking Models by Sharing Weights — K-Means-based Quantization

Group a neural network’s weights into a few representative values with K-Means, and you can shrink the model several-fold with almost no accuracy loss. We explore it with an interactive widget where you watch the clusters converge and the storage shrink in real time. (Deep Compression, Han et al. 2016)

K-means, GMM, EM: Three Nested Russian Dolls of Clustering

K-means is actually an extreme case of GMM, and GMM is the canonical application of the EM algorithm. How these three connect within a single framework, and how information geometry explains the relationship.