https://3rdlayer.uk/categories/ai/Recent content in AI on 3rd layerHugo -- 0.154.5en-USSat, 06 Jun 2026 00:00:00 +0000https://3rdlayer.uk/posts/kmeans-weight-quantization/Sat, 06 Jun 2026 00:00:00 +0000https://3rdlayer.uk/posts/kmeans-weight-quantization/Group a neural network’s weights into a few representative values with K-Means, and you can shrink the model several-fold with almost no accuracy loss. We explore it with an interactive widget where you watch the clusters converge and the storage shrink in real time. (Deep Compression, Han et al. 2016)https://3rdlayer.uk/posts/linear-quantization/Sat, 09 May 2026 00:00:00 +0000https://3rdlayer.uk/posts/linear-quantization/Beyond storing weights as integers — running the multiplications and additions entirely in integer arithmetic at inference. We connect reals and integers with the affine map r = S(q − Z), and explore it with a widget where you change the scale and zero point and watch the quantization error. (Jacob et al. 2018, the basis of TFLite integer quantization)https://3rdlayer.uk/posts/numeric-data-types/Sun, 12 Apr 2026 00:00:00 +0000https://3rdlayer.uk/posts/numeric-data-types/INT8, FP16, BF16, FP8, FP4 — what do the data types you keep seeing in deep learning actually mean, and how do bits turn into numbers? We take them apart one by one, with widgets where clicking a bit updates the formula and value in real time.https://3rdlayer.uk/posts/llm-rei-experiment/Sat, 24 Jan 2026 00:00:00 +0000https://3rdlayer.uk/posts/llm-rei-experiment/We administered the REI-40 dual-processing personality inventory to 5 frontier LLMs. The results reveal distinct ’thinking style’ profiles — from neutral responders to rational enthusiasts.https://3rdlayer.uk/posts/sae-steering-gpt2/Sun, 16 Feb 2025 00:00:00 +0000https://3rdlayer.uk/posts/sae-steering-gpt2/Finding emotion-related features in GPT-2 using OpenAI’s pretrained SAE, then training one from scratch. Feature patching turns ‘good person’ into ‘shit’.https://3rdlayer.uk/posts/em-kmeans-gmm/Sun, 10 Dec 2023 00:00:00 +0000https://3rdlayer.uk/posts/em-kmeans-gmm/K-means is actually an extreme case of GMM, and GMM is the canonical application of the EM algorithm. How these three connect within a single framework, and how information geometry explains the relationship.https://3rdlayer.uk/posts/information-geometry/Sat, 09 Jan 2021 00:00:00 +0000https://3rdlayer.uk/posts/information-geometry/Just as Newton’s F=ma describes the physical world, information geometry describes how AI learns. An intuitive guide for beginners.