
BitsBack Coding
April 1, 2023

Gaussian Processes
March 30, 2023

Variational Autoencoders
March 5, 2023
I’ve been wanting to write a primer on the variational autoencoders for some time. There have been so many papers and blog posts written on this that I am at this point very late to the party. Nevertheless, I will put this out for the exercise of it. Perhaps I will have a slightly different way of thinking about it that someone finds illuminating. …

Shannon vs Universal Compression
February 16, 2023

Variational Inference
July 6, 2022

Ideal Gas Entropy Derivation
June 21, 2022
Derivation of the change in entropy formula for an ideal gas (used in the The Carnot Cycle post) from state space volumes. Discussion about connections between the observer’s information about the gas and how that relates to the reversibility of transformations applied to the gas. …

Liouville Supplemental: Bertrand Paradox
April 5, 2022
I reframe the Bertrand paradox as the statement that uniformity of measure is relative to choice of coordinate system. The objectiveBayesian approach to the problem of priors is to assign a maximally uninformative prior to the given possibility space. What is considered maximally uninformative can be derived with the maximum entropy principle  a generalization of the principle of indifference. In many cases this ends up being a uniform prior. However, we run into a problem since uniformity is relative to choice of coordinates. This is relevant to physics since there is no preferred coordinate system to work in. …

Liouville Supplemental: Coordinate Transformations
April 5, 2022
This is supplemental material for Liouville's Theorem. Specifically I go through a few examples of phase space transformations, canonical and noncanonical. I also show that we can turn arbitrary configuration space transformations into canonical phase space transformations, a result that will be useful for my discussion about the Bertrand paradox (Liouville's Theorem). …

Liouville's Theorem
April 5, 2022
Liouville’s Theorem states that the size of a state region of any closed system remains constant as the system evolves through time. This has consequences for connections between information and physics. …

The Carnot Cycle
March 17, 2022
This a formal description of the Carnot cycle which I hope is a useful reference for anyone who wants to quickly ramp up on thermodynamics. The Carnot cycle is often used as a canonical introduction to classical thermodynamics (specifically the thermodynamics of ideal gasses) since it nicely illustrates the relationship between the entropy, temperature and volume of a gas. …