Talk about zeitgeist. Another low-effort stretch between oklo posts somehow accumulated, and in the interregnum, it seems all at once as if every single conversation dovetails in to focus on AI. ChatGPT-4. Chinchilla’s wild implications. TL;DR we have made contact with alien intelligence, and please note that it didn’t occur by dredging up solar-sail spacecraft debris from the ocean floor, or decoding laser communications from nearby stars, or chewing over Arecibo data to heat up PCs.

Speaking of heat, for irreversible computing, Landauer’s limit imposes a thermodynamically enforced minimum energy cost to “flip” a bit. Moore’s-law like dynamics have generated exponentially improving computational efficiency over the past 70 years. And yet, as discussed in the Black Clouds paper, many orders of magnitude of potential improvement still remain. And meanwhile, of course, as processors become more efficient, there is a simultaneous exponential increase the number of bit operations that are carried out. Directed computation is beginning to incur a macroscopic impact on the planetary energy budget. How do things extrapolate forward given the new computational imperative generated by the large language models?

Among its various merits, GPT-4 sure knows how to scrape websites. This notebook queries the Top500.org website and assesses the development of efficiency with time. Supercomputers have increased their efficiency by roughly a factor of 1,000 over the past twenty years, and we are scheduled to hit the Landauer limit right around fifty years from now.

At the same time, the joint capability of the ten currently-fastest supercomputers has improved by a bit less than four orders of magnitude over the past twenty years. By this metric, computation is getting faster a little faster than it is getting more efficient.

This has some interesting consequences. To accomplish of order 10^22 directed bit operations per second, Earth is already using the equivalent of a fair fraction of the total energy generated by by the daily tides. The other half of that energy, of course, is being employed to push the Moon outward in its orbit by a few centimeters per year.

Which seems to have a certain relevance to my favorite Metaculus question.