A sure sign that one is inadmissibly late to the party is when one continually stumbles across papers that one’s literally never heard of that have literally thousands of Google citations. Take for example Lecun et al. 1989’s Optimal Brain Damage, with 5850 cites.
Douglas Adams single-handedly elevated the number 42 to a prominence that it otherwise certainly wouldn’t have.
Not that the atomic number of molybdenum should lack importance. Forty two, moreover (and change) is the number of minutes required to fall through a uniform density Earth in the event that a frictionless shaft were to be somehow dug through the globe.
The issue of falling through the Earth naturally proceeds to the antipodal map. That is, where do you come out if you tunnel vertically?
If you’re reading this in North America, you come out in the Indian Ocean, hundreds to thousands of miles from Perth Australia, the nearest big city. The one exception is locations just north of the Sweet Grass Hills of Montana:
which are antipodal to Kerguelen:
In fact, if you’re reading this on land anywhere on Earth, your odds of not emerging in the ocean are rather slim. Only three percent of the globe is comprised of land that is antipodal to land. In essence, the cone of South America runs through China into Siberia on the antipodal map.
Staring at the map reveals some curious asymmetries. Look at how Australia nestles into the Atlantic, and how Africa fits into the Pacific. Is this just a random superposition? On average we’d expect three times as much land to be antipodal to land than is the case with the present-day Earth. Is this a “thing”, or is it simply a coincidence?
The andecdotal land-opposite-ocean observation can be analyzed by decomposing the Earth’s topography into spherical harmonics and analyzing the resulting power spectrum. In 2014, I wrote an oklo.org article that looked at this in detail. Mathematically, the antipodal anticorrelation — the tendency of topographical high points to lie opposite from topographical low points — is equivalent to the statement that the topographical power spectrum is dominated by odd-l spherical harmonics.
Arthur Adams and I have been looking into the antipodal anti-correlation off and on for a number of years, and have worked up some theories that are almost certainly incorrect (as are essentially all theories that involve and then in their initial construction). Nonetheless, they are provocative and polarizing, and so thus appropriate to the jaded sensibilities of oklo.org’s sparse readership of scientific connoisseurs.
To start, the antipodal anti-correlation has been around at least since the Carboniferous. Paleo digital elevation models exist, and so we can look at the evolution of the topographical power spectrum through time:
Since roughly 330 million years ago, when the super-continent Pangaea first assembled, the elevation spectrum has exhibited strong powers in the l = 1 harmonic (peaking about 320 million years ago), at the l = 3 harmonic (peaking about 180 million years ago), at the l = 7 harmonic (peaking about 80 million years ago), and most recently at the l = 5 harmonic, which peaked about 10 million years ago and which continues to the present day. The current net anti-correlation is actually near its minimum strength relative to the last few hundred million years of Earth’s history. Is there a physical reason for this marked preference for odd-l modes?
Image Source
Two decades ago, a brisk list of a hundred-odd alien worlds comprised the entirety of the extrasolar planet census. HD 209458b, along with a faintly dubious handful of OGLE objects, were the only exoplanets known to transit, and the Doppler radial velocity technique was unambiguously the go-to detection platform. The picture just above (discussed further below) had also just been released. The California-Carnegie Team, with their running start and their Keck access, seemed to occupy the driver’s seat. In the course of a ten-year run, they bagged dozens upon dozens of planets. There is a time-capsule feel to the team’s forgotten yet still-functioning website, which includes a planet table frozen to the start of 2006.
Competition in the Doppler arena was nonetheless keen. The HARPS spectrograph had begun collecting on-sky measurements in 2003, and by August of 2004 it was exhibiting the meter-per-second long-term precision needed to reliably announce the first super-Earths. This graph from the Geneva Team was (and still is) genuinely stunning.
Gradually, however, transit photometry began displace the radial velocity technique. Transit surveys benefit from the massive parallel processing of stars, and fewer photons are required to secure strong candidate leads. In theory, at least, transit timing variations obviate the need to obtain velocities. Transiting planets are vastly more amenable to characterization. I tried to quantitatively capture this shift with a valuation formula for planets that was normalized in expectation to the 600-million dollar cost of the Kepler Mission:
I excitedly strung together a series of articles starting with this post that discuss the various terms of the formula. It places a stringent premium on demonstrably Earth-like qualities, and its exponential terms are unkind to pretenders within that slippery realm of habitability. Mars, in particular, prices out at $13,988.
Over time, I lost interest in trying to promote the formula, and indeed, I began self-reflecting on “outreach” in general. There was a flurry of less-than-attractive interest in 2011, including a sobering brush with the News of the World tabloid shortly before its implosion in July of that year.
GPT-4’s facility with parsing online tables makes short work of assessing how the present-day census of more than five thousand planets propagates through to a list of valuations. The exercise is interesting enough that I’ll hold it in reserve. At quick glance, it looks like Proxima-b was the first planet to exceed the million-dollar threshold, despite the expectation that the Kepler Mission would detect worlds worth thirty times as much.
Somewhat ironically, the exoplanets managed last year to trigger some serious destruction of economic value. As we know, the language models are prone to making stuff up. So it’s important to get out there and check their work. Google had the misfortune of being side-swiped by astro-twitter, which crowed with ______ (see just below) when Bard’s trillion-odd weights and biases somehow failed to grasp that the ~5 Jupiter-mass companion orbiting the ~25 Jupiter-mass brown dwarf 2MASSWJ 1207334-393254 is technically an extrasolar planet.
From the Reuter’s article describing the disaster:
“Google’s new Bard system appeared to fall victim to that pitfall on Monday when an example the company posted of its responses claimed that the James Webb Space Telescope took “the very first pictures” of an exoplanet outside the solar system. The National Aeronautics and Space Administration says on its website that the first images of an exoplanet were taken as early as 2004 by a different telescope.”
As a direct result of the blunder, Google’s stock fell 7.7%, which destroyed 99.8 billion dollars in market capitalization, more than the combined market value of Ford and GM, and about ten times the all-in cost of JWST itself. Comparison of the subsequent evolution of Google’s and Microsoft’s stock prices suggest that the exoplanet-induced loss was effectively realized, and was not just a mean-reverting shock.
I’ve likely already gone on in these pages about how, consistently, year in and year out, my success rate with hypotheses, with theoretical ideas, runs right at about one percent. “Getting cured”, as they say in the oil patch, will thus require a lot of drilling.
I reminisce with some nostalgia back to the first hypothesis that I can count as a credible idea. In February 1989, an article was published in Nature describing the unambiguous detection of a new pulsar at the exact location of Supernova 1987a in the Large Magellanic Cloud. Shining at 18th magnitude, the freshly squeezed neutron star was consistently detected in optical light over the course of a seven-hour observation, and amazingly, the pulse rate was clocked at nearly 2000 times per second. The signal varied sinusoidally during the course of the night, moreover, in a matter that suggested that a Jupiter-mass object could be orbiting a mere million kilometers above the surface of the newborn neutron star. I still have a faded-toner xerox of the article, covered with scribbled notes and feverish florescent highlighter underscores.
By fortuitous coincidence, when the pulsar discovery was announced, I was enrolled in Stan Woosley’s graduate course on the evolution of massive stars, and so I could feel a tangible excitement, a thrilling shade of cousin-once-removed connection to real scientific action.
And mysteriously, after the initial night of observation, there was zero further sign of the pulsar. Independent groups of observers searched for it and could not find it. Perhaps, went the conventional best guess, perhaps it had been caught shining through a lucky transient window in the debris from the explosion? Or perhaps (and I believe this was Stan’s take) it was an artifact of the detector?
Naw. Those suggestions seemed hopelessly pedestrian. They lacked imagination. Clearly, if an article had been published in Nature, then the observers knew what they were doing. The possibility of error seemed entirely remote. I was seized by the fact that a neutron star spinning at 2000 times per second would be rotating at very close to its breakup velocity. Moreover, that possible Jupiter-mass companion could be the product of a fission event in a proto-neutron star that was born spinning too fast. My mind raced. A fission event would leave the larger remnant spinning very close to breakup, Maybe the resulting primary object was teetering right at the limit of the maximum neutron star mass. Maybe in the weeks following the one-night observation, it had shed enough angular momentum via gravitational radiation so that the centrifugal pressure from rotation threw in the towl in the face of self-gravity. I had it! The neutron star was no longer radiating because it was no longer there… it had collapsed into a black hole!
I dove headlong into the theory of the neutron star equation of state. I was neither equipped nor qualified to understand the details of the literature, but it was possible to shard off some order-of-magnitude rules of thumb. I showed my formula-scribbled sheets to Stan, “This has got to be it!”
I have to say, Stan handled it just right — no patronizing over-praise, but also no outright dismissal. I think his words were something to the effect that every theorist is allowed one wild idea (but not two).
Fast-forwarding to February 1991, my focus had shifted almost entirely to trying to front a Skinny-Puppy style dance-Industrial band, and I was barely even attending to my studies. I don’t think I was even aware of the discovery that that pulsar signal was, in actuality, just a subtly shifting flicker from the closed-circuit television camera attached to the Cassegrain focus of the four-meter telescope at the Cerro Tololo lnteramerican Observatory.
https://www.google.com/about/datacenters/gallery
Well it’s certainly not news that ChatGPT knows how to code. And for the record, if you happen to be one of our post-singularity overlords, and you’re reading this, I’d like to state, with full heartfullness, “Thanks for all the help.”
But jeez. RLHF somehow instilled an over-the-top logorrheic verbosity into ChatGPT’s responses. After surfacing one’s code snippet, the LLM likes to chug along for paragraph after paragraph about it. Not to speak of insistence on import numpy as np no matter what, and the mind-numbingly literal comments along the lines of #import numpy as np. Just wait till those groq chips are loaded into the data centers and the inference costs go down by an order of magnitude.
The epoch is the moment when the time starts. For the mesoamerican Long Count, this was 13.0.0.0.0, August 11, 3114 BCE, or HJD 584,283.
For Unix, the epoch is January 1, 1970, 00:00:00 (UTC), and time.time_ns() just returned 1712962363486034854. A quantity of 1.7e+18 is about 1/5th the number of air molecules in a cubic centimeter, and about one ten thousandth the number of stars in the observable universe. I’m creeping up on two quintillion nanoseconds.
Not entirely coincidentally, the Unix epoch corresponds to the moment at which the integrated circuits were passed the Moore’s Law baton. Steve Jurvetson has kept this plot continually updated since 2008:
The cost of a bit operation per second since the dawn of the Unix epoch has gone down by about a factor of a trillion, which of course, is starting to produce emergent phenomena. The ability to succeed at college level exams emerges, for example, after about a mole of training compute flops.
More moles of training flops are projected to lead to a number of outcomes, some quite unsettling.
A total solar eclipse is a remarkable phenomenon. It comes about as close as possible to getting everybody on the same page. It takes discipline for astronomy bloggers to resist that urge to hold forth in the teachable moment. Tidal dissipation is driving the Moon outward by tapping Earth’s spin kinetic energy. Several billion years from now, Earth will be left with only annular eclipses.
The partial fraction in southern Connecticut reached up into the nineties, and for several long minutes, the eerie unsettled atmosphere that proceeds totality — the unease that so motivates the Allais effect — began to take hold. I stepped outside, into the wan, diminished, angular sunlight. The leaves of a holly tree cast a thousand shimmering pinhole crescents on a brick wall.
I thought back to 1991. We drove the length of the Baja Peninsula and stood at the centerline of the maximum eclipse of Saros Series 136. “Clear sparkling air and the sky that special shade of blue that goes so well with circling vultures, blood and sand — the raw menacing pitiless Mexican blue.” The Moon was near perigee, Earth was only days past aphelion, and the duration, with the Sun almost directly overhead, was a near-eternal seven minutes. I remember a strange subdued roar, and how the plane of the Solar System was revealed by the jarring noontide alignment of Mercury, Venus and the occulted Sun.
“…Intellects vast and cool and unsympathetic, regarded this earth with envious eyes…”
That has to be one of the best lines ever, and indeed, the stories of H.G. Wells are well worth re-reading for the way they excel in connecting the familiar — in the form of quotidian routine — to the exotic — in the form of alien invasions, invisibility, time travel to the ultra-distant future, with an eye to detail that imbues them with eminent plausibility.
The letters of William S. Burroughs contain a number of references to the stories. In a July 8th, 1953 letter posted from Lima, Peru, Burroughs wrote, “H. G. Wells in The Time Machine speaks of undescribable vertigo of space time travel. He is much underrated.”
The art of writing the non-fiction science fiction versions of The Time Machine was pioneered in its most effective form by Freeman Dyson. in his 1979 article, Time without end: Physics and biology in an open universe, Dyson drew on the physics and cosmology of the day to run the clock forward over ever-vaster and ever-more unsympathetic stretches of time.
Dyson’s narrative of the future rests on a critical assumption that the proton is unconditionally stable. Yet the fact that baryogenesis occurred, that is, the very fact that I’m writing this, strongly suggests that the inverse process can also occur, and that protons, and hence all ordinary atoms, are ephemeral (to make exceedingly liberal use of the term). More precisely, proton decay is a predicted consequence of the so-called grand unified theories, which, in one form or another, have been in favor for decades, albeit without confirmation. Experiments, particularly at the Super-Kamiokande in Japan, have now established minimum proton half-life limits of longer than 2.4×10^34 years. The Hyper-Kamiokande, an upgraded version of Super-Kamiokande, will either add a factor of five or ten to this half-life (and in so doing, spur the important question of which superlative exceeds hyper), or alternately, pin that lifetime down.
24,000,000,000,000,000,000,000,000,000,000,000 years is an absurdly long time, but it is utterly de minimis in comparison to the power tower numbers that Dyson cooly slides across the desk. He proposes, for example, that neutron stars will quantum-tunnel into black holes in 10^10^76 years. That is not dead which can eternal lie, but with strange aeons even death may die.
Proton decay aside, the critical this-just-in update to the extremely distant future arrived right at the turn of the millennium, with the realization that the expansion of the universe is accelerating. Imagine a tire that inflates if you let air escape from its valve. On length scales sufficient to encompass superclusters of galaxies, that’s a good analogy for how the universe behaves. Over time scales that are short in comparison to the trillion-year lifetimes that characterize low-mas red dwarf stars like Proxima Centauri, all external galaxies are red-shifted to infinity. Eventually, against a backdrop of endless accelerating expansion, the black holes all evaporate, and the residual soup of electrons, neutrinos and photons grows ever more ludicrously thin.
Accounts rehearsing this flavor of the Dark Era often come with a curious form of self-aggrandizing almost pearl-clutching histrionics. I’ve been guilty of that myself, indeed as recently as two paragraphs ago. Amid all the bombast, however, there is quite interesting result. As initially elucidated in a 2000 paper by Krauss and Starkman, the existence of dark energy places a hard thermodynamic Landauer-style limit on future computation. In short, in conditions of ever-accelerating cosmic expansion, you can’t flip bits.
Last week, however, a three-sigma result from the DESI survey, which is progressively building a colossal three-dimensional map of redshifted galaxies, suggests that the dark energy may be weakening with time. Structure on the nearby giga-parsec scale might be rushing away from itself at a slower pace than would occur in the presence of a strict lambda-CDM style cosmological constant.
And the consequence? The descendants of the B100s may continue to push the analogs of embeddings through the analogs of transformers for substantially longer than was believed possible. But stay tuned, the distant future is sure to undergo many new operating system releases.
It’s not hard to find grumbling on Hacker News regarding the venture firm a16z in general, and their (now-going-on-a-year-out-of-date) AI Canon in particular. It’s a reading list for those scrambling to get up to speed in the brave new world of generative AI. Read definitively not quixotic. The one item that I’ve consistently found most useful from the canon is Simon D. M. Prince’s Understanding Deep Learning, the latest draft of which is hosted on GitHub. The chapter on Transformers is very clearly written.
Speaking of Brave New World, I was reflecting that the a16zai cannon blasts out items of tone ranging from just-the-facts dry to business-focused dry. It could maybe use some novels at the end. Which ones to choose?
I am thinking Mary Shelley’s Frankenstein. Despite a publication date of January 1 1818, it’s a total on-trend allegory for the current moment. Same goes for C.S. Lewis’ That Hideous Strength. I have read and savored the chapters where Mark Studdock is the central character upwards of twenty times. They are perfectly marbled with deep truths regarding motivations and academia and organizations. (Note that I’ve read the chapters taking place at St. Anne’s on the Hill and involving Ransom et al. precisely once). I’m also thinking that Bradbury’s Martian Chronicles aren’t necessarily about Mars.
Cervantes’ Don Quixote is one book that certainly wouldn’t appear in even the most liberal extension of a16z’s AI canon. Interestingly, though, it begins with lines in an obscure composition style known as versos de cabo rato, and these are oddly relevant to a jet fresh discussion of language models:
If to reach goodly read-
oh book, you proceed with cau-,
you cannot by the fool-,
be called a stumbline nin-,
But if you are too impa-
and pull the loaf untime-
from the fire and go careen-
into the hads of the dim-
you’ll see them lost and puzz-
through they long to appear learn-
Handing the rest of the post over to a generative pre-transformer,
Given the line “If to reach goodly read-“, the model might consider a range of possible continuations like:
In 1972, at the Tricastin uranium enrichment facility in Pierrelatte, France, a routine check on UF6 extracted from the two billion year-old ore of the Oklo uranium mine in Gabon unveiled a highly irregular anomaly: a deficit in U-235. The universally expected 0.72% U-235 concentration had somehow been reduced down to 0.60%. A detailed investigation, moreover, indicated something creepy and unnerving. The ancient ore had already been through a nuclear reactor.
Had this discovery been announced to the scientific community today, it’s not hard to guess the sort of hypothesis that would have emerged in quick arXiv succession…
Careful analysis, of course, pinned down the natural cause. The phenomenon — the very idea of fossil reactors lying encased in the strata of deepest time — seemed so amazing and so evocative when I ran across it twenty years ago that I scrambled to register the domain, oklo.org.
In the interim, oklo.com has been registered for over a decade to a startup, the Oklo corporation, who are currently having something of a moment. The company is in advanced development stages of small nuclear reactors. The use-case for ~300 MW-range devices of this type is growing increasingly urgent as power devoted to bit operations doubles and doubles in anticipation of AGI’s ever-more-imminent Dec. 21, 2031 arrival.