Rules without a forecast

A rounded number

Around 1960 (the story is usually dated to the winter of 1961, but Lorenz was already describing the effect in a talk in Tokyo in November 1960) Edward Lorenz, a meteorologist at the Massachusetts Institute of Technology, was playing with a toy weather. His computer, a Royal McBee LGP-30, was about the size of a refrigerator and printed its results on paper. His model had twelve variables, standing in for things like temperature and wind, and it repeated a simple set of equations over and over. Lorenz wanted to see whether the weather it produced could be forecast by statistical rules, which was the fashion in meteorology then.

One day he wanted to look again at a run, from the middle. Instead of starting over, which would have meant waiting, he typed in a row of numbers from the printout and went down the hall for a cup of coffee. The printout showed each number to three decimal places, but the computer kept six. The difference was about one part in a thousand, far smaller than any measurement of the real atmosphere could achieve. (The popular telling, in James Gleick’s book Chaos, gives an example, 0.506127 typed as 0.506. Those figures are Gleick’s illustration. Lorenz’s own account says only that he used the rounded-off values.)

When he came back, an hour later, the machine had simulated about two months of weather. At first the new run followed the old one. Then it drifted, and somewhere in the second month the two had nothing to do with each other. In Lorenz’s words, the differences “more or less steadily doubled in size every four days or so.” He suspected a faulty vacuum tube at first. It was not. The computer was behaving exactly as its equations said it must, and that was the discovery: a lawful system, started from almost the same place, can end up somewhere completely different.

Lorenz was not the first to glimpse this. In 1889 the French mathematician Henri Poincaré won a prize offered by King Oscar II of Sweden and Norway for his work on the three-body problem, the question of how three bodies pulling on each other by gravity will move. While the memoir was being prepared for printing, the editor’s questions exposed a serious error. Poincaré paid for the reprinting, and his corrected version of 1890 described, for the first time, the tangled paths now called homoclinic points: motion so intricate that he declined even to draw it. It was the first glimpse of chaos, and for most of the next century it was left alone.

Lawful, but not forecastable

In Course 4 you met Laplace’s demon, an imaginary intelligence that knows the position and speed of every particle and can therefore compute the future. Lorenz’s runs separate two ideas that people usually run together. Determinism means that the present fixes the future by law, with no dice. Predictability means that we can actually work out the future from what we know. The weather model was perfectly deterministic and, beyond a few weeks of model time, perfectly unpredictable, because any real starting point has some error.

The name for this is chaos, and the physicist Steven Strogatz’s definition is a good short one: aperiodic long-term behavior in a deterministic system that shows sensitive dependence on initial conditions. Each piece matters. Deterministic: no randomness in the rules. Aperiodic: it never settles into a repeating pattern. Sensitive dependence: nearby starting points separate, on average, at an exponential rate. In an ordinary, well-behaved system a small error stays small, or grows slowly: a pendulum that starts a hair off swings a hair off. In a chaotic one the error roughly multiplies by the same factor in each interval of time, which is what “doubling every four days” means.

Popular accounts describe the difference as linear versus nonlinear. That is close, with a correction. In a linear system, effects add up in proportion to causes, so a small push has a small effect. Chaos needs nonlinearity, since feedback loops can amplify a push. But nonlinear is not the same as chaotic: most nonlinear systems, such as a swinging pendulum with a bit of friction, are quite predictable. Chaos is the particular case in which stretching (nearby paths pulled apart) and folding (the paths kept inside a limited region) work together.

The graph on the right is the practical lesson. Measuring the starting point ten times more precisely does not make the forecast ten times longer. It adds a fixed amount of time, the same amount each time. A thousand times better, three tenfold steps, buys about eight more time units here. Because the error grows exponentially, every extra hour of forecast costs a huge improvement in measurement, and sooner or later the cost becomes impossible.

The butterfly, correctly

The butterfly came later, and it came from a joke. In 1963 Lorenz wrote, in a paper in the Transactions of the New York Academy of Sciences, that a meteorologist had remarked that if his theory were correct, one flap of a sea gull’s wings could change the weather forever, and that the evidence, he added, seemed to favor the gulls. In December 1972 he gave a talk to the American Association for the Advancement of Science. He had not chosen a title; his colleague Philip Merilees supplied it: “Predictability: Does the Flap of a Butterfly’s Wings in Brazil Set Off a Tornado in Texas?”

The popular idea is that a butterfly in Brazil really does cause a tornado in Texas. That is not what Lorenz said. He argued that if one butterfly could help cause a tornado, then so could every other butterfly, gull, and sneeze, and any of them could equally prevent one. Tiny disturbances might change when weather events happen, not how often. He ended by saying that the question had to be left unanswered for a few more years. His point was about predictability. Errors on the scale of a thunderstorm can double in hours, those errors feed up into larger weather, and so forecasts of two weeks or more are hopeless however the atmosphere is observed. In 1969 he had shown that a system with many scales of motion has a limit on its predictability that does not go away even when you shrink the starting error toward zero. Modern atmospheric scientists call this the “real butterfly effect”: a barrier of finite time, and not simply a sensitivity to small causes.

Where does that leave weather forecasts? Today’s best have some skill out to about ten days, and a 2019 study of the model of the European Centre for Medium-Range Weather Forecasts estimated that making the starting data ten times more accurate would add up to about five more days: a ceiling in the neighborhood of two weeks. Forecasters respond to chaos not by giving up but by running the model many times from slightly different starting points, an ensemble. The first operational ensembles began at the European Centre in 1992, and today it runs 51 forecasts at a time. If they stay together, confidence is high; if they spread, the forecast says so.

A population in a box

You do not need a weather model to see chaos. A one-line rule is enough. Imagine a population, such as insects in a field, that breeds once a year. Let x be this year’s population as a fraction of the most the field could ever hold, so x is a number between 0 and 1. Next year’s population grows in proportion to the current one, but crowding holds it back as x gets close to 1. The simplest rule with both features is x_next = r × x × (1 − x) where r is a growth rate that you can turn up like a dial. The biologist Robert May made this rule famous in a 1976 paper in Nature, “Simple mathematical models with very complicated dynamics.” (Its continuous ancestor is the logistic equation of the Belgian mathematician Pierre Verhulst; the one-step rule used here is the version May and others popularized.)

Try it on a calculator, or with the dial below. At r = 2.8 the population settles at about 0.64 and stays there. At r = 3.2 it flips between two values, 0.51 and 0.80, forever. At r = 3.5 it cycles through four values. Past about 3.57 there is no cycle at all: the numbers wander, never repeating and never leaving the box. And then, at r = 3.83, order returns for a stretch, a window in which the population cycles through exactly three values, before collapsing again into chaos. At r = 4 the chaos is complete.

Two further facts make this little rule more than a curiosity. First, in 1975 the mathematicians Tien-Yien Li (李天岩, born in Fujian in 1945 and later a professor at Michigan State University) and James Yorke proved a theorem titled “Period three implies chaos,” which says that any such rule that has a cycle of three values has cycles of every length and values that never repeat. Yorke is credited with putting the word “chaos” into mathematics. (A Ukrainian mathematician, Oleksandr Sharkovskii, had proved a stronger result in 1964, but it was not known in the West.) Second, the doubling is orderly. The physicist Mitchell Feigenbaum found in 1975, at Los Alamos, that the gaps between successive doublings shrink by the same factor each time, 4.6692016…, and that the same number appears for a whole family of rules that look quite different. He published it in 1978. This “universality” has been checked in experiments on fluids and electronic circuits, which found the number to within about ten percent (4.4 ± 0.1 in a mercury experiment of 1982, for example). A single number links a field of insects, a heated liquid, and a circuit board.

Is the real world full of such populations? Honestly, not as often as the diagram suggests. In 1997 Robert Costantino and his colleagues bred flour beetles in the laboratory, with adult mortality held high and the rate at which new adults were added varied from run to run, and watched the populations move from a steady state, through cycles, into chaos, as the model predicted. That is a real demonstration. Convincing cases in wild populations are scarce, partly because real populations are noisy and short records make it hard to tell chaos from noise.

The shape that chaos keeps

Plot Lorenz’s three equations, the simplified convection model he derived in 1963 from a larger set by his colleague Barry Saltzman, and the path winds around two loops, like the wings of a butterfly, switching sides unpredictably, never crossing itself, and never leaving a bounded region. That shape is called a strange attractor, a term the mathematicians David Ruelle and Floris Takens coined in 1971, in a paper about turbulence. The attractor has a fine, layered structure; its dimension is about 2.06, just above a flat surface. Warwick Tucker proved in 2002, with a computer-assisted proof, that the Lorenz attractor really is what it appears to be. A quantity called the Lyapunov exponent measures how fast neighboring paths separate: for the Lorenz attractor it is about 0.9 per time unit, which means that an error grows by a factor of ten about every 2.5 time units, the slope you measured in the activity.

Here is the hopeful half of the story. Chaos destroys the detailed forecast, but not all knowledge. The path may be unpredictable, yet the shape it traces is not. We cannot say which wing of the butterfly the system will be on at time 40, but we can say with confidence that it will be on one of the two and how often it spends on each. That is the difference between weather and climate: the weather on a given day next month is out of reach, while the typical weather of the month is not. In chaotic systems the short term can be predicted from the parts; the long term, in the statistical sense, from the whole. (In this sense, the short term of a chaotic system is a matter of following the parts, and the long term a matter of the statistics of the whole.)

The solar system is the largest test. In 1989 the French astronomer Jacques Laskar integrated the planets’ orbits for hundreds of millions of years and found that the inner planets are chaotic: small errors in their positions grow by a factor of ten roughly every ten million years, a Lyapunov time of about five million years. That does not mean the planets will wander off. It means that where in its orbit the earth will be in more than about a hundred million years cannot be computed, however exact the starting data. In 2009 Laskar and Mickaël Gastineau ran 2,501 simulations that differed only by tiny amounts, about a meter, in their starting positions. In about one percent of them Mercury’s orbit stretched enough to become unstable; in one run the inner planets were thrown into disorder 3.3 billion years from now, with possible collisions. That is not a forecast. It is a measure of how much a chaotic system can hide in what looks like a stable one. Pluto’s orbit is chaotic too, with an error-growth time of about 20 million years (Sussman and Wisdom, 1988), and the Saturnian moon Hyperion, shaped like a potato, tumbles chaotically, with a Lyapunov time of about a month, as Jack Wisdom, Stanton Peale, and François Mignard showed in 1984.

Can chaos be used? Yes, a little. In 1990 Edward Ott, Celso Grebogi, and James Yorke showed that a chaotic system contains many unstable cycles, and that a small, well-timed nudge can hold it on one. Experiments followed in lasers and chemical reactions, and in 1992 a group (Garfinkel and colleagues) used timed electrical pulses to turn a drug-induced arrhythmia in a piece of rabbit heart into a regular rhythm. That was a proof of principle in tissue. Whether human heart fibrillation is truly chaotic is debated, and chaos-based pacing is not standard treatment.