The fourth kind of calm

Noise in the headphones

In 1927 two engineers at the Philips laboratory in Eindhoven, in the Netherlands, were studying a very simple electrical circuit. Balthasar van der Pol and Jan van der Mark had a neon lamp, a resistor, a capacitor, and a battery, which together made the lamp flash on its own, like a dripping tap. They then fed the circuit a small alternating signal from outside and listened to the result through telephone headphones. As they changed the capacitor, the flashing locked to the driving signal in steps: once per cycle, once every two, every three, and so on, with the frequency jumping from one step to the next. They reported more than forty steps in a paper in Nature titled “Frequency demultiplication,” published on 10 September 1927.

Between steps they heard something that did not fit. In their words, an irregular noise was heard in the telephone receivers before the frequency jumped to the next value. They called it a subsidiary phenomenon and went on with the steps. Historians of dynamics, looking back, say the paper may hold the first experimental observation of deterministic chaos, and some physicists now think that the noise was chaos; but van der Pol did not identify it as anything of the kind, and nobody then had a concept to identify it with. (Their next paper, in 1928, modeled the heartbeat as a set of coupled oscillators of the same kind, an idea we return to below.)

That small mystery was waiting for a question that no one had yet asked: after the first moments pass, where does a system end up?

Three kinds of calm, and a fourth

The question is about the long-run behavior of a system, what it does once the initial lurching has died away. Think of it as the kind of calm it settles into. By the early twentieth century mathematicians had found three.

• Rest. The system settles to a point and stays there: a ball at the bottom of a bowl, a pendulum hanging still, a market at equilibrium. Mathematicians call it a fixed point.
• A cycle. The system settles into a loop that repeats forever: the beat of a clock, the ticking of a metronome, the van der Pol oscillator itself. It is called a limit cycle. Whatever the start, the system ends up on the same loop.
• A blend of cycles. Two or more independent loops running together, such as the Earth’s daily and yearly motions, which never quite line up. The motion never exactly repeats, but it is perfectly regular and predictable. It is called quasi-periodic motion.

The source course lists exactly these three and then adds the point that matters. A fourth kind of long-run behavior is chaos: motion that follows fixed rules, never repeats, stays inside a bounded region, and responds to tiny differences in its start by diverging exponentially. If chaos appeared only for special, unstable starting points, physicists could set it aside as a curiosity of the mathematicians. It matters because it can be stable, a genuine attractor that nearby motion is drawn toward, and so it is something that nature can actually do for a long time. (Course 14 gives the standard definition, Lorenz’s weather model, and the logistic map; this lesson takes a different road through the same ground.)

How was the fourth kind found? One road began with turbulence. In 1944 the Soviet physicist Lev Landau, and in 1948 the mathematician Eberhard Hopf, pictured a fluid on its way to turbulence as acquiring one new independent frequency after another, a quasi-periodic motion with more and more cycles, until the number was so large that the motion looked random. In 1971 David Ruelle and Floris Takens argued, in a paper called “On the Nature of Turbulence,” that this picture was too special. A motion built from several frequencies is fragile, and after only a few of them the system can fall onto a strange attractor, something with no counterpart in the list of three. That is where the term strange attractor was coined.

A test followed. In 1975 Jerry Gollub and Harry Swinney studied a fluid between two rotating cylinders, Couette–Taylor flow. As the rotation grew, they saw the motion go from steady, to periodic, to a motion with a second frequency, and then to broadband noise riding on top of the sharp lines. Landau’s long ladder of frequencies was not there; the transition to turbulence was shorter. (Later reviews say it fits the 1978 version by Newhouse, Ruelle, and Takens better than the 1971 statement.) Then in 1982 Albert Libchaber, Cécile Laroche, and Stéphan Fauve measured a cascade of period doubling in liquid mercury heated from below, as in the logistic map of Course 14. By the early 1980s the fourth kind of calm had been seen in a laboratory, on a bench, in fluids and circuits.

Pull and fold

What does the mathematics of the fourth kind look like? The first clues came from an unexpected place, radio. In 1938 the British Department of Scientific and Industrial Research sent the London Mathematical Society a memorandum about the equations of radio circuits built with vacuum tubes, which were behaving oddly, and the problem was passed to Mary Cartwright, a mathematician at Girton College, Cambridge, and her colleague John Littlewood. Their equation was the van der Pol oscillator pushed by an outside force. In a paper of 1945 they showed that its solutions had an astonishingly complicated fine structure. Littlewood later wrote that this structure “stared us in the face.” Cartwright, who in 1947 became the first woman mathematician elected to the Royal Society, did not call it chaos; no one yet did. In 1949 Norman Levinson replaced the equation by a simpler one that he could solve exactly, and found the same kind of tangle.

It was a letter from Levinson that changed the course of a young mathematician. In early 1960 Stephen Smale was spending several months at the Institute for Pure and Applied Mathematics in Rio de Janeiro, convinced that typical dynamical systems must be simple, with orderly rest points and cycles. Levinson’s letter, dated 20 February 1960, said that his own work and that of Cartwright and Littlewood contradicted this. Smale studied the examples, and, as the account in the Notices of the American Mathematical Society tells it, translated the analysis into geometry. The result was a picture of the simplest mechanism for chaos, now called the Smale horseshoe. Take a square, stretch it into a long thin strip, fold the strip into the shape of a horseshoe, and lay it back on top of the square. Do it again, and again. (Smale’s famous 1966 Fields Medal was for different work, on the Poincaré conjecture in higher dimensions. The jibe about “the beaches of Rio” belongs to that story.)

The pair of operations has a familiar look. The source course points out that it is the geometry of a Lanzhou hand-pulled noodle, 拉面 (lāmiàn): the cook stretches the dough, folds it back on itself, and repeats. Each fold doubles the number of strands, so n folds make 2ⁿ strands: eight folds give 256 and fourteen would give 16,384 (the exact fold counts of the finest grades are craft lore that I could not confirm). Drop two spots of colored ink on the dough and pull, and the spots are drawn out into long thin lines and quickly smeared across the whole piece, because each pull doubles the distance between neighboring points. Mixing happens in a handful of pulls. In 1984 the fluid dynamicist Hassan Aref showed that the same stretching and folding, produced by stirring a fluid in a regular but unsteady way, mixes it efficiently, and called the effect chaotic advection. In 2002 a team at Harvard (Stroock and colleagues) used the idea to design a tiny mixer, with grooves in the floor of a microscopic channel, that works at the very low flow speeds where fluids normally refuse to mix.

Chaos, or just noise?

There is a harder question than naming chaos: telling whether a particular wiggly record from the real world is chaotic at all. A circuit on a bench can be kept clean, and its equations are known. A stock chart or an epidemic record is short, noisy, and has no known equations. Ordinary randomness can look just like chaos to the eye, and the reverse is true, so researchers developed tests.

One is to look for the hidden rule. If a record is chaotic and low-dimensional, each value is a fixed function of the earlier ones, so plotting each value against the next should draw a thin curve, where noise would fill a cloud. Another is to estimate a number, the dimension of the attractor, as Peter Grassberger and Itamar Procaccia proposed in 1983. A third is to test the data against “surrogates,” fake records with the same statistical properties but no deterministic rule. James Theiler and colleagues proposed this in 1992 so as to avoid claiming chaos when a simpler model, such as noise with some memory, would explain the data. A fourth is to try to forecast: in 1990 George Sugihara and Robert May showed that a method of short-term nonlinear forecasting could separate apparent noise from measurement error in records of measles, chickenpox, and plankton.

These tests have a record of embarrassments. Short records and certain kinds of random signals can fake a low dimension, which is why Theiler’s surrogates and Ruelle’s later warnings matter. Ruelle, who had coined “strange attractor,” wrote in 1994 an article called “Where Can One Hope to Profitably Apply the Ideas of Chaos?” in which he cautioned that finding a famous chaotic pattern in a model of biology or economics usually means little for the real thing. The evidence that holds up best is not a dimension number but an out-of-sample forecast that beats noise.

The heartbeat is the example most often given, and it deserves care. A healthy heart does not tick like a metronome: its interval from one beat to the next varies. In 1987 Robert Kleiger and colleagues recorded the hearts of 808 people who had survived a heart attack, with a 24-hour recorder at about eleven days after the attack, and followed them for an average of 31 months. The strongest single predictor of death was low variability: those with a standard deviation of beat intervals below 50 milliseconds had a risk of death 5.3 times that of those above 100. A more regular heart was the worse sign. Ary Goldberger, Lewis Lipsitz, and others built on this with the idea of a loss of complexity with age and disease, and in 1999 Plamen Ivanov and colleagues reported that the multifractal structure of heartbeat intervals disappears in congestive heart failure.

That is solid. Whether the healthy interval pattern is chaos is not. In 1997 Chi-Sang Poon and Christopher Merrill reported that beat intervals in healthy subjects were consistently chaotic. Others (Kanters and colleagues in 1994, Costa and colleagues in 1999) tested similar records and found no sign of low-dimensional chaos, and Leon Glass, a leading expert on heart rhythms, concluded that normal rhythms come from stochastic, nonlinear mechanisms interacting with a fluctuating environment. The safe statement is that reduced variability and a loss of complexity predict harm, and that “complexity” is a better word than “chaos” for the healthy pattern.

Why it took so long

The fourth kind of calm had been glimpsed again and again: by Poincaré in the 1880s and 1890s, by van der Pol in 1927, by Cartwright and Littlewood in the 1940s, and by Lorenz in 1963. Why did it become a science only in the 1970s? A database count gives a rough measure. Lorenz’s 1963 paper, which Course 14 describes, was cited by about thirty papers in its first thirteen years, 1963 to 1975, and by 69 in 1979 and 90 in 1980 (the older counts are surely incomplete, so treat them as lower bounds). Three things changed in the 1970s. Computers made it cheap to run a nonlinear model and look. Mathematicians such as James Yorke began to carry Lorenz’s paper to colleagues in other fields. And laboratory experiments such as Gollub and Swinney’s showed that the fourth kind was not a mathematical oddity. Ruelle adds a caution against telling the story as a single discovery: error growth, he wrote, had been understood by scientists for more than a century, and the real change was in the mathematical, experimental, and computational means to study it.

The source course adds a more general reason. In this reading, for the founders of modern science order was the rule and disorder only appearance. The three known kinds of calm all fit the picture of a world in which everything, given time, either rests or goes round in a regular cycle, and “chaos” was a word for what the laws had not yet explained. That belief shaped what scientists saw in their own data, and it is much older than the nonlinear era.