Originally published in The Islands of Benoît Mandelbrot: Fractals, Chaos, and the Materiality of Thinking, edited by Nina Samuel. Bard Graduate Center: Decorative Arts, Design History, Material Culture, 2012. 18-61.

From the exhibition: The Islands of Benoît Mandelbrot: Fractals, Chaos, and the Materiality of Thinking.

---

Prologue: Chasing Traces; or, The Blind Spot of a Discussion

One day in the late 1980s the mathematician Alain Connes and the neurobiologist Jean-Pierre Changeux sat down together in Paris to hold “conversations on mind, matter, and mathematics.” Despite being eminent men of knowledge, they were utterly unable to come to an agreement.1 The two adversaries were discussing a fundamental dilemma, the ramifications of which extend far beyond the discipline of mathematics: Can the world of thinking be separated from the world of things? And if not, how can we conceive of the interactions between the two realms? Does thinking—together with mathematical objects—have a material foundation? Does it leave traces? And are thoughts dependent on the instruments that make them possible and give them their shape? Connes argued that there was a mathematical world “independent of us, which escapes sensory apprehension.” Changeux’s position was the opposite: “Granting your position for the sake of argument, I’ve tried to see where this world could be, to see what evidence of it there is in nature. If you form the hypothesis that this mathematical world exists outside of us, and if you call yourself a materialist, then it seems to me you’re obliged to give it a material basis.”2 The men could not agree.

What is striking about this conversation, however, is not that the two men would be in the grip of diametrically opposed ideas. After all, for centuries people have been disputing such things. No, what was striking was what they left out: Never once was the topic of mathematical images raised in any detail—and it was precisely this topic that might have shown them the way to mutually acceptable answers. In this essay, I will attempt to draw attention to the ways in which images and objects have either triggered mathematical thinking or served as its subject matter and helped to shape it.

In what follows, I will touch on three moments in the “personal-pictures” history of fractals and chaos studies as modes of visual thinking, which is being presented here for the first time. The analysis of the private working materials of Benoît Mandelbrot stands at the beginning of this essay, taking up the greatest part, to be followed by two briefer discussions of scientists from the related fields of dynamical systems and chaos theory, Adrien Douady and Otto E. Rössler. Along the way, the surprisingly complicated relationship between mathematics and its images and representations will be explored, along with two different meanings of “mathematical islands,” one figurative and one abstract, each pointing to an antipodal power of visual representation in the thinking process. Pictures can release a productive, imaginative power that advances thinking, but attached to them is a seductive and potentially ungovernable energy as well, one that causes frictions between vision and thought and even, potentially, egregious errors. These two sides of pictures are inseparable and equally necessary.

If all three scholars used images in fundamentally different ways, and if the juxtaposition of their different methods of deriving mathematical ideas from the world of the visible and the material will help distinguish three basic different modes of visual thinking, it is equally true that the thinking of all of them was deeply grounded in visual representations and that their work would not have been conceivable without images. For all of them, experiments with early computer visualizations proved decisive, though hand drawing and other modes of visual and material manipulation were equally indispensable as modes of thinking and understanding in the realm of mathematical ideas.

The Discovery of the Island: Playful Sequences of Similarities

When seeking new insights, I look, look, look, and play with many pictures. (One picture is never enough!)3
—Benoît Mandelbrot

Viewers fortunate enough to view Benoît Mandelbrot’s first fractal computer animations, made in the early 1970s, may well experience a jolt of surprise at the sudden confluence of the worlds of science and science fiction when the words “The Island of Dr. Mandelbrot” appear on the screen for a few seconds.4 Produced by Mandelbrot and programmer Hirsh Lewitan at IBM on a Stromberg Datagraphix 4020, the animations feature the jagged objects that Mandelbrot began referring to in 1975 as “fractals”: collections of dots, lines, surfaces, or solids, “loosely characterized as being violently convoluted and broken up,”5 that cannot be comprehended by means of Euclidean geometry. The films depict such structures as animations generated by chance algorithms. One sees three-dimensional compositions—dots, lines, the relief structures produced somewhat later (his so-called “imaginary continents”)—slowly rotating in front of a black background.

In the line below the intertitle “The Island of Dr. Mandelbrot” is proof that Mandelbrot himself associated his fractals with the 1896 fantasy novel The Island of Dr. Moreau. There, he added in tiny, barely legible letters, “(with apologies to H. G. Wells)”.6 With this twist on the title and the mock “apology,” Mandelbrot deliberately established a connection between “science and science fiction” and, with gentle, endearing irony, related his own work to that of a long line of scientists seen as god-like, if weird, creator figures.7 There is a hint in these first animated simulations of a classic dream of the early pioneers of computer graphics, namely that they might, with the aid of the computer, create a new world, a second reality. By way of the film’s montage technique, that world was given a quite specific structure, one expressed in repeated, rapid zooms. The viewer is drawn in closer and closer—without ever landing— as greater and greater enlargements appear. Zooms first entered film history as a dramatic effect in Hitchcock’s Vertigo (1958), and ten years later they were employed by Charles and Ray Eames in the first version of their famous film Powers of 10 (Rough Sketch, 1968) as a way of staging a journey between atoms and galaxies.8 Mandelbrot knew the Eameses, who also worked in association with IBM, but he characterized their work as “violently anti-fractal propaganda.”9 Unlike the designers’ films, Mandelbrot’s did not emphasize the differences between levels of magnification; instead, as he was at pains to point out, they created a cosmos out of similarities. The sections marked with squares in the film revealed in increasing close-up the same frayed structures as could be seen from farther out. In this way, Mandelbrot dramatically illustrated one of the most important geometric characteristics of fractals—their self-similarity.

Perhaps the two key features of Mandelbrot’s picture making—as well as that of other scientists working in the field— were his attention to similarities and the works’ appearance in a series. Mandelbrot routinely worked with sequences of pictures; virtually every shape appeared again and again, with the very slight differences between them coming only in the form of minimal adjustments to definition, slight shifts in picture details, tiny modifications of parameters, et cetera. There were piles of variant picture shapes that were so similar as to be virtually indistinguishable. That this is characteristic of pictures from this field of study raises the question of how picture sequences and single pictures relate to one another. Can our observation of the material evidence of the working process tell us anything about how serial pictures facilitate mathematical understanding? The pictorial work of Benoît Mandelbrot provides the first answer to this question.

In the Beginning Was the Image; or, In Praise of a Medium Without Touching It

To see is to believe.10
—Benoît Mandelbrot

In 1985, when Columbia University awarded Benoît Mandelbrot the Barnard Medal for Meritorious Service to Science, it did so with the following justification: “In the great tradition of natural philosophers past you looked at the world around you on a broader canvas.”11 Within roughly ten years after his first computer animations, Mandelbrot had managed to create a geometry that brought him renown far beyond the boundaries of mathematics—a geometry associated with a specific worldview. How did he himself view the world? Or perhaps better: How did the public in 1985 view Mandelbrot’s view of the world?

In a press photograph published by IBM around the time, a smiling Mandelbrot can be seen leaning on one of the firm’s computers.12 Behind him, there is a large paper board on which is written the formula for a general definition of the fractal dimension (D). (This dimension is an index for the varying degree of roughness of fractal structures. It quantifies its complexity as a ratio of the change in detail to the change in scale. The dimension D is always a decimal number and distinguishes fractals from Euclidean geometry, in which there are only whole-number dimensions for lines, planes, and solids.13) The word “example” also appears on the board, but the rest of the line is obscured by Mandelbrot’s right shoulder, and thus the role of the example is taken up by a seemingly cosmic scenario on the computer monitor on which Mandelbrot rests. Programmed by his colleague and friend Richard Voss and bearing the title “Planetrise over Labelgraph Hill (Souvenir from a Space Mission that Never Was),”14 the image presents a sphere that looks much like Earth—its colors had been manipulated to create this effect—looming up out of the darkness of the monitor above a jagged and shadowed hilly landscape covered with round craters. It is one of the highlights of the use of fractals for pictorial fantasizing for which Mandelbrot became known to the public. Shifting the formula literally into the background, it not only presents the picture the formula produces, it illustrates the prominence of images in his way of thinking and working—specifically digitally produced images. Indeed, the decision by the IBM photographer to stage this photo as he did makes Mandelbrot’s earlier, quite non-public reference to his work as an imaginary “Island of Dr. Mandelbrot” seem almost visionary. Evidently, the dream of being able to create a second nature with the aid of fractals, and thereby to have deciphered mathematically the principle behind the creation of the world, was now Technicolor reality.

The view of the world that Mandelbrot presented on his monitor had unmistakable precursors in pre-digital picture history. Not without reason does it recall scenes from one of the first American science-fiction classics, Destination Moon (1950). And even though Mandelbrot gave out that the IBM photo showed a “space mission that never was,” such flights were indeed taking place quite frequently then—in movie theaters. Immediately after the publication of Mandelbrot’s first English-language book on fractals, filmmakers began usurping the fractal simulation of nature. It was not long before the first planetary scenes wholly produced by means of his fractals appeared, in Star Trek II: The Wrath of Khan.15

Mandelbrot’s fondness for algorithmically simulated cosmogonies with a science-fiction aesthetic, along with his frequent use of the provocative phrase “to see is to believe,” greatly influenced the way his work was perceived.16 In later statements, he appeared to glorify visual representation. In a subtle attack on the Old Testament proscription against graven images, he rephrased the opening line of the Gospel of St. John: “Wouldn’t ‘In the beginning was the word’ have to be replaced with ‘In the beginning was the picture?’”17 Mandelbrot’s unabashed devotion to pictures was not universally shared; it occasioned heated criticism both within mathematics and outside it.18 In their eternal repetition and similarity, images of fractals and chaos were seen by art historians as breaking with classical mimesis theory, as threatening to collapse “the difference between picture and reality itself.”19 It is significant, however, that these appraisals were primarily based on popularized images that were not produced in the process of mathematical research. They were also reactions to the influence the reception of such pictures was having on postmodern Western discourse: Fractals became a flagship of simulation theory and a symbol of “new” or even “immaterial” picture mediums that were categorically difficult to understand and seemed to break with all pictorial tradition.20

Given his close identification with the computer, it may seem paradoxical that Mandelbrot was wholly incapable of programming one himself. All his computer graphics were team efforts. Not only did he gather about him a host of programmers, he also seems to have been conflicted in his relationship to the device. All his manuscripts and text corrections were either written by hand or typewritten. He avoided touching a computer himself—an evasion that was well known in the firm, and that led the IBM photographer to jokingly comment that he had obviously done so in the photo. In good humor, Mandelbrot corrected him, claiming that he had leaned against the monitor, to be sure, but had not “touched it with his hands.”21 The anecdote may only be embroidered oral history, but it touches upon something fundamental: There is a significant discrepancy between the way scientists employ images and the way those images are seen by the public. As the present exhibition aims to show, in cases where discussions are based on popularly known imagery, as was long the case for fractals and chaos studies, this gap can be immense. Indeed, it provides both material and motivation for the reevaluation of the role of computers as interfaces between the world of thinking and the world of things. On the other hand, the surprising appearance of literary references in Mandelbrot’s newly-discovered early films indicates a second fundamental and opposing dynamic: It was not only in the popularization of such images that they came to be laden with meanings that transcended mathematical or physical issues. Playing with images—with sequences of similarities—has to be understood as a genuine part of research and thinking in general.

Mandelbrot’s focus on visual representation raises questions that can only be dealt with after more detailed analysis of both his work as a whole and his use of images in particular from the perspective of image theory and epistemology. His equation of seeing and believing is doubtless unproven, equally indefensible whether one dismisses it as evidence of a naive faith in images or takes it as either a rhetorical flourish or a deliberate, tongue-in-cheek provocation. It still stands as a polarizing, rhetorically pointed reaction against what he saw as the devastating effects of certain late-nineteenth-century developments in mathematics.

Mandelbrot was firmly convinced, and never grew tired of emphasizing, that

the use of computer graphics is now in the process of altogether changing the role of the eye. The hard theoretical sciences had banished the eye for a long time, and many observers used to believe, and even hope, that it would remain banished forever. But computer graphics is bringing it back as an integral part of the very process of thinking, search, and discovery.22

Where had “the eye” gone, and how had it disappeared?

The Picture as Menace

I see it, but I don’t believe it.23
—Georg Cantor

The progress of mathematics is not always smooth. At the end of the nineteenth century, for example, the French mathematician Charles Hermite reported to his colleague Thomas Jean Stieltjes that calculus was afflicted with a “curse” and that he had turned away in fear and horror “from the lamentable plague of functions without derivatives.”24 Mathematics had been in the midst of a crisis for some time, one brought about by deviations from classical assumptions about elementary concepts.25 At the core of the crisis were functions whose values sprang endlessly back and forth between two constants and for that reason could not be precisely drawn. Such imprecise constructions had previously been banished from mathematical discourse, but even so, they now threatened to lead “to a frontier of the intellect.”26

These oscillating, interdimensional structures could have been a further reason why in the mid-1970s—at the very moment he was coining the word “fractal”—Mandelbrot chose to make the allusion to H. G. Wells’s fantasy novel: The fictional Dr. Moreau had tried to create a new world with man-animal hybrids, and it was for a similar purpose that Dr. Mandelbrot was using those mathematical hybrids from the nineteenth century. Henri Poincaré, among others, had referred to these new functions as “monsters” of mathematical logic and labeled them outrageous attacks on the intelligence of his precursors.27 Roughly a hundred years later, Mandelbrot touted these so-called “monsters” as the first mathematical fractals. He liked to repeat this blasphemy as a way of pointing to his own achievement in having made out of “monsters” a “geometry of nature” that, above all, have “the advantage of being beautiful.”28 By repeatedly emphasizing that he had incorporated monsters into his new, digital nature, he was also aligning himself with classical sixteenth-century natural history, in which nature itself became the realm of “exceptions, the rare, and the marvelous.”29

The crisis in mathematics was also one of seeing. When the mathematician Georg Cantor questioned the principles behind the Euclidean idea of dimension, he rightly associated the shock his discoveries had given him with a disbelief in the reliability of his vision: “I see it, but I don’t believe it.”30 It was as an ironic twist on Cantor’s comment that Mandelbrot chose his own motto: “to see is to believe.” Indeed, Mandelbrot accused the mathematicians of the turn of the twentieth century of “blindness” for failing to recognize the visual qualities and the clear import of their phenomena.31 At the heart of the crisis was a growing mistrust of the cognitive ability of the eye and accordingly of pictorial representations. The more radically even elementary fields of mathematics broke away from the way humans conceive of space visually, the more difficult it became for the eye to follow the discipline, and the more inadequate and misleading even schematic graphics appeared to be. This “plunge of mathematics into the unimaginable” in response to new “monstrously oscillating” functions has been referred to as “iconoclasm” by the media historian Bernhard Siegert.32

And yet there are a few surviving visualizations of the first mathematical fractals: simple line drawings, highly schematic and directly related to structural steps described in texts.33 They record the recognition that the human eye could no longer follow geometry’s experimental thinking since only a few structural steps led it below the visibility threshold. These drawings were symptomatic of a historic change in mathematics, one in which the relationship between geometry and the world of experience, between thinking and seeing, was renegotiated. On the one hand, what made it possible for the “monsters” to become the subject of official mathematical discussion was that spatial perceivability (Anschauung) had ceased to be a normative principle at the time.34 Given that at least a few mathematicians recorded their deliberations in graphic form, it is clear that a certain value was still accorded to the picture and to seeing. But generally it was clear that in order for constructions that broke so radically with fundamental assumptions to be integrated into mathematics, the discipline itself had to be redefined.

The crisis was soon perceived as a threat that shook mathematics to the core; in 1921 Hermann Weyl described it as a “Grundlagenkrise” (foundational crisis) that threatened to destroy the “inner solidity and certitude”35 of analysis. A way out of the crisis was finally proposed by David Hilbert with his redefinition of the “foundations of geometry.”36 As early as 1899, he showed that the discipline could be axiomatized, an initial but momentous step toward the accommodation of the uncertainties caused by an explicit rejection of visualization.37 Hilbert had concluded that geometrical figures were unworthy of the mathematician’s “superior intellect” and as a “practical constraint” were contrary to “freedom.”38 With Hilbert, mathematical symbols became autarkical. They no longer meant or referred to anything outside of mathematics.39 By the end of the 1920s, the prevailing view was that “objects of present-day mathematics … are no longer tangible things; … their structures … are of the most extreme abstraction. … [There is a] tendency toward intellectualization, a disassociation from everything material.”40

In such a climate, the ground was prepared for the ascendency of a group of French mathematicians who would concentrate and intensify the iconoclastic tendencies in mathematics. They radicalized Hilbert’s formalistic approaches and considered pictures to be not only problematic or inadequate but a menace to mathematical thinking. Beginning in 1935, they grouped together under the name “Nicolas Bourbaki.” The Greek-sounding name, which was derived from that of a general in the Franco-Prussian War, served them as a collective pseudonym, the reasons for which they never fully revealed.41 They were united in their championing of a method characterized by its rigorous structuralism and formalism: For them, structure as a “treasury of abstract forms”42 was to be the true subject matter and medium of mathematics. It would be of crystalline design, “transparent down to the depths.”43 With the publication in 1939 of the first volume of their Éléments de Mathématique, they became one of the most influential authorities in mathematical research and teaching.

All of their authorized publications were similar in one crucial respect: There were no illustrations. According to the Bourbakist Pierre Cartier, their rejection of pictures was true to “the Euclidean tradition,” which in their view was originally non-pictorial. In addition, he stated, most of the members were puritanical, among them a great many Protestants and Jews—“Old Testament people”—and as such they were ill-disposed to visual representations of religious and mathematical truths.44 Their iconoclasm was unwritten but extraordinarily effective. For example, the iconoclastic imperative is implicit in the suggestion that mathematical intuition is of importance but in a “manner wholly unrelated to ordinary sense perceptions.”45 Because mathematics was to be thought of as a “poetry of ideas,”46 the “original visual content” of mathematical figures needed to be “consciously excluded.”47 In line with this call for precision, the discipline was to be entirely transformed into text.

Linked to this radical rejection of images was a correspondingly forceful admission of their power. This was expressed most blatantly by a pictorial symbol shaped like a reversed “S” that the Bourbakists invented for their publications. In the “directions for use” (Mode d’emploi de ce traité) that preceded their volumes, they explained its meaning and called it a tournant dangereux (dangerous turn).48 Whenever their reasoning threatened to become precarious, they presented the reader with this hair-pin bend: Just by glancing at the page, the reader would know to prepare for intellectual hardship or logical shoals. The way the bold, squat typography of the sign visually cuts through the arrangement of the typesetting demonstrates that the “iconoclastic” mathematicians conceived of their publications’ layout as a pictorial element in its own right. By understanding and using the image qualities of the typeface, they were able to convey the importance of the symbol in an intuitive and immediate way.

Digging deeper into the pictorial history of the sign the Bourbakists chose, an even more striking level of meaning surfaces. On a formal level, the mathematicians had resorted to the centuries-old figura serpentinata as a primal expression of the creative energy in nature and in art, one that can be identified as a universal “symbol of all drawings” and “the sum of the visual.”49 In 1753, the English artist William S. Hogarth defined it in The Analysis of Beauty as an emblem of “Variety” and accordingly a symbol of all forms of pictorial expression.50 Some two hundred years after that, its rich meanings were still being celebrated and investigated by artists such as Paul Klee, whose graphic work explores the active dynamics of convoluted lines as a foundation for both artistic practice and image-theoretical inquiry.51 The Bourbakists’ stylized serpentine line was used to flag spots that spelled the same kind of danger for the “realm of pure thought”52 that “imprecise” pictures in general did. Thus they banished what they viewed as the dreaded destructive power of the visual by means of the visual. The Bourbakist S-curve became a kind of hex sign that called upon the magical power of the pictorial in a way reminiscent of the ancient stylized appropriation of snakes as a cultic means of defending against them.53 One may assume that they were unaware of the traditional iconic meaning of the sign, but even so, it functioned as a thorn in the flesh of pure, formalized doctrine.

In the following decades, formalistic representation was pitted against geometrical, pictorial representation in official mathematical discourse. The dogma of anti-pictorialism left drastic traces in the lives of certain mathematicians, but it was becoming apparent that pictures could not be completely banned. In one example, the mathematician and Fields Medal–winner Alexander Grothendieck first made a name for himself with works in the Bourbakist mode. But just before his retirement, he experienced a “pictorial turn.” He not only invented his own “picture theory of mathematics,” in the course of this change he literally condemned formalism to hell as being inimical to life.54 In 1971, after one of his last official lectures at the University of Bielefeld, in Germany, he left a drawing in the seminar logbook in which a mathematical notation introduced by the Bourbakists is consumed in raging flames and surrounded by demons armed with pitchforks. The drawing was embedded in the following commentary:

Witch’s Kitchen 1971. Riemann-Roch Theorem: The “dernier cri”: The diagram [displayed] is commutative! To give an approximate sense to the statement about f : X → Y, I had to abuse the listeners’ patience for almost two hours. Black on white (in Springer Lecture Notes) it probably takes about 400, 500 pages. A gripping example of how our thirst for knowledge and discovery indulges itself more and more in a logical delirium far removed from life, while life itself is going to Hell in a thousand ways—and is under the threat of final extermination. High time to change our course!

That change of course seemed to be promised by the invention of the computer as an image-making machine. In an ironic turn, the picture-rejecting method based on Hilbert and elevated to the status of dogma by the Bourbakists had meanwhile become the theoretical point of origin for the computer. The language of computer programming was derived from a strictly textual mathematics, yet with the blossoming of that language in computers it became possible to convert structures no longer accessible to three-dimensional representation into a new form of visibility. The capacity of the computer to continuously repeat a calculation and with each iteration add further detail to the graphic facilitated the generation of this new kind of imagery, which before had been seen as betraying human intuition. Because fractal structures are composed of an infinite number of details, the new technology seemed especially suited to making such highly complex shapes graspable. Technically speaking, fractals and chaotic dynamics lack closed analytic solutions, which means one has to actually count through all the states produced by the dynamics. A human being is at a loss here, particularly because even the calculation of a single step is a challenge. However, the computer is able to mimic the generative process of the dynamics in hand and consequently to visualize patterns inherent to the equation that could not be seen before. In other words, the dynamics themselves are generatively produced, much like a dynamical process in nature.

Mandelbrot held the Bourbakists in part responsible for his having turned his back on Europe and subsequently on academia. With the help of the computer, proof would be provided that the rejection of imagery had been a mistake. It was with a heartfelt sense of triumph that toward the end of his life Mandelbrot was able to declare that “Bourbaki is over.”55

Spotting Islands, Cartographies of Chance: Analogy as a Principle of Thought

I see things everywhere.56
The eye is not specialized.57
—Benoît Mandelbrot

Mandelbrot once characterized the aim of his scientific life’s work as the restoration of a “concrete meaning” of sight in which the seeing eye is accorded the status of a scientific Instrument.58 The reason it was such an effective instrument, he believed, was that it functioned in an interdisciplinary way. To Mandelbrot, this was owing to one of the main epistemic merits of seeing: “The problem with calculations is that they become very much specialized in the different fields. On the contrary, the eye is not specialized. It is a universal tool.”59 That each academic discipline practices its own calculation method and relies on specific knowledge acquired over centuries often renders it barely accessible even to neighboring disciplines. This could be alleviated, he insisted, through the cultivation of seeing, a theory he was fond of citing retrospectively to explain his own interdisciplinary approach and the universal validity of his fractal theory: They were, he claimed, achievements of the “unspecialized eye.” To him, formulas and pictures were inseparable: “[E]very formula evoked in my mind a completely spontaneous shape.”60 But this did not imply that the eye of a “layman” was what was required; rather, he was claiming that seeing had no need of specialization to function as an instrument of understanding.

But this raises the question of how “unspecialized” an eye should be—or can be. How does Mandelbrot’s testimony relate to his use of pictures and theoretical thinking, and what clues are provided by his working materials? At this point, it is necessary to return once again to the profound clues provided by his early reference to the Island of Doctor Mandelbrot, and his interest in islands more generally. Pictures of islands and coastlines were among the most numerous finds in his office in 2011. This was not a coincidence. They were crucial to his ability to bridge the gap between mathematical figures and natural forms that had been created even before it became possible to generate images with a computer. The properties of the edges of islands had already been featured in an article he published in Science in 1967 entitled “How Long Is the Coast of Britain?,” which is now considered the founding text in fractal geometry.61 There, referring to the “monster curves” of the nineteenth century, Mandelbrot argues that coastlines are self-similar and infinite in mathematical terms, constantly revealing jagged new details under enlargement. In the article, he continues to refer to such structures as “fractional,” a term he had adopted from the eclectic mathematician Lewis Fry Richardson, whose findings he discusses in the article.62 It would not be until the early 1970s that Mandelbrot would have access to the technical machinery he needed to generate pictures from these formulas. Looking back, Mandelbrot often described how the sight of these first computer-generated shapes was first perceived with a shock of recognition that operated in a similar manner as a déjà vu:

It is impossible to forget the sleepless nights we spent producing the pictures with programs that we had diverted from their intended purpose so as to be able to use them at all… . Although we hadn’t expected it, relief set in: the curve described the coast of New Zealand and Bounty Island.

[T]he device was very cumbersome, but when the shapes came out, what a revelation! The pictures were so poor… .Even so, there was an overwhelming feeling that what we had drawn was right.63

Reports such as this document the “Columbus feeling” frequently referred to in descriptions of early computer use.64 In the years of the first experiments with fractal coastlines, producing pictures took all night, so that Mandelbrot’s “speckled geographies” were often thrown away the next morning by computer-room operators who thought them to be misprints. Moreover, for a long time, it was only possible to generate lines, the edges of which were composed of the letters O, X, M, and W; uniform surfaces were not possible. Indeed, some of the islands in the catalogue were later amplified by hand . Mandelbrot did ultimately publish the black, irregular specks in figure 16 in his first book, Les objets fractals (1975). He distinguished between them on the basis of their fractal dimension values 1.1, 1.3, and 1.9. Of the shapes identified with D=1.3, he commented that they were the ones that displayed the greatest simulative power. Their similarities to cartography were hard to overlook: The top island was reminiscent of Greenland; with a quarter turn, the one on the left looked like Africa; and after a turn of 180 degrees, one could see New Zealand with Bounty Island in the mass of specks.65 In that same year, Mandelbrot published similar chance shapes in another context, noting that he had now discovered Greece, the mirrored Sea of Okhotsk, the Gulf of Siam, and western Scotland.66 Mandelbrot relied on an associative interpretive logic in dealing with such ambiguity, cheerfully rotating his images in search of familiar shapes.

Mandelbrot’s interpretation of chance artifacts can be related to a pre-digital history of seeing in the realm of art. His method was preceded by a long tradition among artists of exploiting the creative and inspirational potential of chance patterns.67 Prominent examples of this practice can be found as early as the Renaissance: In describing the proper practice of artists, Giorgio Vasari refers to the early stage of composition when artists would make a “first drawing … in the form of a blotch,” which should be considered only as “a rough draft of the whole.”68 He advises that such blotchy sketches (Italian: schizzi) be “hastily thrown off” as a means of testing “the spirit of that which occurs to him.”69

A pictorial technique comparable to Mandelbrot’s associative interpretation of his visualized mathematics can be seen in eighteenth-century landscape drawings. The English artist Alexander Cozens (1717–86) first introduced his method of “blot drawing” in 1759.70 It allowed one to develop landscapes out of arbitrarily placed and presumably nonobjective specks of ink blots. The “blot” itself was “no drawing, but a collection of accidental shapes out of which it is possible to make a drawing.”71

Comparing Cozens and Mandelbrot, one first notes their differences: Mandelbrot’s silhouette-like, large-scale specks of color come to resemble monolithic, menacing boulders when compared to Cozens’s dynamic brushstrokes, which reflect the play of light and shadow and evoke perspective. Whereas Mandelbrot studied his specks from a bird’s-eye perspective, Cozens gazes into a three-dimensional landscape—even when the illusion of space is largely absent in this self-contained scene with “receding hills” and a sparse sky, as the title suggests. And yet there are astonishing similarities between the two. Cozens’s “blots” were thought of as “multifunctional” elements: “One and the same ‘blot’ can lead to different sketches” and consequently serve as the starting point for different imaginarylandscapes.72 Landscapes developed in this way were to be formed “out of the spirit of nature and according to the principles of nature instead of merely replicating a portion of nature already formed.”73 Cozens’s “natural” landscapes could not be identified with any specific locality, and in this respect his associative picture logic, based as it was on the interpretation of chance blots, was comparable to Mandelbrot’s practice. While Cozens aimed for a “reassessment of the relationship between art and nature,”74 Mandelbrot was suggesting new relationships between nature and mathematics.

That Mandelbrot’s images only vaguely resemble the silhouettes of certain countries was part of their message. They were not intended to emulate any specific geography, but rather to evoke the idea or basic principle of nature—to be “true to” it. Indeed, dissimilarities and discrepancies between picture and object were necessary to set him thinking and to construct the “natural fractal.” This corresponds to a basic quality of epistemic images in general as analyzed by the historian of science Hans-Jörg Rheinberger, who has stated that both the “ambiguity of the representational process” and a certain “vagueness” of the object were required of such images.75 By means of his associative method, Mandelbrot was able to access a realm of free thought. Only through the image and its interpretation were completely new meanings generated. It was these that produced the connection between geometry and nature, which would not have been made at all without the image as intermediary.

But it was not computer graphics in the first place that suggested to Mandelbrot that he look at chance distributions and nature together. A good ten years before he had access to such technical facilities at IBM, he had already begun to train his eye by looking at traditional hand-drawn statistical representations. With regard to figure 18, for example, he noted in 1963 that if one ignored the picture’s source it was possible to imagine that one was looking at a “new world.”76 The image in question, an “experimental illustration” that plotted on three scales the results of a sequence of coin flips, came from the two-volume textbook An Introduction to Probability Theory and Its Applications by the mathematician William Feller.77 Mandelbrot borrowed Feller’s illustration without a single change, though he proposed a geomorphological description of it that added an additional level of meaning: “First of all, forgetting the origin of that figure, let us imagine that it is a geographical cross section of a new part of the world, in which all the regions below the bold horizontal lines are under water.”78 By using another mathematician’s picture for his own purposes, Mandelbrot was able to establish a strong and characteristic connection between chance and nature. The chart came to assume an iconic status for him, and he reproduced it in all three of his books on fractal geometry, as well as in a number of articles.79

That Mandelbrot’s way of seeing served to update visual traditions can be discerned in the development of statistical graphics. For example, the now legendary diagrams of the economist William Playfair, considered one of the inventors of the field, were designed to be easily scanned, a quality characteristic of landscape depiction as well.80 His Wheat/Wages Diagram from 1822, for instance, relates the price of grain to the weekly wages of workers between 1565 and 1821. That Playfair wanted the viewer to register the information with ease is indicated by the last three words of the phrase he confidently placed in the middle of the image: “Chart Showing at One View.” The synoptic qualities of the visualization were crucial to him, as can be seen in his choice of a background for the changes in wages: the trope of a silhouetted towered city, familiar from early modern cartography, yet colored in a kind of gray sfumato to reinforce the distance. To obtain the impression of a landscape, he combined graph and column diagram, two modes of representation that are very different. Playfair unambiguously if tacitly answered the importance of the “specialization of the eye”: In learning to read new picture forms, it is necessary that one be presented with well-established, familiar visual impressions. Instant legibility—showing at “one view”—all but requires the use of other genres’ pictorial logic. It is in them that preexisting visual habits are exercised and a picture’s subject can be accepted at first glance as logical because familiar. A similar representational mode was employed in the development of geology as a separate scientific discipline at around this time. Indeed, a glance at the chart may suggest geologists’ characteristic transformations of mountainous regions into panoramic crosssections81— just one among many examples in history of a mode of visualization being handed along from one science to the other.

Playfair’s strategy shows that the eye can accommodate itself to a new picture form if it comes into contact with familiar ways of perceiving reality, as in an image that has been designed in a particular way or, as Mandelbrot demonstrated in his reading of Feller’s diagram, an existing image that has been cleverly re-interpreted. Mandelbrot’s repurposing of Feller’s diagram allowed him to visualize chance in the jaggedness of the earth’s surface; in turn, this allowed him to treat the features of a certain part of nature—the “sculpture of mountains”—as something that could be analyzed using statistics.82

Both Mandelbrot’s first impressions and the later configuration of his images combined meanings and visual traditions from different contexts. These not only influenced the dissemination and popularization of Mandelbrot’s work, they were part of his exploratory thinking process, shaping thereby his conceptions themselves.

As for the use of images in the construction of a fractal “geometry of nature,” Mandelbrot’s “unspecialized eye” sought to figuratively reinterpret representations of abstract knowledge. Schemata that could no longer be deciphered on the experiential level were approximated to visual habits; thus they functioned as an experimental field for new constructions. Indeed, such readings could take on a significance of their own that affected Mandelbrot’s interpretation of their content.83 Mathematical structures that were unrecognizable in visual terms, and accordingly appeared to exclude the eye as an instrument of understanding, were objectified, in analogy to geomorphological features, as visually identifiable spaces.

The way Mandelbrot saw and interpreted his images tended to break down traditional distinctions between the use of pictures in art and in science whereby images in science are supposed to be unambiguous.84 Instead, as Mandelbrot helped to demonstrate, pictures in the sciences have to be seen as fundamentally obstinate, intractable instruments whose vagueness can be productive in research. Scholars such as Mandelbrot have utilized these qualities of the visual in a playful, associative manner, one that has led them to new insights.

Many of the first artists to experiment with the computer in the 1960s sought to use scientific methods, “with their claim to objectivity, transparency, and verifiability,” to reconstruct “an aesthetic on a scientific basis.”85 Mandelbrot took the opposite approach; he made use of artistic techniques that exploited the subjectivity of visual impressions to create natural forms out of images of chance. In this way, a kind of “role reversal” occurred between motives conventionally attributed to the realms of art and science respectively.

At the same moment, this apparent breakdown between traditional distinctions also led to new confusions. One has to wonder about the degree to which this perceived “role reversal” influenced the scientists’ self-image. Here it is worth noting a seemingly insignificant detail in some of the pictures in the exhibition: Mandelbrot’s signature. On my first visit to his office in 2008, he insisted on signing all the pictures, individually, and with a fountain pen, before he allowed me to scan them for my research— seventy in all. This procedure offered clear evidence of a pronounced sense of authorship— even for those pictures that had been misprinted, torn, or stained.

Signing a picture is a practice mainly associated with fine art, a way of claiming responsibility for one’s oeuvre.86 Mandelbrot candidly admitted that he played with this ambivalence: “For Who’s Who I put down MATHEMATICIAN AND SCIENTIST or even MATHEMATICIAN, SCIENTIST, SCHOLAR, AND ARTIST, which upsets editors, who prefer a single word.”87 How did he come to think that? Was this mere hubris or evidence of a misunderstanding?

There would be no point in suddenly declaring Mandelbrot’s signed pictures to be “artworks.” The question itself is unproductive, as it tends to lump together the varied pictorial products of art and science and seduces one into “blurred simplifications.”88 More to the point is whether a separation between mathematics and art is justified at all with regard to pictures and their possibilities— particularly their epistemic possibilities.

Mandelbrot’s islands provide a perfect example of how his creative method left its mark on the world of mathematics. Having spent years testing the effects of various dimensional values in order to determine how he might achieve the greatest simulation of “nature,” Mandelbrot arrived at an important mathematical thought solely on the basis of a visual impression. Looking at some of his pictures, preparing the manuscript for what was to become Fractal Geometry, he once again had the sense that he was looking at landforms and then all of a sudden, “I screamed: Stop it, stop it! … I see an island, I see a continent. It’s amazing—this is Spain, this is Great Britain … and then Scandinavia!”89 Since he had previously used the fractal dimension of 1.3 for such distinct impressions of nature, he now conjectured that these pictures must also have the value 4/3 (rounded to 1.3). That Mandelbrot’s intuition was correct was ultimately proved in 2000 by the mathematicians Gregory F. Lawler, Oded Schramm, and Wendelin Werner.90 This gap between hypothesis and proof was not untypical of Mandelbrot, whose creative, associative way of looking led him to conjectures rather than to rigorous mathematical statements. Lawler himself compared Mandelbrot’s approach to that of the archaeologist who digs and makes exciting discoveries but leaves it to others to provide the explanation or the context—or in Lawler’s words: “If he had not produced the questions, other people would have not found the answer.”91

The creation of visual analogies—the pursuit of analogical seeing—was the engine that drove Mandelbrot’s research. Indeed, he came up with the most appropriate symbol for this method himself. In 1974, having set about creating his own designs for the cover of Les objets fractals, he began to cut out illustrations from the book and paste them as a collage on its front cover. He produced three separate versions of the design, which was not in fact used, based on his model of the distribution of galaxies, his speck-like “islands,” and his “imaginary continents.” True to Mandelbrot’s credo—“before beginning to understand what fractals are, one should know what they look like”—an overview of the phenomena was to precede even the opening of the book.92 On the back cover, a collection of straight lines—a basic element of Euclidean geometry—was to appear. Mandelbrot identifies this collection of lines with Parisian streets93—but if it is an imaginary city map, it looks as if it had been struck by an earthquake. The viewer feels as if he or she is looking through the lines, which are of various widths, into a box of great depth, while the direction of the streets has been whirled into a confusion that extends across the entire sheet, an effect generated by the application of calculated pseudo-chance. Thus, a stochastically transformed “culture” and a programmed “nature” are juxtaposed and interrelated. In the jacket text, Mandelbrot states: “[I]t will be suggested that there are far-reaching analogies between all these objects that are intuitively comprehensible. Despite their novelty, this must be seen as fundamental.”94 Mandelbrot himself provided the key word there: “analogy.” This was a concept fundamental to his thinking. The juxtaposition of images on his book-cover designs recontextualized his picture shapes and gave them new meaning. More generally, the tableau as a configuration of shapes in a common picture space serves to symbolize his entire approach. This gives a first answer to the initial question of how mathematical thinking can be organized and make sense given a surfeit of similar pictorial material in a series. The single picture showing (or rather embodying) the exemplary individual case becomes more general—demonstrating a way of seeing that creates analogies and reveals associative potential as part of a multitude—while still keeping its validity as a unique image. In short, a logic of seeing takes hold that can be described as the “logic of the tableau.”95 But it should also be noted that with his theory that computer graphics alone had led to a “return of the eye” in mathematics, Mandelbrot underplayed the role of sense impressions in the acquisition of knowledge and in so doing helped to skew people’s assessment of his own contribution. On the contrary, it was his imagination, initially developed independently from the computer, that had inspired his research and served as an antidote to the dominance of calculation.

The Emptiness Between Islands: Media and Visibility; or, The Computer and Its Blind Spot

I saw it but I didn’t know that I was seeing it.96
—Benoît Mandelbrot

How visible are islands? Does an island stop where the water begins? And how can one discover what lies between islands? Such questions may seem strange at first, yet they lead to the crux of a riddle that was still unsolved in 1980—and to a second way that islands were important to Mandelbrot’s pictorial thinking. From imagination and the figurative meaning of the islands as geomorphological structures, the focus is thus shifted to their abstract counterparts: “islands” and “continents” as metaphorical terms that Mandelbrot used for decisive pictorial details on his computer-generated prints when he was about to make one of his most well-known discoveries, the Mandelbrot set. These components, and especially the mysterious spaces between them, raised serious issues about the mutual dependency of the world of mathematics and the image technologies that generated them. It was the riddle of those islands and the question of whether they were really peninsulas that was plaguing Mandelbrot and that revealed the twofold, antagonistic power of the visual for thought: the potential to seduce the mind through mere appearance. As the pictorial history of this famous symbol of chaos theory demonstrates, vision and thought had by no means always been perfectly in harmony. Rather, seduction can be determined as a second force that coexists with imagination. Imagination and seduction are two sides of the same coin in visual thinking.

The majority of the pictures from Mandelbrot’s office on display in the exhibition are related to experiments he carried out in 1979–80, shortly before his discovery of the set. Yet if these images are dominated by mere blankness, patterned by a covering of small dots, or structured by fuzzy shadows or indefinite outlines, the Mandelbrot set became generally known in a wholly different form, dazzling and brightly colored, as in the computer graphic that appeared on the cover of Scientific American in August 1985, which looked like a dynamic fireball racing down a black slope. Above all in the margins, the Mandelbrot set is distinguished by impressive ornamentality and self-similarity. This is all the more striking given that the formula it is based on seems simple even to non-mathematicians.97 This picture is typical of the popular composition and distribution of the set, which in the mid-1980s became one of the first scientific “icons” in the digitalization era.

Despite its shaded, vivid, and relief-like appearance, the Mandelbrot set is at heart nothing but a series of numbers that can be graphically represented within the complex number field. Because complex numbers consist of two components (real and imaginary), the picture space of a digital graphic that sits in a coordinate system with its x and y axis allows us to produce a correlation between the invisible equation and the visible surface: Each pixel on the screen corresponds to one precisely defined complex number. That said, the discrete, grid-like structure of digital-visualization technology has also generated its own zones of insecurity for this correlation: While a non-linear dynamic structure generally takes up an infinite and continuous space, if it is to be represented digitally, on a computer, it has to be reduced and approximated. As a consequence, pictorial details that seem unrelated or “far apart” on an image may well be connected or “very close” as far as their provable mathematical properties are concerned. In short, visibility and existence in a mathematical sense—what can be seen and what can be rigorously proven—are not the same.

The gap between appearance and “reality” became a serious challenge for Mandelbrot’s investigation of his “mathematical islands.” On the aforementioned 1985 cover, for example, there appears a black bulbous shape that Mandelbrot would have called a “continent” (i.e., the main, central part of his set); on the upper level, in a black squiggly blaze, one can make out a black shape of similar appearance, only smaller: an “island” (i.e., a self-similar replica). In this image, there are no empty spaces apparent between islands and the continent. Both are connected visually through various color nuances and shades. But this was by no means the case during the discovery process. At first, continents and islands were separated by empty spots. What was the significance of the transition from empty intervals to color connections? To explain the mystery of the islands and their problematic visibility—and hence their image-theoretical relevance—it is necessary to take a closer look at the process by which they were discovered.98

The discovery of the Mandelbrot set was based on work done by the French mathematicians Gaston Maurice Julia and Pierre Fatou at the beginning of the twentieth century on the dynamics of polynomials in the field of complex numbers.99 Their papers were held to be extremely difficult masterpieces and as such were long considered all but impossible to understand, even by mathematicians. Mandelbrot studied under Julia at the École Polytechnique and knew his work, but until the late 1970s, he had never concerned himself closely with it.100 It was only when he realized that a computer might be used to produce pictures from Julia’s analyses that his interest was awakened. At a colloquium in Princeton in 1977, Mandelbrot was made aware of computer graphics by the mathematician John Hubbard, with whom he made contact a year later.101 Hubbard was one of the very first people to produce graphic depictions of cubic polynomials. Though Mandelbrot spoke of Hubbard’s pictures as being “of rather poor quality,”102 they must have made a strong impression on him, as he soon invited Hubbard to IBM to learn about his ability to generate computer visualizations from Julia’s formulas. This was not long before he himself, armed with some of the most powerful computers of the time, began producing pictures of so-called Julia sets, a practice that would ultimately lead to the definition of the Mandelbrot set.

It is possible to distinguish two phases in Mandelbrot’s research. In the first, at IBM in 1979, he devoted himself exclusively to the graphic depiction of complex non-quadratic equations. In the second, as a visiting professor at Harvard during the winter semester of 1979–80, he focused under less than ideal technical conditions on quadratic equations and their illustration. The two phases can be characterized in terms of sense perception, the first being a phase of seeing and the second being a phase of discovery: “I did see those elements [i.e., what were later called the small “islands” as part of the Mandelbrot set] in 1979 but could not organize and describe them. Therefore, they remained undiscovered.”103 That his seeing did not take place at the same time as his discovery contradicts the all-too-simple assumption that the Mandelbrot set simply suggested itself spontaneously once calculating capacities reached a certain level. Such a view is no more accurate than the claim that the set’s discovery would have been possible before the invention of the computer. The history of the Mandelbrot set can neither be disassociated from the requisite technology nor attributed to it alone. The main difficulties that stood in the way of its discovery at the beginning resulted from Mandelbrot’s having to deal with pictures new to this field, pictures that needed to be observed. With the computer, observation became a new epistemic category of mathematics.

During the phase of merely seeing and of playful experimenting with the new shapes, Mandelbrot, together with his programmer at the time, Mark R. Laff, produced an extensive series of black-and-white graphics. During my survey of Mandelbrot’s office, I came across some 150 images from this period which he characterized first as “state-of-the-art,” then as “used furniture,” and finally as “valuable antiques.”104 They document most impressively how comprehensive his experiments were and how difficult it was to make anything visible at all. Mandelbrot did not publish these pictures at the time because it was simply not possible to make sense of them or to “clean up their theory.”105 At the beginning, it was by no means certain that any of his experiments would create a clearly recognizable pattern at all. An equation’s complexity was often reflected in an apparitional blurring of its contours—or even, as happened frequently, in the fact that the picture simply remained white, and nothing at all could be seen. Some of the images are immediately reminiscent of glimpses into distant galaxies. The appearance and disappearance of the patterns seem to be synthesized in a single picture space.

Whereas the choice of coloring is arbitrary and depends on the technology used, the assignment of colors—or the decision to just use black and white—has to follow neatly defined rules that have to be fixed before the image is generated. These rules identify the range of values—the threshold— for which a specific color will be assigned. To generate a graphic, the computer takes each of the numbers in the plane (i.e., pixels), inserts its value into the formula, and iterates it for a previously defined number of times to find out what specific color has to be assigned. In chaotic dynamics, it is a characteristic of the equations to react strongly even to the tiniest changes, a quality known as “sensitive dependence on initial conditions.” Consequently, small changes in the parameters of an equation can lead to completely different behaviors. The technological limits of the resolution entail two major problems. First, two patterns that are theoretically distinguishable may blur in the computer graphics and thus appear to be indistinguishable. Secondly, error in rounding-off induce artificial changes in state, which in turn influence behavior in the long run because of the equation’s dependence on initial conditions. Previously distinct shapes and patterns may, therefore, start to fade and blur. The technology and its limits of resolution determine the degree of fuzziness.

The difficulty of achieving precision in the representation of equations explains why, in many of the images—especially the huge series of 150—only a few outlines are clearly recognizable, with the majority disintegrating into irregular cloudlike spots: “Throughout,” wrote Mandelbrot, “I felt like someone trying to photograph the Cheshire cat in Alice in Wonderland at the very moment it is about to disappear.”106 The pictures that he compared to the transitory appearance of the famous mythical feline from Lewis Carroll’s novel reveal the fragile condition of the number field known among mathematicians since 1918 as “Lattès chaos”.107 The closer the parameter value λ approached the number 1, the larger the gray areas in the picture became.108 Like the photographer who controls his shot by his choice of lens and his use of lighting, Mandelbrot identified more or less photogenic and picture-worthy motifs in the complex number field. Instabilities between appearance and non-appearance are indicative of the fundamental characteristic of non-linear equations. Because of their great sensitivity to changes in parameters, they lend themselves to the illustration of the transition between order and chaos in dynamical systems. What was to be considered order and what chaos depended on the visual arrangement: For example, gray areas with interference, from which a pattern was wrested out of multiple variations of the most diverse parameters, belonged to the classic iconography of chaos. Mandelbrot acknowledged in retrospect that in places in the series it might already have been possible to recognize the familiar bulbous shape of the Mandelbrot set. He subsequently glorified these as “advance-shadows of the Mandelbrot-set-to-be,”109 which he had not yet identified as a shape of its own.

The blurring and the empty spaces in these pictures, however, point to the complex relationship between formula and image. Visualizations can always only be approximate values of non-linear sets. Since the calculations inevitably have to be performed with a finite number of iterations, the resulting images always bear traces of a compromise. The encounter between formally infinite mathematics and the finite abilities of the computer owing to its limited processing power necessarily produced indefinite and hence blurry areas. As a result, Mandelbrot focused on investigating appearances and disappearances and searching for the right picture moment for recognizing a pattern, as well as on the different possible meanings of the empty spaces on the sheet of paper. It was not always possible to determine unambiguously how the visualization and the mathematics related to each other. Owing to the technique and its limitations, there were built-in contradictions and areas of friction between theory and image. It was because of those interstices and empty spaces that the island issue would soon become acute—and tempt Mandelbrot into hastily drawing mathematically unproven conclusions.

The problems of visualization became especially critical when Mandelbrot moved to Harvard. There he had neither an office of his own nor his own machines to work with. He was dependent on the older computers in the basement of the Science Center that were available to everyone, shimmering green monitors with cathode-ray tubes and dated printers. The brief working time available and comparatively poor equipment forced him to reduce the complexity of his research subject. Turning away from cubic polynomials, he decided, together with his assistant Peter Moldave, to experiment with quadratic polynomials with a single variable parameter. What he expected was a simple, unambiguous result. Yet what appeared was unlike anything he had experienced before. The first pictures “looked awful: filled with apparent specks of dust that the Versatec printer produced in abundance… . I was motivated to sniff out the ramifications of those specks of dirt”.110 According to all reports, the monitor at Harvard was subject to constant malfunctions and the decrepit printer made it uncertain what one was really seeing. Were the specks in fact dirt or “dust”? Were they smears and blots of the sort that the machine produced in abundance or visualizations of mathematics? For the moment, the boundary between image and accident was blurred. Suddenly, something surfaced that could not be understood and that raised new kinds of questions: Were the specks part of the picture or not?111

In order to distinguish between picture and phantom images, Mandelbrot decided to examine the specks more closely, employing in effect the computer as a microscope. It was only his perception of symmetries in the “dirt” that elevated the “reality probability” that the specks belonged to mathematics: “Since bona fide dirt is not symmetric, both kinds of specks demanded to be blown up for close inspection. … The symmetric points had more chances of being real.”112 From a series of enlarged pictures, it was determined that some of the specks were in fact dirt from the machinery, but others proved to be self-similar copies of the basic shape. The presumed glitch was thus transformed into a decisive discovery.113

Mandelbrot dubbed the self-similar “messy” speck structure surrounding the main body of the Mandelbrot set as “islands”—“offshore islands”—though sometimes he also referred to them jokingly as “pearls in the pigsty” because they had been discovered in the dirt.114 For years he tried to ascertain their precise structure. The crucial question was whether or not they were linked to the main structure of the set. The computer images gave them the appearance of free-floating elements. Yet how did the picture relate to the theory? Could their mathematical nature be related to the way they appeared? Were the visible parts of the picture also existent in a mathematical sense? Could he believe what he saw?

Because the small specks were deceptively similar to interference or printer smudges, they were repeatedly retouched by conscientious computer-lab and print-shop employees. For that reason, in the years 1980–81, Mandelbrot got in the habit of posting warnings on his output equipment: “Do not clean off the dust specks, they are real.” According to his own reports, the fact that his early fractal pictures were in constant danger of being erased had infuriated him on many occasions. Likewise, his first “chance geographies” had repeatedly landed in his co-workers’ wastebaskets as presumed discards or misprints. With regard to the “reality status” of his visualized mathematics, a striking reversal can be seen in his attributions: He now called his simulations of natural forms “imaginary,”115 while the visualizations of his complex equations were classified as “real.”116 Mandelbrot was fond of insisting that he had created his new geometry “out of the dirt”—clearly a further jab at the formalistic Bourbakists, to whom “dirt” was the avowed enemy of mathematical, “crystalline” logic.117

Mandelbrot’s unconventional and creative reliance on his “unspecialized eye” would never have been possible without pictures. Here, however, it can be seen that his trust in it had its drawbacks: In his first publication of the Mandelbrot set, in 1980, Mandelbrot confused the mere “appearance” of the small islands with their mathematical existence. In his manuscript, he noted: “A striking fact, which I think is new, becomes apparent here: FIGURE 1 is made of several disconnected portions.”118 The specks of islands seemingly detached from the main body of the set were declared, in a fateful confusion of image and proof, as a “fact,” although this assertion was clearly unproven, as it was not supported by any mathematical theorem.

Moreover, observant viewers of the picture published in the “Annals of the New York Academy of Sciences” could not even make out the specks discussed in the text, for once again they had been deleted as supposedly unwanted printing flaws by the editor: “Horrors! It is now free of specks! … Clearly, gremlins in the printing business had … repeated that evil deed.”119 In offprints of the article that he sent to colleagues, Mandelbrot reinserted the deleted picture details by hand.120 Since he did not see any connecting lines and had unlimited faith in his computer, he did not believe they existed, a faith he reinforced in the drawing. Suddenly, Mandelbrot’s “seeing is believing” had changed to “drawing is believing”: to trust in the digitally produced shape, which could be—and had to be—authenticated in drawing to exist. By means of his own successive acts of reinterpretation, the same pictorial details mutated from a hint to a proof and finally ended up as the main result: A material trace became mathematical evidence.

The mathematicians Adrien Douady and John Hubbard provided the counter-evidence in 1982. In an article in Comptes Rendus, they determined that fine lines did connect the specks with the main body but that they could not be made visible at that point in history: “These islands are in fact attached by threads that evade the computer.”121 Douady compared the complex problem of making these structures visible with trying to see tiny mountain rivulets from an airplane flying overhead.122 When Mandelbrot was making his experiments, it was technically impossible to draw the connecting lines, or filaments, with the existing graphic programs. There was a built-in impenetrability between the two “planes” of the digital image, its visible surface, and the indiscernible columns of numbers that generated the picture (i.e., the operational undersurface that processed the formula).123 Accordingly, an unbridgeable divide was created between the “islands” and the “continent.” A link between the two planes was achieved only a short time later through depictions in shades of gray or colored graphics. Color was able to fill the previously blurry or empty zones at least visually and to distinguish a variety of different behaviors in the dynamical system. In consequence, Mandelbrot felt compelled to expand his earlier slogan, “To see is to believe,” into “To see in color may lead to an even higher intensity of belief.”124

If one were to write a history of the sciences as a history of pictures, one that told the story of the construction of the visible out of the invisible, the issue of limitations on what can be visualized—i.e., visibility thresholds—would necessarily be a primary topic. At what point does something come into view, when does it disappear again, and when does one decide that it is “visible enough” to be perceived as an actual, provable phenomenon—to be judged as a mathematical “fact”? This touches on basic questions concerning the epistemic function of technical images and the elementary relationship between picture-producing methods, pictures, and knowledge. New picture-making methods can open up new realms of visibility and understanding to the same degree that they can also once again limit our seeing.

In Mandelbrot’s case, the realm of visibility also defined the horizon of understanding; or, as the science historian Thomas Schlich has written regarding the problematics of microscope pictures, “The limits of understanding coincided with the limits of the visualization technique.”125 Since Mandelbrot placed unlimited trust in his machines and had not seen the connecting lines—could not see them—he had failed to credit their existence and had confirmed as much again in his drawing after the unwanted retouching. His own equation of seeing and believing had misled him into claiming he had discovered a “striking fact” from the mere appearance of the pictures.

Nevertheless, the story of the Mandelbrot set is not one of confusing pictures with reality or of subjective naiveté. On the contrary, it raises the question of how one arrives at understanding through computer graphics in a paradigmatic way. It clearly shows how difficult it can be to maintain a distance between what the eye can see and what can be proved analytically. As the story of a struggle for agreement between observation and theory, it illustrates equally clearly how necessary it is for the eye to be schooled on pictures and how risky thinking inspired by pictures can be. In short, it says something about the fundamental ambiguity of visual perception.

When one considers all of the subsequent events, one comes to recognize that the story of the Mandelbrot set also includes an element of tragedy. Looking back on these events in an article published six years later, Mandelbrot not only regretted the false assumption made in 1980 and the “awkward surrogate”126 of a conclusion that resulted from it, he also searched for ways to exonerate himself after the fact. One notes this specifically from his retrospective narrative of the story and from the caption beneath his subsequent publication of Julia sets in 2004. There he insists that an analysis of the theory had taught him as early as 1980 that the images of Julia sets were deceptive, and then he adds: “These graphs were important to my thinking because they sufficed to show that the broken-up early M set pictures were compatible with connectedness.”127 It would be a simplification to call this surprising statement a mere contradiction. First and foremost, it emphasizes the intellectual struggles that computer-generated mathematical images were able to cause and the ambivalence that they triggered. Second, it demonstrates an attempt to prove after the fact that his thinking was actually correct despite his previous erroneous assertions. Given the importance Mandelbrot accorded to the virtues of pictures in all of his work, his visually triggered misinterpretation in the first publication of the Mandelbrot set in the “Annals of the New York Academy of Sciences” in 1980 must have sat like a thorn in his flesh.

Dr. Mandelbrot’s islands and the issue of their visualization played a key role in his work as a mathematician. On the one hand, they illustrate the virtues of the “unspecialized” eye and thus of the interdisciplinary effectiveness of visual perception. On the other hand, they highlight the seductiveness that adheres to computer-generated visualizations of formulas. In this way, they get to the heart of the question of how pictures are able to convey knowledge and can be seen as symbolic of the two opposing, centrifugal forces involved in pictorial thinking.

The formula “Seeing is believing” fails to give due credit to the particular role that the visual plays in the process of scientific cognition. Pictures have to raise doubts and stir the imagination if they are to generate understanding. In that way they can serve as the raw material for thinking or as thought thresholds for the mind.

The Linear Makes the Non-linear Visible: Drawing Is Seeing

Drawing is a discipline of vision.128
—Edward Hill

If one wants to pursue the investigation of the interdependency of the visible sphere and the world of mathematical ideas, it is especially fruitful to continue to look for interactions between early computer-generated pictures and hand drawings. As can be seen in Mandelbrot’s work, one of the capacities of drawing is the authentication of computer-generated results. But the potentiality of drawing goes far beyond this function. To identify other epistemic roles of drawing, it is now time to leave the work of Mandelbrot and have a closer look at the practices of some of his contemporaries as a comparison, and first of all, at the work of Adrien Douady.

Not all mathematicians who worked with computers shared Mandelbrot’s conviction that computer graphics had “brought back the eye” into mathematics. Douady, a French Bourbakist, an expert in the field of complex dynamics, and the man who first proposed naming the Mandelbrot set after Benoît, was of the opinion that pictures and the eye had never wholly disappeared from mathematics insofar as geometry-oriented mathematicians had been making drawings all along. He reported that even when the dogma of iconoclasm was in ascendance, the visual had found secret yet effective ways into the discipline. Mathematicians had been known to climb rooftops in Paris to draw gigantic diagrams with crayons on walls and to make sub rosa sketches on pieces of paper under tables during lectures.129 He stressed that “the eye” had only disappeared from the sphere of publications (“expression”) but never from the sphere of thinking and the creation of ideas (“conception”). As an experimental tool, the computer had revealed new structures, to be sure, figures that one would never have been able to see otherwise—like all the visualizations of iterated polynomials that Mandelbrot had produced in 1979—but that alone had by no means facilitated a mathematical seeing: “Sur cette donnée brute, vous n’y voyez rien.” (On this donnée brute, you don’t see anything.)130 The French term “donnée brute,” which Douady uses for digital pictures both seen on the screen and as printouts, can be translated as “raw data”; it refers to pictorial material that has yet to be processed analytically. A second meaning of the term, the “raw given,” makes it clear that here the mathematician is operating from the start not only with data but with something visually concrete.

In their basic building blocks—and because of the low resolution of computer graphics in the early 1980s—representations of Julia sets consisted of exploded black rectangles that only outlined raw shapes; they had yet to be made accessible for seeing, at least in the sense of mathematical comprehension. Computer-graphic representations of formulas contained a surfeit of information, which Douady translated in an epistemological three-step process from a digital to an analog medium in order to interpret them. One of Douady’s worksheets shows how he first placed the “raw,” “unpolished” rectangles derived from a formula on a sheet for preliminary consideration (on the left), then drew connections between some of those tiny rectangles (in the middle), ignoring others or accenting them with small circles (interprétation). In a third step, seen at the far right of the picture, he reconstructed and abstracted the information it obtained in a branching line. That line, employed as a tool in the analysis of the dynamics of Julia sets, is reminiscent of a curved branch, sharply bent at the ends (arbre plongé), that would later come to be known as a Hubbard tree.131

Douady’s drawing was an independent tool for understanding that played both a mediating and a constructive-transformative role. The mathematician first explored the computer-generated picture with his pen. Then the independent line itself, no longer embedded in the coordinate system, became a subject for mathematical investigation. In a radical reversal of the Platonic dictum that discursive thinking (διάνοια) requires no visible forms,132 the drawing became both a tool that helped Douady to understand the computer-supported mathematics of complex dynamics and a mathematical object to be analyzed by his διάνοια.

The three steps from donnée brute to drawing recall the function of disegno in early modern times, as employed, for example, by the artist, biographer, and theoretician Giorgio Vasari in the mid-sixteenth century in a preliminary drawing for his fresco The Studio of the Painter (after 1561). There the viewer gazes into a pair of barrel-vaulted spaces connected by a corridor and doors. On the right, female models are assembled in front of a multi-breasted Diana Ephesia meant to embody the laws of nature. They are about to step over into the left half of the picture, where a painter is working at his easel, his gaze directed at additional posing models. The center compartment contains a niche holding a male statue, identified by its attributes and triple profile as a personification of disegno. Vasari here translated his theory of art, formulated in the second edition of his Lives of the Artists (1568), into a concrete pictorial allegory: As a draftsmanly step in the development of an idea, disegno, the basis and source of all arts, plays an intermediary role between nature and art.133

Douady’s conversion of the computer picture into a curving line translated the classic use of disegno into a method for generating a mathematical theory. In a “corrective overview,”134 the mathematician extracts a mathematical concept (concetto) from the “raw given” by means of drawing. In both cases, it is only the drawing that makes possible the process of visual transformation. As opposed to Vasari’s practice, however, the end product of draftsmanly conceptualizing in mathematics is not a unique work of art or the perfect image but rather a theoretical notion coagulated into a drawn line, a “snapshot” of knowledge, which is fed into the thinking process and used for further deliberations.

Once the telling lines had been extracted from numerical mathematics, they could function as a way of stemming the tide of serial similarities characteristic of images in this field of study. The repetition of nearly identical images can be tiring. Douady relates that he once almost fell asleep in a Paris café in 1982 when presented with a series of similar pictures composed of mathematical colleague John Hubbard had placed before him.135 Fresh from the printer on so-called continuous paper perforated at the edges, they represented seemingly endless variations of Julia sets, graphic depictions of rational functions produced by a dot-matrix printer.136 Hubbard was unable to make out the significance of the pictures and so had confronted his colleague with a whole stack of computer graphics. Douady tells of the encounter in such a colorful way that it is worth quoting him directly: “He asked me: ‘Can you recognize anything?’ I had nearly fallen asleep, and could only see a succession of clouds that said nothing to me. Suddenly I said to him: ‘There, an upside-down rabbit!’—‘Where?’—I took an orange marker and traced it in the picture.”137 Douady had observed the succession of similarities, and it was only the recognition of a shape and his drawing that relieved him from boredom. The “rabbit” that Douady felt he could see in the clouds of dots was a playful term for a specific quadratic Julia set, the characteristics of which he had intently studied for years. It owes its name “fractal rabbit” (lapin fractal) to what appear to be ears extending from either side of the globular body.138 In his draftsmanly retracing of the lapin, the scientist was treating the digital form as if it were clay or rubber: The rounded center section was taken apart and the “ears” were folded down along the inner edges and placed in perfect alignment with the sides of the printed pattern. Douady had turned the graphics into usable material for thought. In the process of his trying to understand the data before him, the line derived from the cloud of pixels came to seem like a plastic material, pliable and ductile. While the drawing in the first example helped him to review the basic structure, he now adapted the branching line of the Hubbard tree as if it were made of wire and could be bent at will. What was of importance mathematically was the way it converged in the center of the picture with the rabbit’s sweeping semicircular arcs and the loops surrounding the image.

Despite its humor and subjective, anecdotal character, Douady’s description of his discovery is of central importance in the history of the representation and understanding of complex dynamics. What Douady and Hubbard first discovered together in pictures they were later able to transform into a valid mathematical proof. They gave the name “matings” (“accouplement fractal”) to their theorem because the mating of two Julia sets of different classes of equations completely fills the field of complex numbers that mathematicians refer to as the Riemann sphere.139 Although the proof had to be worked out in an analytical way based on formulas, Douady insists that it was precisely the act of drawing that led to the mathematical insight: “And just at that moment, quite suddenly, as soon as we had made the drawing, we understood what was going on.”140

In his handwritten chronology of the events, Douady emphasized the moment of recognition with a drawing interrupting the lines of text. Its shape, somewhere between an oval and a heart with loops turned inward, gives the impression of an eye gazing out between the lines. It can be regarded as a symbol for drawing as a thought process in mathematics.

A number of Douady and Hubbard’s drawings for the “matings” are known. One of their preferred methods in the early 1980s was drawing on semitransparent foil. In figure 35, one can see how Douady traced different curving lines in red, blue, green, and violet. The “rabbit” is visible in red. The drawing is anchored in the transparent picture space by two slightly curving vertical lines. Vigorously curving loops and loosely distributed circles alternate with trial lines, some of which are struck through with hatching. These are traces of mathematical exploration that constitute a virtual allegory of the process of cognition, of its tentative searching, calculating, making of mistaken assumptions, and experiencing of reversal and reinforcement. Here, only by means of drawing, mathematical reflection could be seen as both controlling the digital space and becoming independent of it: With the use of foil as a transparent picture support, the visualized theory could be literally liberated from the mass of raw data. This drawing of course was not the proof, but it did provide evidence of what could later be confirmed. This is the core notion of the second possibility presented in this essay: that it is hand drawing that enables mathematical understanding in the realm of chaos, where serial pictures form sequences of similarities.

Analog and digital techniques are by no means mutually exclusive in fractal mathematics; they require and support one another. The drawing is not replaced by computer graphics; it enhances them and makes them intelligible. In fact, it is precisely owing to computer graphics that the pencil has become the mathematician’s indispensable thinking tool when extracting a theoretical concept from the mass of visual data.

Douady, a skilled draftsman, produced a large number of other versions of his lapin. Made up of innumerable curves, curlicues, spirals, swirls, and snaking lines, his lapins explore geometrical relationships that can be proved mathematically, to be sure, but cannot be calculated and depicted by any computer. The center composite figure, labeled “the fat rabbit,” is composed of five larger almond shapes and small, rounded buds. This basic figure is transposed into new configurations of serpentine lines in the surrounding drawings.141 Douady’s imaginative figures, drawn with great verve, show that research in this field of mathematics, first opened up by computer graphics, involves working with pictures even when the computer “blinds out,” the formulas having become too complex for digital representation. At the time Douady drew them, the various curlicues he imagined were as yet nonexistent in a mathematical sense inasmuch as their mathematical properties were not accompanied by a rigorous proof. The proof would be the job of someone else: What Douady’s drawing disclosed was a realm of possibility, a potential mathematical scenario.142 Dan Sørensen, a doctoral student of Douady’s, finally provided the proof of the scenario’s existence—and of the efficacy of Douady’s loopy imagination—in his dissertation.143

In Douady’s sketches, the backward S-curve that the Bourbakists placed in the margins of their Éléments de Mathématique celebrated not only a return but an ecstatic revival. They also came to be symbolic of the factual force of drawings, for it was possible to prove mathematically the existence of the curlicue dynamic. Douady can thus be identified not only as a second-generation Bourbakist, but also as an exquisite Bourbakist draftsman.

Computer-graphic representations of fractals, especially those based on combinations of polynomials, are characterized by complex filigree structures around the edges. In order to study the relative position of their individual components, mathematicians draw “thought trees.” For example, as a combinatorial study, a drawing by Jean-Christophe Yoccoz from 1983 represents a small portion of the Mandelbrot set as mathematicians “see” it.144 In his drawing, the calculation becomes a kind of hybrid between a tree and a molecular structure that fills the entire sheet. A base function is seen at the bottom edge of the picture, a curving line reminiscent of a small hill. Rising up out of it is a gnarly trunk with straight and curved branches that sprout tiny twigs in every direction, an organic tangle whose leaves or fruits are numbers. Each of the tree’s branches is the expression of a calculating thought movement, and together they result in a complex dynamic of their own: The upper right is dominated by rhythmic lines curving to the left, repeating the movement of the exuberant formation in the upper left. Douady explained the combinatory practice as “tree drawing,” which represents one of the basic methods for working with polynomials and serves to tame the structures visualized in computer graphics.145 Mathematical draftsmen thus become engineers of a computer-generated observation room; in drawings, digitally calculated abstract space is surveyed and virtually reconstructed. Computer graphics may have been required to disclose these realms to mathematics, but in scientific debate they had to be interpreted in drawings. Though the computer helped access a realm of pure thought in which the world of numbers might map itself seemingly unaided, drawing was essential for the return of the eye, which in Mandelbrot’s view was primarily manifested in digitalized mathematics.146

Drawings will always play an indispensable role in mathematical thinking at the theoretical stage. Yet the popular media are dominated by computer graphics that by and large mask the process of coming to understanding through drawing. To the media public, the presumably objective computer graphic is better at illustrating phenomena from the exact sciences. But the shape of the graphic has most likely been designed by a drawing hand, outside of public view, and only later presented in polished form.

Of course, it is possible to think in formulas—or, like Mandelbrot, in analogies— but arriving at understanding can also be a matter of fiddling with pencil and paper, making drawings that lead to a fruitful collaboration between the computer and the eye and thereby add to mathematical knowledge.147

Douady ended the article in which he presented his theory of mathematical disegno in 1982 (published, ironically, in the Séminare N. Bourbaki) with a highly technical demand: Construction on the foundations of the “chirurgy of dynamic systems” must continue.148 His wording can be related to a humourous photomontage from his private archive, a spoof on a French leftist poster of the French Communist Party (PCF) in which John Hubbard and Adrien Douady have taken the places of the Gaullist politicians Jacques Chirac and Charles Pasqua. Douady’s left hand rests on a three-dimensional drawing of a Julia set, which appears to hover slightly above a table (where in the original had been a map of France), while Hubbard is brandishing a butcher knife in his right hand. They are gazing at the viewer with determined expressions. The two mathematicians’ intention is spelled out in large letters next to their heads: Charcutage Polynomial (Polynomial slaughter). This poster is a lively indication of the sparkling sense of humor for which Adrien Douady was widely known and appreciated. But there is a deeper meaning hidden in this joke: Had the two mathematicians not laid a hand and a pencil on the “slaughterhouse of the Julia sets,” they would not have derived any information from the raw digital shapes of visualized mathematics. In this context, drawing became a mode of mathematical seeing: It was the linear that made the non-linear visible.

Material for Thought: Paper as a Threshold to the Mind; or, Paper Poiesis

A mathematician, on the other hand, has no material to work with but ideas.149
—Godfrey Harold Hardy

Anyway, I think with my knee.150
—Joseph Beuys

Experimental thinking, the cultural historian Thomas Macho has explained, does not take place solely in the mind: “Intellectual experiments are generally experiments on paper. We have to produce Flatland on our writing tables in order to be able to give an account of it.”151 Macho was of course referring to Edwin A. Abbott’s 1884 novel in which the author describes a world where geometrical shapes play the main roles. Ninety years later, Otto E. Rössler’s way of using pictures would provide an indication of how fundamentally important material evidence can be to experimental thinking in mathematics. With Rössler, our concluding case study is devoted to one of the most radically conceived and implemented collapsings of the boundaries between the world of thinking and the world of things that I encountered during my research in this field. We have already seen that in their mathematical thinking, Benoît Mandelbrot and Adrien Douady employed pictures in quite different ways. Rössler, in his work on the non-linear dynamics of complex systems, utilizes physical representations in his thinking in yet a third way. As with Douady, drawing is of utmost importance, but with Rössler, its emphasis expands dramatically from a focus on the visual side to its material qualities, which, in the manner of a feedback loop, directly influence the properties of the mathematical objects under study.

Rössler, a biochemist by training, is considered a pioneer in the field of “chaos theory” because of his mathematical works from the mid 1970s.152 For the moment, the term “chaos” has deliberately been placed in quotation marks, as it conveys something very different to mathematicians than it does to laymen, for whom it is synonymous with “confusion” or “lawlessness.” Chaotic systems form an order at the highest level. Accordingly, a more precise term is “deterministic chaos.”153 “Chaotic” means “not precisely predictable in the long term,” to be sure, but by no means “arbitrary” or “unruly.” Scientists working in this field have discovered forms of chaos as varied as they are precise. At the same time, they have suggested making chaos to a certain extent controllable; pictures of chaos are therefore always also “images of controllability.” The field of fractals is linked to that of deterministic chaos. It can be thought of as a geometry of chaos. Thus, if deterministic chaos describes a temporal complexity, fractals deal with a spatial complexity.

In 1975, Rössler discovered an “attractor,” a key concept in dynamics. Attractors can be described as spatially complex geometric shapes. In simple terms, they illustrate all the states a dynamic system can pass through over time. They are symbolic representations of long-term predictions. If they are chaotic, they are called “strange.” The “Rössler attractor” is one such “strange” shape.154 Rössler was deliberately searching for a “simplification” of the chaotic model discovered in 1963 by the meteorologist Edward Lorenz at MIT.155 The Rössler model is extremely important in chaos research because it is reduced to the minimum requirements necessary to exhibit chaotic behavior and therefore keeps analysis of the dynamic system as simple as possible.

To explain how he first developed this shape, Rössler drew spiral-like tracks on two sheets of paper. He then proceeded to manipulate them in complex ways in three-dimensional space in an effort to understand them. By studying them one above the other but at an angle, he managed to “force in the paper”156 a figure whose structure approaches the limit of what can be represented and imagined: an infinitely flat, two-dimensional thing described by the movement of a one dimensional point that nevertheless occupies a three-dimensional space. It was only the manipulation of the two sheets of paper in various positions that gave him the sudden idea that the desired shape could only be realized by a dimensional leap. Once the line had arrived in the third dimension in its temporal development, it necessarily had to be catapulted back into two dimensions. Since it could be continually extended and retracted between the two sheets of paper arranged in space, it was indeed possible to return it to the second dimension.

This account of his research methods and the circumstances in which he made his discovery in December 1975 dates from some thirty years after the events. Looking back, Rössler confirmed that it was in making the drawings and playing with them in space that he understood what he was after analytically; his subsequent development of the related formula—“the intrusion of a wedge of proof”157—was merely a mechanical act of realization. “[E]verything that one can imagine one can also draw,” Rössler claimed, “and indeed much better, because in drawing one has what is essential… . It always begins with the pictures.”158 To emphasize this mechanism, Rössler noted a provocative triad that he had written on one of his sketch sheets: “forcing— snapping—death bite” (“erzwingen— knacken—Tötungsbiß”). Retrospectively surprised by the strong metaphors he had invented, he later commented that this “death bite” would certainly be a “friendly” or “natural” one, as achieved, for example, by the reflexes of a dog. It was as if the chaotic attractor had to be tamed in an actual tour de force, or in a process resembling “catching the prey.” One could dismiss Rössler’s description of his discovery through drawing as a fanciful projection after the fact were it not that recently discovered sketches document the process in detail. In scientific practice, preparatory sketches leading to a discovery are generally trashed, hardly worth preserving once the thinking process is finished. The preservation of this series of drawings is accordingly a historic stroke of luck. Dating to the period between November 23 and December 11, 1975, they provide eloquent testimony to Rössler’s habit of thinking by drawing. One sees how he took up the basic shape of the Lorenz model—two interconnected spirals— in the first drawing and then varied it step by step. He stretched the shape into an oval, deformed it into wavy shapes, and pressed it into the bottom corner of the coordinate system.159 Somewhat offhandedly, he wrote, “soft watches (Dalí)” on another drawing. What was the name of the famous Surrealist painter doing among Rössler’s working sketches? One has to know that in chemistry there are cyclical reactions known as “chemical clocks.” On the left in figure 40, one sees a “diagram clock” that Rössler drew like an ear being struck by sound. Yet this equation does not produce a chaotic reaction, for in order to trigger one, the materiality of the image from chemistry—the clock—would need to be literally made “to melt,” as in the famous Dalí painting The Persistence of Memory (1931). This is explored in the other Rössler sketches here. Folding and stretching had already been identified in mathematics as characteristics of chaotic dynamics in the sixties; what was new was that by transferring the material malleability of the stable “chemical clock,” a “chaos machine” might be created.160 The idea that the material might be deformed was crucial to Rössler’s subsequent discovery of the chaotic attractor through drawing. On the sheet with the “soft watches,” there is also a sketch illustrating the step that immediately preceded Rössler’s discovery. It pictures the loops, now layered one behind the other in space, placed in a position to make possible the decisive step: the dimensional leap that Rössler characterized as the switch between two sheets of paper. In a small and seemingly unimposing sketch in the center of the last sheet, one sees the dimensional leap taking place in the more darkly hatched area between two broken lines. Once Rössler had become aware of the ramifications of his discovery, he marked the small spiral drawing quite clearly with a green arrow—a kind of ex post facto exclamation point extolling the virtues of working out one’s thinking on paper.

It was only after working out this series of simulations on paper that Rössler managed to come up with the equations for his “handmade” chaos model and to visualize them with the help of his analog computer.161 The crucial preliminary role of his “paper models” and the relationship between the substance of his thinking and that which his hands could touch was highlighted still further when in 1977 he attempted to expand his model into the fourth dimension. To conceive of a new dimension at all, his hands had to first experience it intuitively. His experiments were now carried out with paper reshaped in space. Once again, a dimensional leap was required in the mathematical model, one that he could only grasp by rolling the paper.162 Rössler published drawings of this paper model in the Bulletin of Mathematical Biology.163 Thanks to the physical and tactile aids in his research, he was able to “pre-envision” what could only later be depicted in computer graphics with the help of formulas and algorithms. The computer alone would not have sufficed. Rössler’s colleagues greeted his procedure with amazed praise; as the mathematician Bruce Stewart put it: “What was brilliant about what Otto Rössler did was that he conceived of the shapes before the equations.”164

Rössler’s procedure can thus be seen as a radical expansion of Douady’s position. Not only did the drawing trigger the sudden moment of understanding, the material substance—the paper model—seemed to sculpt the supposedly immaterial digital image itself. This opened up his thinking to the realm of possibility: “It is a mental experiment, but it is always real things that one manipulates.”165 The rolling of the paper and even the scratching of the pencil can be identified as “material springboards to thought” or “thinking thresholds,” which influence the structure of creative thought, whether in art or mathematics. Rössler was, moreover, a sharp critic of digital pictures. Not only were they incapable of achieving a return of the eye into mathematics; strictly speaking, he insisted, they were not even “pictures”: “A picture that is developed by way of algorithms is not truly a picture.”166

Rössler describes the process of arriving at understanding by handling paper or drawing on it as one that has audible, even physical aspects: “If one makes a drawing on paper, or on two sheets of paper, at some point it snaps into place, it clicks. One has to force it, ‘adjust’ the shape so to speak. It then clicks audibly into place: It makes your ears pop.”167 A mental image takes shape through the gesture of drawing: It is a physical act just like any other physical experimentation, and as such it leads to understanding. Rössler quotes the behaviorist Konrad Lorenz’s well-known statement that “Thinking is handling things.”168 In his reports, Rössler repeatedly relates how his own body becomes involved in his experiments. For example, he tells of a “nose perception” that he claims occurred as he was making his discovery. He had the feeling that a string kept winding around and around his nose. The figure of the spiral attractor taking shape in his imagination was virtually assaulting him. The researcher’s own body here becomes a virtual “support” for the mathematical objects in defiance of the Western belief that the physical plays no part in “objective research.”169

There is, at times, an air of the esoteric or even occult in Rössler’s descriptions of his working method, one that is symptomatic of the intersubjectivity of mathematical picture worlds in general. To mathematicians, pictures that lead to understanding are often difficult to describe. The French mathematician Jacques Hadamard, who described his thinking in images as sequences of “undefined specks,” provides a famous historical example. In his proof that there are prime numbers greater than 11, his first step read: “I consider all primes from 2 to 11, say 2, 3, 5, 7, 11.” To his thinking, this was equivalent to saying: “I see a confused mass.” Then in a final step, the closing solution: “I see a place somewhere between the confused mass and the first point.”170 One has to imagine the intervening mental steps—that is to say, the path from “confused mass” to the previous point—as moving images or a dynamic sequences of pictures. It is not known whether or when Hadamard made drawings. But his insight that mathematical thinking occurs in visual and not always clearly definable images unites him with Rössler.171 In Rössler’s case, however, one has to add that these “mental images” can no longer be separated from the material world. As a process, thought itself becomes material, and this materiality of thinking can be seen as a habit that does not necessarily have to be conscious: “You just introduced the term haptic—that was a new formulation for me, but you are correct. It is only new to me as a name, not as experience.”172

In this, Rössler can be said to agree with Friedrich Nietzsche that “our writing tools influence our thinking.”173 Whereas Nietzsche loved to “think while writing,”174 to Rössler there is something erotic about pictures: “Real pictures are always erotic pictures as well. Topology or geometry is like looking from all sides as though in outer space. That is all completely tactile. Without touching and without kneading it is impossible to imagine. Materiality does matter: Essentially mathematics is nothing more than pottery.”175

One has to resist confusing Rössler’s creative process as a mathematician with that of Prometheus, who as a skilled sculptor is said to have fashioned mankind out of clay. Rössler’s approach is more reminiscent of the “organic machines” designed in the mid-seventies by the artist Joseph Beuys, in which materiality was basically perceived from the point of view of its energy potential. In Honey Pump in the Workplace, created by Beuys for documenta 6 in 1977, the artist pumped honey, conceived of as a source of energy, through a system of plastic tubing. With this “model of man,” Beuys designed a “technology … that is first of all, in fact, an internal one, that is a conceptually creative one in mankind himself.”176 Just as the circulating honey can be understood as the circulation of blood of such an “organic machine,”177 Rössler’s rolling of paper sheets was a way of designing chaos by thinking in material terms. Prometheus molded man out of clay so as to breathe life into him and give him the ability to reason. Rössler molded paper as a way of opening up to his thinking new realms of possibility.

Epilogue: Interfaces

The mathematical mind has always seen itself confronted with the problem of generating sense out of sequences of similarities. Here we have been able to distinguish three basic ways of pictorial thinking by examining the working materials of three different scientists. Benoît Mandelbrot elevated analogy to a guiding principle and saw new pictures of the world taking shape in oscillating similarities, while his islands pointed to the double-edged, imaginative, and seductive power of the visual and identified drawing as means of authentication. For Adrien Douady, the drawing of a line as a kind of mental sword through the drift of dots in a series of similar computer graphics led to a breakthrough in understanding. To Otto E. Rössler, the experimental manipulation of things was equivalent to thinking—and all the images in the various configurations of his sheets of paper were one and the same picture. His research method attested to the dignity of the single picture as such. Rolling his sheet of paper while pondering four-dimensional chaos, he was not merely rolling paper. His thinking was rolling, or alternatively, he was thinking the rolling, or perhaps even better, the rolling was thinking him. As Maurice Merleau-Ponty was also aware, “One has to understand that things own us, not we things.”178

Creative processes in art and the methods of the sciences both suggest that mental and physical activities are interdependent and influence each other. The practice of “thinking with one’s hand,” the subject of the present book and the related exhibition, coincides with Beuys’s thinking “with my knee”179—both are suggestive of a performative model of thought. It is simply mistaken to assume that there is an autonomous mental realm independent of the world of sense impressions, one in which ideas and concepts can be pursued without reference to the material world—whether in the practice of art or mathematics.

When trying to categorize more closely the evidence—or traces—that mathematical objects leave behind in the world, as Jean-Pierre Changeux proposed at the very beginning of this essay, it is only possible to conceive of them as interfaces between the world of mathematical thought and visible reality. It is their very double allegiance that gives them their potency and is the basis for their unique dynamic: They do not dissipate somewhere between the two spheres but leave behind traces, not only on paper but in thinking itself.