Reflections on AGI from 1879

Forgotten musings on the economics of AI, an intelligence explosion, and machine consciousness

Pierre Mornet, Illustration of George Eliot

George Eliot, or Mary Ann Evans as she was known to her friends, was one of the most influential writers of Victorian England. Amongst her greatest hits were The Mill on the Floss, a novel about two siblings grappling with family life, and Middlemarch, a meditation on marriage, idealism, and self-interest. 

Not many people have Eliot down as an influential player in the history of AI — and they would be right. Her contribution has been for most part overlooked, even as folks begin to unpick the role of fiction shaping the trajectory of the AI industry. I wrote a short piece for AI and Ethics, for example, about MIT’s computer scientists and their relationship with fiction, media, and the future. But Eliot, who wrote incredibly early (and as we’ll see, thoughtfully) on the subject of AI and society has yet to get a look in. That’s what today’s post is about: drawing a little bit of attention to her work and connecting her thinking to some of the longest running debates in the field. 

The vehicle for her treatment of AI was the 1879 Impressions of Theophrastus Such. The book, written as a series of conversations between the titular character and his acquaintances, is a satirical look at a whole bunch of themes like science, ageing, friendship, family, and religion. It’s generally considered to be Eliot’s most experimental work because it departs from the usual longform novels written as commentary on social life and characterised by psychological depth, keen observation, and detailed realism. 

In one of these conversations towards the end of the book, ‘The Shadows of the Coming Race’, we find Eliot’s treatment of AI. The chapter is structured as a conversation between Theophrastus and his friend Trost, with the pair exchanging ideas about how society is likely to adapt to the introduction of advanced machines. Trost is introduced as a person who “is no optimist as to the state of the universe hitherto, but is confident that at some future period within the duration of the solar system, ours will be the best of all possible worlds.” Very accelerationist indeed. 

This line more or less sets the tone for Trost, who as we shall see, believes in both the emancipatory power of AI to extract people from mundane jobs and difficult lives, while also freeing up cognitive labour for people to do more fun, meaningful work. Of course, we know better: in a world where machines are capable of doing any physical or cognitive task there aren’t exactly many roles that a human being can play that are additive to the economy. Each time a person engages in work that could be done more efficiently by a machine, the employing party—be that company, government or charity—will have to bear the opportunity cost of using AI. It’s like when a toddler ‘helps’ you wash the dishes. 

But we’re getting ahead of ourselves. In Impressions, Trost begins by assuring Theophrastus that dull, dangerous, and dirty work will “soon be done by machinery". Sectors that involve this sort of work have long been the target of automation, precisely because they provide clearly beneficial opportunities for AI to be used to boost safety and save lives. This is, by the way, a process that has long been underway in modern industry, with a 2020 report finding that the introduction of robotics “reduces work-related injuries by approximately 16%” in the manufacturing industry. Another study in 2021 found that “one additional robot in exposure per 1,000 workers decreased the accident rate at the mean by 15.1 percent” in American businesses. 

After the pair begin to consider the application of this dynamic to all jobs, Theophrastus has something of a realisation. As he explains, “Am I already in the shadow of the Coming Race? And will the creatures who are to transcend and finally supersede us be steely organisms, giving out the effluvia of the laboratory, and performing with infallible exactness more than everything that we have performed with slovenly approximativeness and self-defeating inaccuracy?” 

In response, Trost says that machines are “extensions of the human organism…limbs immeasurably more powerful, ever more subtle finger-tips, ever more mastery over the invisibly great and the invisibly small.” He argues that we shouldn’t worry because these ‘wonder-workers’ will obey human commands, necessitate human oversight, and require humans for their construction and maintenance. 

Theophrastus disagrees, arguing that machines could “be made to carry…conditions of self-supply, self-repair, and self-reproduction” to render the human race obsolete. This idea is essentially an early precursor to the intelligence explosion, made famous by the British mathematician Irving John Good in his 1966 article Speculations Concerning the First Ultraintelligent Machine. The intelligence explosion formed one of the core components of the now ubiquitous ‘technological singularity’ hypothesis, a hypothetical point in time at which technological growth becomes uncontrollable and irreversible. 

The first use of the term 'singularity' is attributed to the Hungarian polymath John von Neumann, with Polish mathematician Stanisław Ulam writing in his obituary that von Neumann once described to him “the ever accelerating progress of technology and changes in the mode of human life, which gives the appearance of approaching some essential singularity in the history of the race beyond which human affairs, as we know them, could not continue.” Much later, American author Vernor Vinge wrote of ‘the coming technological singularity’ in a 1993 essay of the same name, while Ray Kurzweil cemented the term’s popularity in his 2005 book The Singularity is Near.  

We should remember, though, that Good wrote over eighty years after Eliot. Although Eliot is writing fiction—and Good sketched the outline of what such a machine might look like and how it might evolve over time—we ought to consider Impressions of Theophrastus Such to be a meaningful contribution to the lineage of the concept. After all, both the idea of the ultraintelligent machine and the degree to which AI will enjoy productive, economic, and intellectual superiority over humans are discussed by Eliot. (As for the latter, Good addresses the economy frequently in his text, arguing that an ultraintelligent machine would be worth a ‘megakeynes’ in reference to the estimated 100 billion pounds of value of the economist John Maynard Keynes to the world.)

I’ve mentioned Eliot’s treatment of the economic implications of intelligent machines, but it is worth saying that the arguments go slightly deeper. After Theophrastus makes the case that machines could work any human job (including the production and repair of machines) Trost insists that machines will see “minds set free from grosser labour” to be occupied by jobs that are more creatively or intellectually satisfying. 

To make his case, he uses the example of scavengers in London who, through the use of AI, would be able to spend their time “concerned in the production of epic poems or great judicial harangues.” Given poetry and speechmaking don’t sound like particularly lucrative pursuits, there are two ways that I read his response. It is either that machines will allow the creation of an economy where these pursuits are a) deemed to be economically valuable and b) unable to be fulfilled by AI; or that the introduction of machines en masse has created so much wealth that is redistributed to allow workers to undertake these sorts of pursuits. Had Eliot been more explicit here I would have connected it with Henry George (who was also a contemporary of Eliot’s) and his proto-UBI. But she’s not, so I won’t.  

As for the nature of intelligence, Impressions doesn’t use the term specifically. It does, however, get into minds, brains, evolution, and consciousness (more on those in a moment). The closest Eliot gets is in a passage whereby Theophrastus responds to Trost stating that, with respect to the creation of new economies, “the sequences it [a machine] carries out throughout the realm of phenomena would require many generations, perhaps aeons, of understandings considerably stronger than yours, to exhaust the store of work is lays open.” Theophrastus then introduces the analogy of the carp’s relationship to humans in response:

"But I, from the point of view of a reflective carp, can easily imagine myself and my congeners dispensed with in the frame of things and giving way not only to a superior but a vastly different kind of Entity. What I would ask you is, to show me why, since each new invention casts a new light along the pathway of discovery, and each new combination or structure brings into play more conditions than its inventor foresaw, there should not at length be a machine of such high mechanical and chemical powers that it would find and assimilate the material to supply its own waste, and then by a further evolution of internal molecular movements reproduce itself by some process of fission or budding. This last stage having been reached, either by man's contrivance or as an unforeseen result, one sees that the process of natural selection must drive men altogether out of the field.” 

This takes us to the next area that Eliot considers: the relationship between biological and synthetic beings in the context of evolutionary pressure. Having made the point that species with more intelligence are more likely to be successful, Theophrastus suggests that humans, the ‘feeble’ race, will “vanish as all less adapted existences do before the fittest.” Lots has been said about the question of evolution as it relates to AI and AI safety, with perhaps the recent well-known intervention coming in March last year arguing that natural selection favours AI over humans. Here, Eliot sides decisively with the contemporary literature (or perhaps it is more accurate to say the contemporary literature sides with Eliot).

Finally, the conversation turns to consciousness. Theophrastus closes by asking whether human consciousness is a ‘stumbling’ on the way to ‘unconscious perfection’. The idea here is that consciousness is a bug rather than a feature, and that it is not clear whether a conscious species will be any more effective at achieving goals than an unconscious one. Put simply: intelligence is not the same as consciousness. 

This is a debate that gets a decent amount of airtime today, with lots of people in the AI industry getting put out about arguments that conflate the two. But, for the avoidance of doubt, they are not the same thing. My own (simplified) view is that intelligence is the ability to achieve goals and consciousness is the phenomenon through which we experience qualia, but that is for another post. 

In any case, Theophrastus goes on to demonstrate that he—and, if I had to guess, Eliot—absolutely understands that intelligence and consciousness are not one and the same. To show that beyond doubt, the chapter closes with a provocation: “this planet may be filled with beings who will be blind and deaf as the inmost rock, yet will execute changes as delicate and complicated as those of human language and all the intricate web of what we call its effects, without sensitive impression, without sensitive impulse.”     

Ultimately, Impressions proves both the value of fiction as a mechanism for unbraiding some of the trickiest (and oldest) questions about the arrival of intelligent machines and shows Eliot to be something of a clairvoyant. That the chapter considers the economics of sophisticated AI, grapples with the nature of intelligence and self-improvement, and gets under the skin of machine consciousness is both impressive and concerning in equal measure. It reminds us that, no matter how novel our ideas may seem to us, someone else probably got there first.