The Week in Examples #8 [14 October]
The State of AI, open source absolutism, and AGI ‘already here’
Welcome back to The Week In Examples, a weekly roundup of the most important news in AI ethics, safety, governance, and policy.
This time around we have the 2023 edition of the State of AI report (disclaimer: I was a reviewer for this year’s edition), an interesting essay about whether AGI is already here, and one-time Doom developer John Carmack’s comments about open source models.
As always, make sure to tell me what works and what doesn’t or just drop me a line to say hello at hp464@cam.ac.uk.
Three things
1. It’s that time of year again: State of AI 2023
What happened? Nathan Benaich and the team from Air Street Capital have released the latest version of the annual State of AI report, split into five sections spanning research, industry, politics, safety, and predictions for the year ahead. The research section focuses on large models, with discussion of GPT-4 and its competitors, while the industry section is focused primarily on compute. The politics section, meanwhile, tackles different models of national AI regulation, geopolitical tensions between the US and China, and the relationship between AI and defence. Finally, the safety section looks at the popularisation of catastrophic risk, the tensions between open and closed approaches to AI development, fundamental challenges associated with RLHF, and problems in benchmarking powerful models.
What’s interesting? The predictions are probably my favourite section of the report, and I like that the authors review how well they fared on last year’s attempts at getting ahead of a notoriously difficult field. Of the nine predictions made in 2022, five were correct, three were incorrect, and one was partially correct. Those that came to pass included forecasts around data scaling, audio generation models, big tech investment, and a commercial deal between an AI developer and a user-generated content site. They also marked the prediction that $100M+ dollars will be invested in an AI safety company as correct by pointing to Amazon’s investment in Anthropic, which I think is probably on the generous side given the firm’s pivot away from its original focus on safety.
What else? The State of AI report is a good opportunity to step back and look at the progress of the field in the round. To zoom all the way out, 2023’s effort makes it pretty clear all of the major players are organising around the large model paradigm. This means that the single most important factor for determining capability is the amount of compute that you have access to, with everything else secondary (as far as performance is concerned). The fact that, according to the report, usage still isn’t as sticky as other consumer tech products gives us an opportunity to make sure models are developed in the safest way possible. More optimistically, though, is that convergence on the large model approach means it should be possible to build standardised approaches to AI governance to make that happen.
2. And is ‘AGI’ in the room with us right now?
What happened? Google Research’s Blaise Agüera y Arcas and Stanford’s Peter Norvig (also a former researcher at Google) argued that, while today’s most advanced AI models have many flaws, they will be regarded as early manifestations of artificial general intelligence in the future. As they explain: “decades from now, they will be recognized as the first true examples of AGI, just as the 1945 ENIAC is now recognized as the first true general-purpose electronic computer.” The point here is that, though there are clear limits, frontier models demonstrate the core property of generality through a command of a large number of topics, tasks, languages, and modalities. They are also capable of ‘instructability,’ which refers to the idea that frontier models are capable of learning from real-world interaction after they have already been trained.
What’s interesting? I buy into the author’s perspective that we ought to think about intelligence in terms of a multidimensional scorecard, rather than a single yes or no proposition. It seems unlikely to me that there will be a single moment in time in which we all agree that AGI is here (in fact, given that people are already saying that it’s arrived, that means we can already discount that as a possibility). What is more likely is that the system will become increasingly capable and little by little more people will align on the idea that we’re dealing with AGI. This is similar to the idea put forward by Sam Altman comparing the release of large models to the release of the latest iPhone.
What else? The authors introduce four reasons why people generally do not describe frontier models as AGI: a healthy scepticism about metrics for AGI, an ideological commitment to alternative AI theories or techniques, a devotion to human (or biological) exceptionalism, and a concern about the economic implications of AGI. These are all compelling ideas, other than concerns about the impact of AI on the economy, which the authors don’t really unpack. For me, though, there are two other elements that we ought to consider. First, that—should the authors be proved right—we are at the very early stages of the emergence of a powerful new technology. Some people will recognise the significance of a particular technology faster than others, so we shouldn’t expect universal agreement over the very short term. That’s just how innovation goes. Second, though, is that researchers are still human. Part of the reason experts like to be critical is because they, like all of us, suffer from social desirability bias. Why call AGI now when you can wait and see?
3. Open source absolutism is here (to stay)
What happened? John Carmack, the man behind 1990s video games like Doom and Quake, reiterated his support for open source approaches to AI development. Carmack, who left the world of games behind to found AI firm Keen Technologies, said “They [people who favour a moderate approach to access] believe that giving free access to state of the art algorithms and models without any guardrails constitutes a danger to society, that the public can’t be entrusted with a research model that wasn’t hammered into a box of their designated dimensions.” He compared this to a time in the 1990s in which legal battles ensued to determine whether or not software providers could implement strong encryption, with the concern being that doing so would help bad actors to cause harm. Comparing that moment to our present, Carmack’s response was simple enough: “Be an Open Source Absolutist!”
What’s interesting? There’s a lot to say here, and the historian in me insists that I start with the historical parallels. The thing to consider here is that encryption wasn’t some battle that was conclusively won or lost in the 1990s, but rather an issue that remains unresolved even today. In the US and especially the UK, the long-standing dispute between law enforcement agencies and tech companies centres on whether a backdoor can be provided to read some messages without undermining the broader network (it can’t). It’s very much a live debate, and the fact that it continues in search of a compromise (even if a backdoor is infeasible) is actually pretty instructive for understanding the open source issue. As regular readers will know, my own view is that model access is not a binary. There should be compromises available to us that avoid maximalist, or indeed, absolutist, positions.
What else? To illustrate his point, Carmack suggests that open source approaches would prevent us from bumping into the all too familiar phenomenon of reading the phrase “As a large language model, I cannot…” The problem I have with this framing is that it seems to imply that the only way to avoid these sorts of guardrails is to download an open source model and strip out any safety measures. Unless you’re trying to do something blatantly harmful like look for a step-by-step guide to build a powerful explosive, I suspect that we will see the emergence of a plurality of models that have different guardrails for what they deem to be acceptable usage (much like social media platforms today). That aside, though, while some restrictions can appear to be a little overzealous, it is not all that clear to me that we should deem a bit of annoyance to be a price worth paying for empowering bad actors to engineer bioweapons or create sophisticated malware.
Best of the rest
Friday 13 October
Autonomous AI systems in the face of liability, regulations and costs (Nature)
Ukrainian AI attack drones may be killing without human oversight (New Scientist)
How a billionaire-backed network of AI advisers took over Washington (POLITICO)
Could an AI-created profile picture help you get a job? (BBC)
Dumbing down or wising up: how will generative AI change the way we think? (The Conversation)
Thursday 12 October
Foundation models in the public sector (Ada Lovelace Institute)
First word discovered in unopened Herculaneum scroll by 21yo computer science student (Vesuvius Challenge)
Friend or foe: Labour’s looming battle on AI (POLITICO)
Google to defend generative AI users from copyright claims (Reuters)
The hard questions about dangerous capability evaluations (Substack)
Wednesday 11 October
Recommendations for the next stages of the Frontier AI Taskforce (Apollo Research)
The New AI Panic (The Atlantic)
OpenAI’s technology explained (OpenAI)
How AlphaFold and other AI tools could help us prepare for the next pandemic (Nature)
It’s time to move on from chatbots (Substack)
Tuesday 10 October
Meta’s AI personas are here (X/Twitter)
How AI reduces the world to stereotypes (Rest of World)
Max Tegmark asks Hugging Face to stop sharing his copyrighted work (X/Twitter)
NVIDIA plans to crush competition (Substack)
The Geometry of Truth (arXiv)
Monday 9 October
G7 countries have agreed on guidelines for AI (POLITICO)
New Technology Requires New Regulatory Ambitions (The Regulatory Review)
The political geography of AI infrastructure (University of Oxford | Project)
Graphcore Was the UK's AI Champion—Now It’s Scrambling to Survive (WIRED)
Divide-and-Conquer Dynamics in AI-Driven Disempowerment (arXiv)