Video generation, antagonistic AI, and compute governance [TWIE]
The Week In Examples #26 | 17 Feb 2024
Some housekeeping: I’m on holiday for the next two weeks, so this is the last The Week In Examples until 9 March. Instead of the usual roundup, I’ll be sending a short history of the National Institute of Standards and Technology in two parts on 24 Feb and 2 March (these are posts from the Learning From Examples archive — but they haven’t been sent out via Substack). Look out for those if you want to read a bit about the evolution of NIST since the early 1900s and the introduction of its ‘standard reference’ versions of items like bullets, cigarettes, or (yes, really) shrimp.
As for today, we have research making the moral case for antagonistic AI, new video models from OpenAI and Meta, and a paper assessing the governance of compute. As always, it’s hp464@cam.ac.uk for comments, feedback or anything else!
Three things
1. Sky’s the limit for video generation
What happened? OpenAI showed off a new video generation model, Sora, through a blogpost and a series of demos on X. Sora (which translates to ‘sky’ in Japanese) can generate videos up to a minute long that seem to far exceed the current publicly available state of the art using a version of the popular diffusion model architecture that involves generating and removing static noise. The outputs are undeniably impressive, which I suspect is the reason that the announcement caused more backlash than these kinds of demos usually tend to get (see here, here, and here). I imagine this type of response will increasingly be the norm for new AI models as capabilities get good enough to make people feel worried about their livelihoods, though for Sora, a film-maker friend of mine felt video generation models were likely to create more jobs than they displace in the industry.
What's interesting? But it’s not really that clear what the economic impact of extremely capable models is likely to be over the medium term. In lieu of good evidence, we could do worse than looking at the introduction of robotics to industries like manufacturing. I’m simplifying, but lots of studies tend to show that robots increase productivity (Acemoglu and Restrepo, 2020) in the round, though only by way of labour displacement effects (Dauth et al., 2019). Manufacturing is very different to film-making, but the comparison induces two core questions: where exactly will labour displacement effects be concentrated, and to what extent will they be masked by broader economic benefits connected to the introduction of AI?
What else? It was a big week for video models, with Meta announcing the release of V-JEPA, a method for teaching machines to understand and model the physical world by watching videos. The emphasis on the ‘physical world’ here is especially interesting given OpenAI also singled out the ability of Sora “to simulate some aspects of people, animals and environments from the physical world” in its technical report. Despite current limitations related to physics modelling, the company said “continued scaling of video models is a promising path towards the development of capable simulators of the physical and digital world, and the objects, animals and people that live within them.” In other words, expect impressive text-to-3D work in the not too distant future.
2. AI has to be cruel to be kind, say researchers
What happened? Researchers from Harvard and Université de Montréal wrote a paper arguing that antagonistic AI systems—defined as those that are deliberately disagreeable—may demonstrate benefits such as forcing users to confront assumptions, build resilience, and develop healthier relationships. The basic idea is that techniques such as reinforcement learning from human feedback (RLHF) designed to align AI with human values are deployed in a way “assumes that subservient, ‘moral’ models…are universally beneficial — in short, that good AI is sycophantic AI.”
What’s interesting? The motivation behind this research strikes me as quite similar to that behind GOOD-Y 2, a model so safe that it refuses to answer any user questions. In this paper, though, the authors convincingly argue that human beings benefit from being challenged. It seems odd to write that sentence given how self-evident it is, but—as any interaction with certain AI systems shows—being “good” isn’t always a good thing. Perhaps it would be better to aggressively convince me to avoid buying that takeaway rather than making a polite suggestion. Maybe I want to practise a difficult conversation with all of the unpleasantness that doing so brings with it. And at the risk of saying something truly controversial, it may even be that it is better to challenge certain behaviours than it is to let them be.
What else? The paper shows us the fundamental problem at the heart of projects to democratically align AI. Put simply: the seemingly straightforward idea that a model ought to be—to use an approach favoured by some developers—helpful, honest, and harmless necessarily means misaligning a system with the values of some subset of users who may reject that framing. Of course, at the other end of the spectrum, we might then say “ok, let’s not try to encode any values at all” but that is itself a value-laden choice that also guarantees a model will not be aligned with the preferences of another (likely larger) subset of users. Alternatively, we could allow users to turn the dial on certain settings themselves, but this is no free lunch either because it means that one person may be able to create a model aligned with their preferences that impinges on the preferences of someone else who encounters it. If it sounds like there’s no easy way out, that's because there isn’t. Any choice sees some people win and others lose. The challenge for projects seeking public input into AI is to understand these trade-offs and present possible compromises—not pretend there are none.
3. Compute governance poses calculated risk
What happened? A report from researchers at GovAI, OpenAI, and elsewhere assesses the various levers that compute offers for AI governance. It identifies four core properties of compute: ‘detectability’ (the capacity to detect the resources used to develop AI models); ‘excludability’ (the physical aspect of hardware that can prevent end-users from accessing AI chips); ‘quantifiability’ (a quality that enables easy measurement and verification of computational power); and ‘supply chain concentration’ (the idea that there are a limited number of powerful actors in the AI supply chain).
What's interesting? These properties, the authors suggest, mean that we essentially have three major mechanisms at our disposal should we want to use compute as a lever for affecting the governance of AI. We can track and assess AI development and use, we can allocate compute to exercise control over the creation of AI systems, and we can ensure that AI complies with regulations and standards by using compute as an enforcement mechanism. The second point, allocation, is where various national policies to build compute capacity come into play. See a great December report from TBI about global compute capacity if you’re interested in that sort of thing.
What else? What I like about the governance report is that the authors emphasise the risks and limitations of these mechanisms. They highlight threats to personal privacy, potential leaks of sensitive information, and the risk of centralising power, which—in the worst case scenario—may lead to the monopolisation of AI by an oppressive or illegitimate government. To counteract these downsides, the group suggests guardrails for compute governance like excluding small-scale AI and non-AI compute, employing compute-based controls only when necessary, and periodically reviewing control mechanisms. These are good ideas, though the problem is that ultimately what we are talking about in practice is building an extremely powerful surveillance apparatus and hoping that we can implement guardrails to prevent its abuse. While the report suggests omitting consumer-grade applications from governance regimes, it also accepts that individual GPUs (commonly used for gaming) may in the future be bundled together to train models. “In such situations,” they write, “governments may need to rely more on tools beyond compute governance to meet their goals.”
Best of the rest
Friday 16 February
Join OpenAI Forum to discuss, learn, and shape AI (OpenAI)
California takes on the biggest AI risk (Vox)
University of Michigan Is Selling Student Data to AI Companies (Gizmodo)
OpenAI's Sam Altman Seeks US Blessing to Raise Billions for AI Chips (Bloomberg)
The real quandary of AI isn't what people think (FT)
Thursday 15 February
How to design an AI ethics board (AI and Ethics)
Governing AI: A blueprint for the UK (Microsoft)
The OpenAI business model (FT)
News media versus AI: What if we win? (Press Gazette)
AI firms surge after chip giant Nvidia discloses stake (Reuters)
Offenders confused about ethics of AI child sex abuse (BBC)
Wednesday 14 February
Software in the natural world: A computational approach to emergence in complex multi-level systems (arXiv)
Little-known startup takes the AI weather prediction crown (Semafor)
Using Counterfactual Tasks to Evaluate the Generality of Analogical Reasoning in Large Language Models (arXiv)
Disrupting malicious uses of AI by state-affiliated threat actor (OpenAI)
AI won't take our jobs and it might even save the middle class (The Register)
Two OpenAI book lawsuits partially dismissed by California court (The Guardian)
Tuesday 13 February
Frontier AI ethics (Aeon)
Silicon Valley Has a Harvard Problem (TIME)
Memory and new controls for ChatGPT (OpenAI)
Sam Altman Wants $7 Trillion (Astral Codex Ten >> Substack)
OpenAI’s Altman Says UAE Could Be an AI ‘Sandbox’ For the World (Bloomberg)
Nvidia CEO Says Tech Advances Will Keep AI Cost in Check (Bloomberg)
Monday 12 February
Technical AI Governance ( XYZ >> Missed this last week!)
Tech companies axe 34,000 jobs since start of year in pivot to AI (FT)
The line between risk and progress (AI Policy Perspectives >> Substack)
AI at Work (The Verge)
"AI native" Gen Zers are comfortable on the cutting edge (Axios)
UK’s AI Safety Institute ‘needs to set standards rather than do testing’ (The Guardian)
Job picks
These are some of the interesting non-technical AI roles that I’ve seen advertised in the last week. As a reminder, it only includes new roles that have been posted since the last TWIE — though many of the jobs from the previous week’s email are still open. If you have an AI job that you think I should advertise in this section in the future, just let me know and I’d be happy to include it!
Director, Cybersecurity and Emerging Tech Policy, Microsoft, Washington, DC
Policy Outreach Lead, Anthropic, San Francisco
Gen AI Security Strategist, Global Team, Slalom, San Francisco
Head of AI Regulation Strategy and Implementation, UK Gov, London
Head of Programs, Events and Communications, FAR AI, San Francisco
One for the road
