Self-Improving AI Models Look Genuinely Scary

More than a hundred protestors marched between the San Francisco headquarters of Anthropic PBC and OpenAI earlier this month, calling for a pause on the race to improve artificial intelligence. Take one look at their paint-scrawled signs — “Pls do not kill me” and “QuitGPT” — and some might think them crackpots with a surface-level understanding of the technology. But their fears point to one of the hottest new trends in the field: so-called recursive self improvement, or AI that can develop its