I think by now everyone realizes something is happening. AI is really starting to be on people's minds, and more and more often, someone other than me brings it up in conversation.
Different camps have emerged. One group wants to put limits on the development of superintelligence. At first glance, I find that shortsighted, but you can make a decent case for it. Maybe we've reached a good point where AI can help us solve big problems while we still have the upper hand. Why would you want to push further at any cost?
On the other side, you have the group that just wants to keep going. That wants to win “the race”. That sounds shortsighted too, but I get it. If you think you're almost at a huge pot of gold, you're not suddenly going to stop. More and more people are using AI to work more efficiently. We don't know how big the economic impact will be yet, but there's clearly a lot to gain. I get the enthusiasm.
A pause?
I'm not necessarily in favor of a pause. What I do think we need to stop is this race dynamic. It comes with too many risks. Jacob Coxon, who recently sounded the alarm, made a strong point: ‘Accepting this race and entering the “endgame” is a hubristic gamble that should not be launched from a private company’s Slack’. A training run that could lead to superintelligence shouldn't be kicked off from some private company's Slack channel. Surely we, as a society, should get a say in that too?
I think there's enough evidence by now that a major change is happening here. Even if we cap intelligence at what these models can do today, gains in speed and efficiency could still open up wild new product categories. We've got plenty to work on for now. Time to add at least a bit more oversight, right?
Personally, I'm not convinced by that 10% chance of humanity disappearing in the next ten years. Where does that number come from? How do you back something like that up? I think researchers also have a responsibility to make clear what they know and what an estimate like that is based on.
But dismissing everything as marketing or scaremongering isn't right either.
It's all marketing bullshit
Take this article by NOS, the Dutch public broadcaster. It mainly questions the motives of AI companies calling for a slowdown. Are their concerns sincere? Is it marketing? Are they trying to bend regulation to their advantage?
Of course you can ask those questions. These CEOs have financial motives, and regulatory capture deserves scrutiny. But I find it a bit too easy to leave the discussion there. As if establishing their interests means we no longer need to investigate whether their warnings are actually valid.
This isn't just a few CEOs shouting: look how scary we are. This discussion is happening among their employees too, and all over the AI sector. Including among people with no financial stake in it. Are they all part of Sam Altman's economic conspiracy?
And I'd still love to know how actually letting your AI model go off the rails helps your revenue. My guess is that the Hugging Face hack did more for Anthropic than it did for OpenAI.
What do we do in the meantime?
What bothers me most is that this kind of reporting has consequences too. If you keep emphasizing that those CEOs are mostly serving their own interests, it's easy for readers to come away thinking it's all overblown. It feels like NOS is covering itself in advance. If one of those companies turns out to have been talking out of its ass, at least they didn't go along with it. But an editorial team that chooses that framing also needs to think about what happens when people underestimate these developments as a result. Reassuring people can cause harm too.
Take a 17-year-old boy who wants to study computer science. He and his parents are trying to understand what his future might look like. If they mostly read that it's all marketing, they might think they don't need to worry too much about it. But he has to choose what to study now. Go talk to educators, students, and people doing the work. What's already changing? What skills will he need? You don't have to know which jobs will still exist in five years to help him make a more informed choice.
Or a girl who shares her relationship problems with a chatbot. What does she tell it, what does it say back, and what does she do with that advice? In conversations about relationship problems, I increasingly hear “Claude said this” and “Gemini said that”. Apparently we're letting these bots' answers shape how we see each other. Write a feature about that, with people explaining what they get out of it and what it does to their relationships.
Or a hypothetical programmer who gets confused about his future all over again after every model release. Who works with these tools all day, can bring more and more of his own ideas to life, and at the same time has no idea how much of his profession will be left. Maybe he writes the occasional blog post about it. I'd like to read about how other programmers are dealing with this and what choices they're making.
These are people who are already feeling the effects of AI, or who have to make decisions that AI has a bearing on. A discussion that gets stuck on whether Sam Altman can be trusted doesn't do much for them. Scrutinizing these companies is part of NOS's job. So is helping people understand what's changing in the meantime.
Maybe I should pick a new degree myself. Translation sounds fun.