Will Smith consuming spaghetti and different bizarre AI references that took off in 2024

When an organization releases a brand new AI video generator, it is not lengthy earlier than somebody makes use of it to make a video of actor Will Smith consuming spaghetti.

It is turn out to be each a meme and a reference: seeing if a brand new video generator can realistically depict Smith slurping a bowl of noodles. Smith himself parody the development in an Instagram put up in February.

Google Veo 2 did it.

We lastly eat spaghetti. pic.twitter.com/AZO81w8JC0

-Jerrod Lew (@jerrod_lew) December 17, 2024

Will Smith and pasta are only one instance weird “unofficial” landmarks to take the AI ​​neighborhood by storm in 2024. A 16-year-old developer has created an app that offers AI management of Minecraft and checks its skill to design constructions. Elsewhere, a British programmer has created a platform the place AI performs video games like Pictionary and Join 4 in opposition to one another.

It is not as if there aren’t extra educational checks of AI efficiency. So why did the bizarre ones explode?

On the one hand, most industry-standard AI benchmarks do not imply a lot to the common particular person. Firms typically cite their AI’s skill to reply Arithmetic Olympiad examination questions or discover believable options to doctoral-level issues. Nonetheless, most individuals – together with yours actually – use chatbots for issues like respond to emails and basic searches.

Participatory sectoral measures are usually not essentially higher or extra informative.

Take, for instance, Chatbot Arenaa public benchmark that many AI lovers and builders observe obsessively. Chatbot Area permits anybody on the net to guage AI efficiency on explicit duties, like creating an internet utility or producing a picture. However reviewers are typically unrepresentative – most come from AI and tech {industry} circles – and vote based mostly on hard-to-pin down private preferences.

Ethan Mollick, a professor of administration at Wharton, not too long ago identified in a job on X, one other downside with many AI {industry} benchmarks: They do not examine a system’s efficiency to that of a median particular person.

“The truth that there aren’t 30 totally different references from totally different organizations in drugs, regulation, high quality of recommendation, and so forth. is an actual disgrace, as a result of individuals use programs for these items it doesn’t matter what. so be it,” Mollick wrote.

Bizarre AI checks like Join 4, Minecraft, and Will Smith consuming spaghetti most definitely are not empirical – and even all of this generalizable. Simply because an AI passes the Will Smith check does not imply it should generate, say, a hamburger.

One knowledgeable I spoke to about AI benchmarks steered that the AI ​​neighborhood give attention to the downstream impacts of AI somewhat than its capabilities in slim areas. It is cheap. However I’ve a sense the unusual landmarks aren’t going away anytime quickly. Not solely are they entertaining, however who does not love watching AI construct Minecraft castles? – however they’re simple to grasp. And like my colleague Max Zeff recently wrotethe {industry} continues to battle to distill expertise as advanced as AI into digestible advertising and marketing.

The one query that involves my thoughts is: what new benchmarks will go viral in 2025?