AI was mentioned a few comments back. I saw this today in my news feed, seems relevant to the discussion if only tangentially:
"The web's collection of content, the material that inspires advanced models to generate contrived images or churn out convincing LinkedIn posts, is itself a finite resource. Even the vastness of the internet ends somewhere.
That's triggered a mad dash among AI companies to seek more content: pilfer copyrighted works, transmogrify videos into text, or even use AI-generated material as training data for AI systems.
But relying on synthetic data, as researchers have shown, degrades the quality and reliability of AI models, highlighting a major limitation in the promise of advanced AI.
Researchers at Rice University likened the danger of training generative models on synthetic material to "feeding cattle with the remains (including brains) of other cattle," crafting an AI training analogy to mad cow disease.
The explosion in AI tools has already littered the web with synthetic content, which continues to make up more and more of the internet. You've probably already noticed it gaming search engine results — authorless, unspecific, and, in the end, useless articles that get your click and brief attention as you search for trustworthy, human information.
This, of course, means that existing AI systems have already ingested their own results. And their results' results.
"It really is about brains corrupting future brains," said Richard Baraniuk, professor of electrical and computer engineering at Rice University, who co-authored the paper.
The limits of human-made content are just the latest example of the AI story confronting impassable boundaries. There's an array to choose from.
AI models “are just insatiable in terms of their thirst” for electricity, Rene Haas, the CEO of the chip design company Arm (ARM), said earlier this week.
"By the end of the decade, AI data centers could consume as much as 20% to 25% of US power requirements," Haas told the Wall Street Journal. “That’s hardly very sustainable.”
And these words are coming from a CEO, not a hater.
His remarks echo a January report from the International Energy Agency that said a query to ChatGPT requires almost 10 times as much electricity as the average Google search. Measuring from 2023, power demand by the AI industry is expected to grow at least 10 times by 2026, the agency said.
Other drags on the AI dream are closer to home.
Tech companies are scrambling to reduce their dependence on outside suppliers of AI chips, pouring billions into hardware and infrastructure. Google (GOOG, GOOGL) and Meta (META) unveiled new homegrown chips this week, flashing their costly commitments.
The investments are tickets to prosperity in the AI-led future. But the spending — like the warnings over data and resources — will bring them closer to having to prove it."