AI 模型出现崩溃迹象
https://www.solidot.org/story?sid=81407一位一直用 AI 进行搜索的使用者认为，AI 模型出现崩溃迹象。理由是过去几个月，所有主流 AI 搜索机器人的查询结果都变差了。作者认为是因为 Garbage In/Garbage Out(GIGO)导致的。在 AI 领域这被称为 AI 模型崩溃。基于自身输出进行训练的 AI 系统会逐渐丧失准确性、多样性和可靠性，错误在模型迭代中累积，导致数据分布扭曲和性能出现"不可逆的缺陷"。结果就是模型被自身对现实的投射而中毒。
www.theregister.com/2025/05/27/opinion_column_ai_model_collapse/ https://www.theregister.com/2025/05/27/opinion_column_ai_model_collapse/ AI + ML 54 Some signs of AI model collapse begin to reveal themselves 54 Prediction: General-purpose AI could start getting worse Steven J. Vaughan-Nichols Tue 27 May 2025 // 10:31 UTC Opinion I use AI a lot, but not to write stories. I use AI for search. When it comes to search, AI, especially Perplexity, is simply better than Google. Ordinary search has gone to the dogs. Maybe as Google goes gaga for AI, its search engine will get better again, but I doubt it. In just the last few months, I've noticed that AI-enabled search, too, has been getting crappier. Guide for the perplexed – Google is no longer the best search engine READ MORE In particular, I'm finding that when I search for hard data such as market-share statistics or other business numbers, the results often come from bad sources. Instead of stats from 10-Ks, the US Securities and Exchange Commission's (SEC) mandated annual business financial reports for public companies, I get numbers from sites purporting to be summaries of business reports. These bear some resemblance to reality, but they're never quite right. If I specify I want only 10-K results, it works. If I just ask for financial results, the answers get… interesting, This isn't just Perplexity. I've done the exact same searches on all the major AI search bots, and they all give me "questionable" results. Welcome to Garbage In/Garbage Out (GIGO). Formally, in AI circles, this is known as AI model collapse. In an AI model collapse, AI systems, which are trained on their own outputs, gradually lose accuracy, diversity, and reliability. This occurs because errors compound across successive model generations, leading to distorted data distributions and "irreversible defects" in performance. The final result? A Nature 2024 paper stated, "The model becomes poisoned with its own projection of reality." Model collapse is the result of three different factors. The first is error accumulation, in which each model generation inherits and amplifies flaws from previous versions, causing outputs to drift from original data patterns. Next, there is the loss of tail data: In this, rare events are erased from training data, and eventually, entire concepts are blurred. Finally, feedback loops reinforce narrow patterns, creating repetitive text or biased recommendations. I like how the AI company Aquant puts it: "In simpler terms, when AI is trained on its own outputs, the results can drift further away from reality." I'm not the only one seeing AI results starting to go downhill. In a recent Bloomberg Research study of Retrieval-Augmented Generation (RAG), the financial media giant found that 11 leading LLMs, including GPT-4o, Claude-3.5-Sonnet, and Llama-3-8 B, using over 5,000 harmful prompts would produce bad results. RAG, for those of you who don't know, enables large language models (LLMs) to pull in information from external knowledge stores, such as databases, documents, and live in-house data stores, rather than relying just on the LLMs' pre-trained knowledge. You'd think RAG would produce better results, wouldn't you? And it does. For example, it tends to reduce AI hallucinations. But, simultaneously, it increases the chance that RAG-enabled LLMs will leak private client data, create misleading market analyses, and produce biased investment advice. As Amanda Stent, Bloomberg's head of AI strategy & research in the office of the CTO, explained: "This counterintuitive finding has far-reaching implications given how ubiquitously RAG is used in gen AI applications such as customer support agents and question-answering systems. The average internet user interacts with RAG-based systems daily. AI practitioners need to be thoughtful about how to use RAG responsibly." That sounds good, but a "responsible AI user" is an oxymoron. For all the crap about how AI will encourage us to spend more time doing better work, the truth is AI users write fake papers including bullshit results. This ranges from your kid's high school report to fake scientific research documents to the infamous Chicago Sun-Times best of summer feature, which included forthcoming novels that don't exist. America's cyber defenses are being dismantled from the inside It's fun making Studio Ghibli-style images with ChatGPT – but intellectual property is no laughing matter Why Google's Chrome monopoly won't crack anytime soon AI running out of juice despite Microsoft's hard squeezing What all this does is accelerate the day when AI becomes worthless. For example, when I asked ChatGPT, "What's the plot of Min Jin Lee's forthcoming novel 'Nightshade Market?'" one of the fake novels, ChatGPT confidently replied, "There is no publicly available information regarding the plot of Min Jin Lee's forthcoming novel, Nightshade Market. While the novel has been announced, details about its storyline have not been disclosed." Once more, and with feeling, GIGO. Some researchers argue that collapse can be mitigated by mixing synthetic data with fresh human-generated content. What a cute idea. Where is that human-generated content going to come from? Given a choice between good content that requires real work and study to produce and AI slop, I know what most people will do. It's not just some kid wanting a B on their book report of John Steinbeck's The Pearl; it's businesses eager, they claim, to gain operational efficiency, but really wanting to fire employees to increase profits. Quality? Please. Get real. We're going to invest more and more in AI, right up to the point that model collapse hits hard and AI answers are so bad even a brain-dead CEO can't ignore it. How long will it take? I think it's already happening, but so far, I seem to be the only one calling it. Still, if we believe OpenAI's leader and cheerleader, Sam Altman, who tweeted in February 2024 that "OpenAI now generates about 100 billion words per day," and we presume many of those words end up online, it won't take long. ® More about AI Google Large Language Model More like these × More about AI Google Large Language Model Microsoft OpenAI Narrower topics Active Directory AIOps Amazon Bedrock Android Anthropic App stores Azure Bing BSoD ChatGPT Chrome Chromium Copilot DeepSeek Excel Exchange Server Gemini Google AI Google Cloud Platform Google I/O Google Nest GPT-3 GPT-4 G Suite HoloLens Internet Explorer Kubernetes LinkedIn Machine Learning MCubed Microsoft 365 Microsoft Build Microsoft Edge Microsoft Fabric Microsoft Ignite Microsoft Office Microsoft Surface Microsoft Teams .NET Neural Networks NLP Office 365 OS/2 Outlook Patch Tuesday Pixel Pluton Privacy Sandbox SharePoint Skype SQL Server Star Wars Tavis Ormandy Tensor Processing Unit TOPS Visual Studio Visual Studio Code Windows Windows 10 Windows 11 Windows 7 Windows 8 Windows Server Windows Server 2003 Windows Server 2008 Windows Server 2012 Windows Server 2013 Windows Server 2016 Windows Subsystem for Linux Windows XP Xbox Xbox 360 Broader topics Alphabet Bill Gates Search Engine Self-driving Car More about 54 COMMENTS More about AI Google Large Language Model More like these × More about AI Google Large Language Model Microsoft OpenAI Narrower topics Active Directory AIOps Amazon Bedrock Android Anthropic App stores Azure Bing BSoD ChatGPT Chrome Chromium Copilot DeepSeek Excel Exchange Server Gemini Google AI Google Cloud Platform Google I/O Google Nest GPT-3 GPT-4 G Suite HoloLens Internet Explorer Kubernetes LinkedIn Machine Learning MCubed Microsoft 365 Microsoft Build Microsoft Edge Microsoft Fabric Microsoft Ignite Microsoft Office Microsoft Surface Microsoft Teams .NET Neural Networks NLP Office 365 OS/2 Outlook Patch Tuesday Pixel Pluton Privacy Sandbox SharePoint Skype SQL Server Star Wars Tavis Ormandy Tensor Processing Unit TOPS Visual Studio Visual Studio Code Windows Windows 10 Windows 11 Windows 7 Windows 8 Windows Server Windows Server 2003 Windows Server 2008 Windows Server 2012 Windows Server 2013 Windows Server 2016 Windows Subsystem for Linux Windows XP Xbox Xbox 360 Broader topics Alphabet Bill Gates Search Engine Self-driving Car TIP US OFF Send us news
-- gReader Pro
