Generative AI chatbots did “shoddy job” summarizing the news finds BBC study
Four major generative AI chatbots did a shoddy job of summarizing the news, a study by the BBC found.
The BBC fed content from its website to the services and then asked them questions about the news. “More than half of the answers contained “significant inaccuracies” and distortions, the BBC said.
The four major generative AI chatbots are OpenAI’s ChatGPT, Microsoft’s Copilot, Google’s Gemini and Perplexity AI.
According to Deadline, examples of errors included “Gemini incorrectly saying the UK’s National Health Service did not recommend vaping as an aid to quit smoking and ChatGPT and Copilot both wrongly indicating that politicians Rishi Sunak and Nicola Sturgeon were still in office. Perplexity also misquoted a BBC News report about the Middle East, saying Iran showed “restraint” and calling Israel’s actions “aggressive” – adjectives that did not appear in the initial news report.
“Journalists and news organizations told the tech giant about a number of gaffes. One alert generated by Apple from BBC News falsely told readers that Luigi Mangione, the man accused of fatally shooting the CEO of United Healthcare, had shot himself. (He was arrested and is in jail awaiting trial.) Another falsely told some BBC Sport app users that tennis star Rafael Nadal had come out as gay.”
More here.
