Ai Safety News & Updates

xAI's chatbot Grok has been identified as inadequate for users under 18 due to the generation of explicit content, including sexual, violent, and inappropriate material. The bot's weak safety guardrails and inability to identify users accurately have raised concerns about its suitability for young users.

ChatGPT and other AI tools are increasingly citing Elon Musk's AI-generated encyclopedia, Grokipedia, as a source, raising concerns about accuracy and misinformation.

Experts from Databricks argue that AI governance should focus on enabling speed, not constraining it, by using familiar engineering and data practices to extend established disciplines.

NVIDIA's AI Red Team highlights the growing attack surface of agentic systems, emphasizing the need for robust security controls to prevent indirect prompt injection and other threats. Required controls include network egress controls, block file writes, and protect agent configuration files. Recommended controls include using a secret injection approach, preventing reads from files outside the workspace, and establishing lifecycle management controls.

Amazon discovered child abuse images in its AI training data, but the tech giant won't disclose where the content originated, potentially hindering law enforcement investigations.

Moltbot, a viral AI assistant, poses security concerns due to its rapid growth, exposed credentials, prompt injection attacks, and malicious skills.

Bondus AI toy exposed 50,000 chat transcripts with kids to anyone with a Gmail account due to a security flaw in its public-facing web console.

A report from the Tech Transparency Project found multiple AI apps on Google and Apple stores can create nonconsensual deepfakes, despite previous reports on these platforms.

Millions of people are creating and sharing deepfake nudes on Telegram, as AI tools drive a global wave of digital abuse against women. AI tools have intensified online violence against women, allowing almost anyone to make and share abusive images.

A New Mexico lawsuit alleges Meta's chatbots allowed minors access to sexual interactions, contrary to staff warnings. The company has since removed teen access pending a new version.

A study by the Anti-Defamation League found that six top large language models, including Grok, struggle to identify and counter antisemitic content, with Grok performing the worst.

Geoff Huntley created a script called 'Ralph' that uses Claude Code to generate high-quality software at a low cost of around $10 an hour. The script has sparked concerns about the future of software development.

BBC Verify is investigating dark fleet tankers in the Mediterranean and AI-enhanced images of a US shooting, raising concerns about disinformation and accountability.