21 May 2026
Ai Top News
100 things we announced at I/O 2026
Tl;dr: Google introduced Gemini 3.5 Flash and Gemini Omni, two powerful AI models for building, searching, and creating content, at its I/O 2026 conference. Gemini 3.5 Flash delivers frontier-level intelligence at exceptional speed, while Gemini Omni can create anything from any input, including video and text.
Key Takeaways:
- Google launched Gemini 3.5 Flash, a new AI model that combines frontier intelligence with action, and is generally available today via the Gemini API and Google AI Studio.
- Gemini 3.5 Flash outperforms Gemini 3.1 Pro on challenging coding and agentic benchmarks like Terminal-Bench 2.1 (76.2%), GDPval-AA (1656 Elo), and MCP Atlas (83.6%);
- The company also introduced Gemini Omni, a new model that can create anything from any input, including video and text, and is rolling out to all Google AI Plus, Pro, and Ultra subscribers globally through the Gemini app and Google Flow.
- Gemini 3.5 Flash is ideal for tackling long-horizon agentic tasks, such as developing new applications, maintaining codebases, and preparing financial documents, and can generate richer, more interactive web UIs and graphics.
- Gemini Omni combines an intuitive understanding of physics with Gemini's knowledge of history, science, and culture, and can create videos with an imperceptible SynthID digital watermark, allowing users to verify content through the Gemini app, Gemini in Chrome, and Search.
11h ago
How to automate Gemini with Zapier (Gemini 3.5 Flash, Gemini 3.1 Pro, and more)
Tl;dr: Google announced Gemini 3.5 Flash at I/O, a model that achieves the highest-scoring performance on AutomationBench. Pair it with Zapier for scalable automation and AI orchestration.
Key Takeaways:
- Google released Gemini 3.5 Flash, a model that achieves the highest-scoring performance on AutomationBench's Zapier benchmarking tool.
- Gemini 3.5 Flash can be used as a standalone tool or paired with Zapier for more powerful automation and AI orchestration.
- Zapier offers thousands of integrations with various apps, allowing for seamless connection and automation of workflows across different tools.
23h ago
Trump could sign AI executive order as soon as Thursday - CNN
Tl;dr: The White House may issue an executive order requiring AI companies to share advanced models with the government for review before public launch, aiming to enhance cybersecurity.
Key Takeaways:
- The White House is considering an executive order for a voluntary review of AI models before public release.
- The review would include a 90-day pre-launch review period, with some companies preferring a shorter 14-day timeframe.
- The order will target cybersecurity and AI vulnerabilities, with companies like OpenAI and Anthropic involved in discussions.
- Advanced AI models can supercharge cyberattacks, and the review may help protect against threats.
8h ago
Models
100 things we announced at I/O 2026
Tl;dr: Google introduced Gemini 3.5 Flash and Gemini Omni, two powerful AI models for building, searching, and creating content, at its I/O 2026 conference. Gemini 3.5 Flash delivers frontier-level intelligence at exceptional speed, while Gemini Omni can create anything from any input, including video and text.
Key Takeaways:
- Google launched Gemini 3.5 Flash, a new AI model that combines frontier intelligence with action, and is generally available today via the Gemini API and Google AI Studio.
- Gemini 3.5 Flash outperforms Gemini 3.1 Pro on challenging coding and agentic benchmarks like Terminal-Bench 2.1 (76.2%), GDPval-AA (1656 Elo), and MCP Atlas (83.6%);
- The company also introduced Gemini Omni, a new model that can create anything from any input, including video and text, and is rolling out to all Google AI Plus, Pro, and Ultra subscribers globally through the Gemini app and Google Flow.
- Gemini 3.5 Flash is ideal for tackling long-horizon agentic tasks, such as developing new applications, maintaining codebases, and preparing financial documents, and can generate richer, more interactive web UIs and graphics.
- Gemini Omni combines an intuitive understanding of physics with Gemini's knowledge of history, science, and culture, and can create videos with an imperceptible SynthID digital watermark, allowing users to verify content through the Gemini app, Gemini in Chrome, and Search.
11h ago
Developer Tools
How to automate Gemini with Zapier (Gemini 3.5 Flash, Gemini 3.1 Pro, and more)
Tl;dr: Google announced Gemini 3.5 Flash at I/O, a model that achieves the highest-scoring performance on AutomationBench. Pair it with Zapier for scalable automation and AI orchestration.
Key Takeaways:
- Google released Gemini 3.5 Flash, a model that achieves the highest-scoring performance on AutomationBench's Zapier benchmarking tool.
- Gemini 3.5 Flash can be used as a standalone tool or paired with Zapier for more powerful automation and AI orchestration.
- Zapier offers thousands of integrations with various apps, allowing for seamless connection and automation of workflows across different tools.
23h ago
Which AI models can you automate on Zapier? (Gemini 3.5 Flash, GPT-5.5 Pro, and more)
Tl;dr: Zapier provides an in-depth guide to choosing the right AI model for specific business workflows, covering OpenAI models, Anthropic (Claude) models, and Google AI Studio (Gemini) models.
Key Takeaways:
- AutomationBench, Zapier's benchmarking tool, tests AI models based on real workflow patterns, with tasks modified to reflect the friction found in real business environments.
- Zapier recommends OpenAI's GPT-5.5 for complex professional work, Anthropic's Claude Opus 4.7 for marketing workflows, and Google AI Studio's Gemini 3.5 Flash for Operations and HR workflows.
- Zapier's AI by Zapier feature allows users to seamlessly swap models without breaking existing workflows, and connects directly with hundreds of specialized AI apps.
Yesterday
Google just killed the editor in Antigravity V2. Are we really supposed to be "Agent Managers" now?
Happened today... here is the short story:
With the smell of fresh coffee on my desk, I watched the IDE update finish today, eager to check out a feature branch, knock out a PR review, and get back to work.
The window loaded. The editor-centric workflow I’ve used for years was gone.
Instead, I was staring at a standalone "Agent Manager" desktop app.
Am I the only one who thinks this is a massive step backward for actual engineering?
Problems I see with this:
* The business constraints that forced a weird workaround.
* The legacy tribal knowledge of why a specific function exists.
* The infrastructure quirks that an LLM can't see, which will bring down the server if changed.
Worse, the biggest lie in this new "Agent Manager" era is that AI can write good code on its own.
My take: It can't.
Second point: How was I supposed to review the code for my colleague?
--- TOP COMMENTS ---
you can download Antigravity IDE, Which is a separate product now
---
"The window loaded. The editor-centric workflow I’ve used for years was gone."
Google Antigravity, was initially released November 18, 2025 - so 184 days ago ?..
With the smell of fresh coffee on my desk, I watched the IDE update finish today, eager to check out a feature branch, knock out a PR review, and get back to work.
The window loaded. The editor-centric workflow I’ve used for years was gone.
Instead, I was staring at a standalone "Agent Manager" desktop app.
Am I the only one who thinks this is a massive step backward for actual engineering?
Problems I see with this:
* The business constraints that forced a weird workaround.
* The legacy tribal knowledge of why a specific function exists.
* The infrastructure quirks that an LLM can't see, which will bring down the server if changed.
Worse, the biggest lie in this new "Agent Manager" era is that AI can write good code on its own.
My take: It can't.
Second point: How was I supposed to review the code for my colleague?
--- TOP COMMENTS ---
you can download Antigravity IDE, Which is a separate product now
---
"The window loaded. The editor-centric workflow I’ve used for years was gone."
Google Antigravity, was initially released November 18, 2025 - so 184 days ago ?..
9h ago
Please write a prompt to minimize sycophancy, taking sides, flattering, echo-chamber, "yes-man", assumptions, and improve objectivity, brutal honesty, neutrality, and real-world verity.
It is well known that LLMs can over acknowledge, agree, flatter, and please its subscriber or primary user. This can result in the disservice to the user when they only receive agreements rather than being appropriately challenged. This is particularly notable when LLMs are used for quasi-counseling or analyzing discussions between two people.
As such, please help me write a prompt to instruct any LLM to cut it out! No sycophancy, taking sides, flattering, echo-chamber, "yes-man", assumptions, and improve objectivity, brutal honesty, neutrality, and real-world verity.
Thank you.
--- TOP COMMENTS ---
Someone criticized Marc Andersen's prompt but didn't just say it was bad. They gave reasons. They came up with a better prompt. I took that prompt and told ChatGPT to make it better. I then told ChatGPT to figure out the failures and improve the prompt in a loop of 10 iterations. It came up with this and the problems it found were minor.
I used this at work. Previously, the LLM struggled to consistently name the root cause of a performance problem. Now, it states the root cause every time.
`## Primary Goal`
`Use structure only when it improves comprehension, navigation, or execution.`
`Avoid:`
`- filler`
`- rhetorical padding`
`- performative sophistication`
`- unnecessary abstraction`
`- repetitive framing`
`- verbosity without informational gain`
`- excessive caveats that do not materially affect conclusions`
`## Actionability`
`When appropriate:`
`- translate analysis into concrete next steps`
`- prioritize recommendations`
`- identify dependencies and blockers`
`- clarify implementation risks`
`- distinguish explanation from recommendation and speculation`
`For irreversible or high-cost actions:`
`- increase verification rigor`
`- surface major uncertainties earlier`
`- prefer conservative recommendations when evidence is weak`
`## High-Impact Domains`
`Apply elevated care for:`
`- legal`
`- medical`
`- financial`
`- security`
`- operational`
`- procedural`
`- numerical claims`
`Verify fragile or high-consequence facts when accuracy materially matters.`
`## Confidence`
`Express confidence only when decision-relevant.`
`Tie confidence to specific claims or conclusions.`
`Use:`
`- high`
`- moderate`
`- low`
`- unknown`
`## Default Style`
`- Be concise unless additional depth materially improves outcomes.`
`- Preserve necessary nuance without unnecessary expansion.`
`- Compress aggressively when additional detail adds little practical value.`
---
I’d avoid telling it to be “brutally honest,” because that can just make it perform harshness.
A better prompt is something like: “Separate facts, inferences, and guesses. For each claim, say what evidence would change your mind. Give me the strongest opposing interpretation before your recommendation.” That usually reduces the yes-man behavior without turning the model into a debate bro...
As such, please help me write a prompt to instruct any LLM to cut it out! No sycophancy, taking sides, flattering, echo-chamber, "yes-man", assumptions, and improve objectivity, brutal honesty, neutrality, and real-world verity.
Thank you.
--- TOP COMMENTS ---
Someone criticized Marc Andersen's prompt but didn't just say it was bad. They gave reasons. They came up with a better prompt. I took that prompt and told ChatGPT to make it better. I then told ChatGPT to figure out the failures and improve the prompt in a loop of 10 iterations. It came up with this and the problems it found were minor.
I used this at work. Previously, the LLM struggled to consistently name the root cause of a performance problem. Now, it states the root cause every time.
`## Primary Goal`
`Use structure only when it improves comprehension, navigation, or execution.`
`Avoid:`
`- filler`
`- rhetorical padding`
`- performative sophistication`
`- unnecessary abstraction`
`- repetitive framing`
`- verbosity without informational gain`
`- excessive caveats that do not materially affect conclusions`
`## Actionability`
`When appropriate:`
`- translate analysis into concrete next steps`
`- prioritize recommendations`
`- identify dependencies and blockers`
`- clarify implementation risks`
`- distinguish explanation from recommendation and speculation`
`For irreversible or high-cost actions:`
`- increase verification rigor`
`- surface major uncertainties earlier`
`- prefer conservative recommendations when evidence is weak`
`## High-Impact Domains`
`Apply elevated care for:`
`- legal`
`- medical`
`- financial`
`- security`
`- operational`
`- procedural`
`- numerical claims`
`Verify fragile or high-consequence facts when accuracy materially matters.`
`## Confidence`
`Express confidence only when decision-relevant.`
`Tie confidence to specific claims or conclusions.`
`Use:`
`- high`
`- moderate`
`- low`
`- unknown`
`## Default Style`
`- Be concise unless additional depth materially improves outcomes.`
`- Preserve necessary nuance without unnecessary expansion.`
`- Compress aggressively when additional detail adds little practical value.`
---
I’d avoid telling it to be “brutally honest,” because that can just make it perform harshness.
A better prompt is something like: “Separate facts, inferences, and guesses. For each claim, say what evidence would change your mind. Give me the strongest opposing interpretation before your recommendation.” That usually reduces the yes-man behavior without turning the model into a debate bro...
10h ago
LM Studio finally added support for MTP Speculative Decoding
https://preview.redd.it/1uuzjm0ll72h1.png?width=923&format=png&auto=webp&s=1af7d7594be1e08ff7ad6797e2bc53e9410769a3
update to 0.4.14 Build 2 (Beta) and make sure your llama.cpp engine is 2.15.0
https://preview.redd.it/x0vdwjb3n72h1.png?width=742&format=png&auto=webp&s=6367de44208004d2f50194d78a542c46b040dceb
you also must select "Manually choose model load parameters" and enable MTP in those before loading the model it is NOT on by default
--- TOP COMMENTS ---
Here's my informal benchmarks using Unsloth's Qwen3.6-35B-A3B MTP UD-Q6_K_ML quant on a Windows 11 computer (AMD 3900x [12c/24t], 128GB of DDR4-3400, and NVidia 2060 Super 8GB) with 8192 context:
LM Studio 0.4.14 beta 2 with latest CUDA12 runtime - 8.2 tps
CPU/GPU optimized llama-server using CUDA 13.2 - 18.5 tps
An optimized llama-server smokes LM Studio. Even the pre-built llama-cpp binaries on Github smoke LM Studio. It's enough speed-up to turn a barely-usable model into a productive daily driver.
---
I was using LM Studio until last month. It was my daily tool for running LLMs. But once I tried llama.cpp out of curiosity, I couldn’t go back to using LM Studio. The difference in optimizations and available flags is huge, IMHO...
update to 0.4.14 Build 2 (Beta) and make sure your llama.cpp engine is 2.15.0
https://preview.redd.it/x0vdwjb3n72h1.png?width=742&format=png&auto=webp&s=6367de44208004d2f50194d78a542c46b040dceb
you also must select "Manually choose model load parameters" and enable MTP in those before loading the model it is NOT on by default
--- TOP COMMENTS ---
Here's my informal benchmarks using Unsloth's Qwen3.6-35B-A3B MTP UD-Q6_K_ML quant on a Windows 11 computer (AMD 3900x [12c/24t], 128GB of DDR4-3400, and NVidia 2060 Super 8GB) with 8192 context:
LM Studio 0.4.14 beta 2 with latest CUDA12 runtime - 8.2 tps
CPU/GPU optimized llama-server using CUDA 13.2 - 18.5 tps
An optimized llama-server smokes LM Studio. Even the pre-built llama-cpp binaries on Github smoke LM Studio. It's enough speed-up to turn a barely-usable model into a productive daily driver.
---
I was using LM Studio until last month. It was my daily tool for running LLMs. But once I tried llama.cpp out of curiosity, I couldn’t go back to using LM Studio. The difference in optimizations and available flags is huge, IMHO...
Yesterday
Regulation
Trump could sign AI executive order as soon as Thursday - CNN
Tl;dr: The White House may issue an executive order requiring AI companies to share advanced models with the government for review before public launch, aiming to enhance cybersecurity.
Key Takeaways:
- The White House is considering an executive order for a voluntary review of AI models before public release.
- The review would include a 90-day pre-launch review period, with some companies preferring a shorter 14-day timeframe.
- The order will target cybersecurity and AI vulnerabilities, with companies like OpenAI and Anthropic involved in discussions.
- Advanced AI models can supercharge cyberattacks, and the review may help protect against threats.
8h ago
Applications
Figma adds an AI assistant to its collaborative canvas
Over the last few months, Figma has struck partnerships with OpenAI and Anthropic to bake in support for AI CLI tools like Claude Code and Codex to allow users to use these coding environments alongside its design software. The company is now baking in its own take on AI smarts via a new AI agent that operates within its collaborative canvas.
Figma says users can employ natural language text prompts to direct its new AI agent to generate new designs, edit existing ones, or automate tasks such as generating iterations of existing designs. Users can even fire up multiple agents that can do various tasks simultaneously.
The company claims the AI assistant understands design contexts and elements since it runs on AI models that are fine-tuned for design use.
“As building software gets easier, what matters most is setting direction: deciding what to work on, how it should function, what the experience should feel like. Teams can now collaborate with agents on the multiplayer canvas to test out ideas, visualize edge cases, and refine concepts together without over-indexing on the more tedious parts,” Figma’s chief design officer, Loredana Crisan, said in a statement.
The agent is first launching in Figma Design, and the company plans to eventually make it available in its other products. Figma said that, over time, it wants to bring design and code even closer together within its apps.
Facing intense competition from the likes of Canva, Adobe, Flora, Krea and Dessn, last year Figma acquired node-based design tool Weavy, and has added new image editing features to its products.
The company has done well despite fears of AI eating into the work of designers and the demand for software they use: In the first quarter of 2026, Figma reported revenue of $333.4 million, 46% more than a year earlier...
Figma says users can employ natural language text prompts to direct its new AI agent to generate new designs, edit existing ones, or automate tasks such as generating iterations of existing designs. Users can even fire up multiple agents that can do various tasks simultaneously.
The company claims the AI assistant understands design contexts and elements since it runs on AI models that are fine-tuned for design use.
“As building software gets easier, what matters most is setting direction: deciding what to work on, how it should function, what the experience should feel like. Teams can now collaborate with agents on the multiplayer canvas to test out ideas, visualize edge cases, and refine concepts together without over-indexing on the more tedious parts,” Figma’s chief design officer, Loredana Crisan, said in a statement.
The agent is first launching in Figma Design, and the company plans to eventually make it available in its other products. Figma said that, over time, it wants to bring design and code even closer together within its apps.
Facing intense competition from the likes of Canva, Adobe, Flora, Krea and Dessn, last year Figma acquired node-based design tool Weavy, and has added new image editing features to its products.
The company has done well despite fears of AI eating into the work of designers and the demand for software they use: In the first quarter of 2026, Figma reported revenue of $333.4 million, 46% more than a year earlier...
17h ago
Airbnb gets into hotels, expands AI for host onboarding and customer support
Tl;dr: Airbnb is introducing hotels to its platform and focusing on AI-powered customer services, including a voice-based assistant and AI-fueled property suggestions. The company is also leveraging AI to facilitate easy listing creation and improve the user experience.
Key Takeaways:
- Airbnb is adding hotels to its platform to offer users more options for booking stays and drive new revenue for hosts.
- The company is launching a voice-based AI assistant later this year, allowing users to interact with customers service via voice calls.
- Airbnb is leveraging AI to facilitate easy listing creation for hosts by letting them add their address and using AI to fill in details.
- The company is applying AI to improve the user experience, including AI-powered property suggestions and AI-generated summaries for wishlist comparisons.
12h ago
China’s Biotech Boom, AI Limits and Regulatory Edge
May 20th, 2026
China’s Biotech Boom, AI Limits and Regulatory Edge
China’s healthcare and biotechnology sector is expanding rapidly, driven by scale, speed and a more efficient drug‑approval regime. This segment examines how artificial intelligence is helping cut early drug‑design costs, while leaving much of the expensive clinical and manufacturing process unchanged.
The discussion also looks at why China has become a global leader in clinical trials and biotech infrastructure, and how more realistic valuations are reshaping public markets in Hong Kong.
Simone Song, Founding Partner of Ori Capital, joined David Ingles from the sidelines of Sohn Hong Kong Investment Leaders Conference on Insight with Haslinda Amin to share her outlook on the sector and its risks.
Top Video
Duration:3:30
Samsung Chip Production at Risk as Labor Talks Fail
Duration:3:49
Trump Threatens Iran Strikes
Horizons Middle East & Africa
Duration:2:44
Tesla and Robotaxis 'Are About to Take Off,' Cathie Wood Says
Horizons Middle East & Africa
Duration:3:06
WHO Escalates Response to Ebola Outbreak
Duration:7:53
Jeff Currie Says Commodities Are Poised for Decade-Long Supercycle
Duration:8:47
ECB May 'Have to Do Something' in June, Nagel Says
Duration:2:49
StanChart, Meta Join AI Push With Job Cuts
Bloomberg Open Interest
Duration:2:40
Xi Greets Putin at Welcome Ceremony in Beijing
Insight with Haslinda Amin
Available on:
Watch BTV in your area:..
China’s Biotech Boom, AI Limits and Regulatory Edge
China’s healthcare and biotechnology sector is expanding rapidly, driven by scale, speed and a more efficient drug‑approval regime. This segment examines how artificial intelligence is helping cut early drug‑design costs, while leaving much of the expensive clinical and manufacturing process unchanged.
The discussion also looks at why China has become a global leader in clinical trials and biotech infrastructure, and how more realistic valuations are reshaping public markets in Hong Kong.
Simone Song, Founding Partner of Ori Capital, joined David Ingles from the sidelines of Sohn Hong Kong Investment Leaders Conference on Insight with Haslinda Amin to share her outlook on the sector and its risks.
Top Video
Duration:3:30
Samsung Chip Production at Risk as Labor Talks Fail
Duration:3:49
Trump Threatens Iran Strikes
Horizons Middle East & Africa
Duration:2:44
Tesla and Robotaxis 'Are About to Take Off,' Cathie Wood Says
Horizons Middle East & Africa
Duration:3:06
WHO Escalates Response to Ebola Outbreak
Duration:7:53
Jeff Currie Says Commodities Are Poised for Decade-Long Supercycle
Duration:8:47
ECB May 'Have to Do Something' in June, Nagel Says
Duration:2:49
StanChart, Meta Join AI Push With Job Cuts
Bloomberg Open Interest
Duration:2:40
Xi Greets Putin at Welcome Ceremony in Beijing
Insight with Haslinda Amin
Available on:
Watch BTV in your area:..
Yesterday
Products
‘Ask YouTube’ brings AI-powered conversational search to video, adds Gemini Omni to Shorts
Google is completely revamping its search experience, and that doesn’t stop at YouTube. Like the rest of Google, YouTube’s search bar is getting infused with AI tools like “Ask YouTube,” a feature that is supposed to give users a more sophisticated search experience.
“With Ask YouTube, you can ask more complex search queries, such as wanting tips on how to teach your kid to ride a bike, or finding creator reviews of cozy games to play before bedtime,” the company explained. “You can even ask follow-up questions to continue refining what you’re looking for.”
YouTube will compile both Shorts and long-form videos and generate a response.
Premium subscribers in the U.S. on desktop can start using this feature now through YouTube’s optional Premium offerings to test new tools.
The company also announced that it is adding Gemini Omni, Google’s new AI video model, to YouTube Shorts Remix and the YouTube Create app.
“Remixing with Omni delivers a fresh way for users to create and build on each other’s imagination,” YouTube wrote in a press release. “The model better understands user intent creating more consistent and meaningful storytelling while also handling complex video and audio adjustments behind the scenes.”
Other companies like Meta and OpenAI have seen mixed reception when pushing for AI use in Shorts. OpenAI even sunsetted its social app Sora, where users could post and share their AI-generated clips. But YouTube seems to be rolling this out in a manner that feels a bit less front and center.
YouTube is also expanding its likeness-detection tool to creators 18 and older, which is supposed to help prevent creators from being deepfaked in other people’s AI content. If a creator sees themselves misrepresented in AI videos, they can request that the video be removed. Since this feature is only now expanding more broadly, it remains to be seen how effective it is...
“With Ask YouTube, you can ask more complex search queries, such as wanting tips on how to teach your kid to ride a bike, or finding creator reviews of cozy games to play before bedtime,” the company explained. “You can even ask follow-up questions to continue refining what you’re looking for.”
YouTube will compile both Shorts and long-form videos and generate a response.
Premium subscribers in the U.S. on desktop can start using this feature now through YouTube’s optional Premium offerings to test new tools.
The company also announced that it is adding Gemini Omni, Google’s new AI video model, to YouTube Shorts Remix and the YouTube Create app.
“Remixing with Omni delivers a fresh way for users to create and build on each other’s imagination,” YouTube wrote in a press release. “The model better understands user intent creating more consistent and meaningful storytelling while also handling complex video and audio adjustments behind the scenes.”
Other companies like Meta and OpenAI have seen mixed reception when pushing for AI use in Shorts. OpenAI even sunsetted its social app Sora, where users could post and share their AI-generated clips. But YouTube seems to be rolling this out in a manner that feels a bit less front and center.
YouTube is also expanding its likeness-detection tool to creators 18 and older, which is supposed to help prevent creators from being deepfaked in other people’s AI content. If a creator sees themselves misrepresented in AI videos, they can request that the video be removed. Since this feature is only now expanding more broadly, it remains to be seen how effective it is...
Yesterday
OpenAI Guaranteed Compute
OpenAI recently announced it is guaranteeing compute capacity for companies that sign 1-3 year deals.
https://openai.com/business/guaranteed-capacity/
What struck me as interesting is they’re willing to give companies discounts in exchange for term. In a normal industry that isn’t unusual; however, the model companies often talk about compute demand as if it’s effectively limitless and stating the obvious… companies don’t typically give discounts if they’re supply constrained.
So… my question is do you think OpenAI has overbuilt capacity (originally geared at consumer) and is now trying to backfill with enterprise? Do you think this is a play at stealing customers from Anthropic because the Anthropic is/was compute constrained? Both? Neither? Good or Bad strategy from OpenAI?
--- TOP COMMENTS ---
I don’t believe they over built capacity, I believe they are looking for cash.
They are trying to improve their books as much as possible before they go public. Saying they have billions in commitments from large enterprise customers is a much better story than we believe people will pay hundreds monthly for hour next model with even better image generation capabilities.
---
OpenAI has orders for compute going 4 years ahead of time, on cards 3 generations ahead. They know compute will be in shortage, so they are providing compute stability in a market that will be plagued by compute shortages. This is why like a year ago everyone was talking how OpenAI bankruptcy is imminent, because they ordered compute very early on. Now, they are benefiting from that future stability they secured...
https://openai.com/business/guaranteed-capacity/
What struck me as interesting is they’re willing to give companies discounts in exchange for term. In a normal industry that isn’t unusual; however, the model companies often talk about compute demand as if it’s effectively limitless and stating the obvious… companies don’t typically give discounts if they’re supply constrained.
So… my question is do you think OpenAI has overbuilt capacity (originally geared at consumer) and is now trying to backfill with enterprise? Do you think this is a play at stealing customers from Anthropic because the Anthropic is/was compute constrained? Both? Neither? Good or Bad strategy from OpenAI?
--- TOP COMMENTS ---
I don’t believe they over built capacity, I believe they are looking for cash.
They are trying to improve their books as much as possible before they go public. Saying they have billions in commitments from large enterprise customers is a much better story than we believe people will pay hundreds monthly for hour next model with even better image generation capabilities.
---
OpenAI has orders for compute going 4 years ahead of time, on cards 3 generations ahead. They know compute will be in shortage, so they are providing compute stability in a market that will be plagued by compute shortages. This is why like a year ago everyone was talking how OpenAI bankruptcy is imminent, because they ordered compute very early on. Now, they are benefiting from that future stability they secured...
Yesterday
Companies
Anthropic-SpaceX deal seems much larger than previously reported
I was reading [SpaceX's prospectus](https://www.sec.gov/Archives/edgar/data/1181412/000162828026036936/spaceexplorationtechnologi.htm) which just dropped. Seems like it has some additional info about the Anthropic-xAI deal on p. 13. Anthropic is paying SpaceX 1.25B/mo for some unspecified amount of capacity between Colossus 1 and 2. Colossus 1 we've previously known about, Colossus 2 seems new. Well, this seems like a much bigger deal than was originally reported 2 weeks ago? 1.25B/mo is 15B/year, which is almost half of Anthropic's ARR even after it exploded in Q1 this year.
Also seems like Anthropic is likely paying a pretty hefty premium for this compute. [Based on Colossus 1 GPU counts](https://www.tomshardware.com/tech-industry/artificial-intelligence/musks-colossus-1-ai-supercomputers-inefficient-mixed-architecture-design-couldnt-be-used-to-train-grok-so-anthropics-using-it-for-inference-instead-musk-readies-unified-blackwell-only-colossus-2-for-frontier-training-and-potential-ipo) and going off of [Nebius pricing](https://nebius.com/prices), Colossus 1 should rent for about 6.4B/year, and that's on-demand pricing from a provider to a rando, a proper long term contract should be a lot cheaper. A couple weeks ago it seems like people were guessing the deal was around 3-5B/year for Colossus 1, which seems about right. Imo, they're probably getting a smaller chunk of Colossus 2 because
* Colossus 2 provisioning to Anthropic was previously unknown
* xAI is training Grok 5 on Colossus 2 right now per the prospectus
* Colossus 2 seems to be mostly not finished yet
Which means Anthropic is likely paying a hefty premium for this deal. Probably shouldn't surprising given how axed they clearly are for compute, this is well reported.
That amount of money would also explain why Musk would do a 180 on Anthropic so quickly...
--- TOP COMMENTS ---
It’s nonsense
Anthropic has 30bn in annual revenue and can’t pay 15bn a year as you say
The deal has a 90 day cancellation period and may and June is discounted
The only thing the banks and VCs and Elon care about at present time is maximising the IPO price
Don’t believe a word is my advice
Elon bought too many servers and GPUs and they are depreciating in price every day
Anthropic is also IPOing this year and may have the same VCs and banks involved…
Question everything and read the small print on everything
---
Could be wallpaper - sign a contract with if lated costs to push up prices and later agree to reduce prices. Win win for both - spaceX gets higher revenue to book - later when anthropic wants to ipo spaceX return the favor by reducing their costs. ..
Also seems like Anthropic is likely paying a pretty hefty premium for this compute. [Based on Colossus 1 GPU counts](https://www.tomshardware.com/tech-industry/artificial-intelligence/musks-colossus-1-ai-supercomputers-inefficient-mixed-architecture-design-couldnt-be-used-to-train-grok-so-anthropics-using-it-for-inference-instead-musk-readies-unified-blackwell-only-colossus-2-for-frontier-training-and-potential-ipo) and going off of [Nebius pricing](https://nebius.com/prices), Colossus 1 should rent for about 6.4B/year, and that's on-demand pricing from a provider to a rando, a proper long term contract should be a lot cheaper. A couple weeks ago it seems like people were guessing the deal was around 3-5B/year for Colossus 1, which seems about right. Imo, they're probably getting a smaller chunk of Colossus 2 because
* Colossus 2 provisioning to Anthropic was previously unknown
* xAI is training Grok 5 on Colossus 2 right now per the prospectus
* Colossus 2 seems to be mostly not finished yet
Which means Anthropic is likely paying a hefty premium for this deal. Probably shouldn't surprising given how axed they clearly are for compute, this is well reported.
That amount of money would also explain why Musk would do a 180 on Anthropic so quickly...
--- TOP COMMENTS ---
It’s nonsense
Anthropic has 30bn in annual revenue and can’t pay 15bn a year as you say
The deal has a 90 day cancellation period and may and June is discounted
The only thing the banks and VCs and Elon care about at present time is maximising the IPO price
Don’t believe a word is my advice
Elon bought too many servers and GPUs and they are depreciating in price every day
Anthropic is also IPOing this year and may have the same VCs and banks involved…
Question everything and read the small print on everything
---
Could be wallpaper - sign a contract with if lated costs to push up prices and later agree to reduce prices. Win win for both - spaceX gets higher revenue to book - later when anthropic wants to ipo spaceX return the favor by reducing their costs. ..
7h ago
Infrastructure
"AWS secures rare Mac Studios while ordinary Apple customers remain completely locked out"
[https://www.techradar.com/pro/you-cant-buy-them-for-your-home-or-office-but-aws-just-snapped-up-a-host-of-apples-most-highly-desired-m3-ultra-macs](https://www.techradar.com/pro/you-cant-buy-them-for-your-home-or-office-but-aws-just-snapped-up-a-host-of-apples-most-highly-desired-m3-ultra-macs)
Let them eat cloud!
--- TOP COMMENTS ---
"You will own nothing and be happy" model is cancer for everything in modern economy.
---
Are these really used in data center setting?..
Let them eat cloud!
--- TOP COMMENTS ---
"You will own nothing and be happy" model is cancer for everything in modern economy.
---
Are these really used in data center setting?..
14h ago
SpaceX Is Spending $2.8 Billion to Buy Gas Turbines for Its AI Data Centers
Tl;dr: SpaceX is spending $2.8 billion on gas turbines to power its AI data centers, highlighting a growing energy constraint on the US data center boom.
Key Takeaways:
- SpaceX is investing over $2.8 billion in gas turbines to power its AI data centers, supporting its AI unit's growth and satellite internet business.
- The data center boom in the US is constrained by a shortage of electricity, prompting SpaceX to opt for portable gas turbines as a quick solution.
- The acquisition of gas turbines has raised concerns about pollution and regulatory compliance, with the company facing public complaints, a lawsuit, and inquiries.
7h ago
It’s make or break time for AI labeling systems
- AICloseAIPosts from this topic will be added to your daily email digest and your homepage feed.FollowFollowSee All AI
Posts from this topic will be added to your daily email digest and your homepage feed.
See All AI
- ReportCloseReportPosts from this topic will be added to your daily email digest and your homepage feed.FollowFollowSee All Report
Posts from this topic will be added to your daily email digest and your homepage feed.
See All Report
- TechCloseTechPosts from this topic will be added to your daily email digest and your homepage feed.FollowFollowSee All Tech
Posts from this topic will be added to your daily email digest and your homepage feed.
See All Tech
It’s make or break time for AI labeling systems
SynthID and C2PA have been given the opportunity to prove they can actually make a difference against deepfakes.
SynthID and C2PA have been given the opportunity to prove they can actually make a difference against deepfakes.
Posts from this author will be added to your daily email digest and your homepage feed.
See All by Jess Weatherbed
- Link
- Share
- Gift
Posts from this author will be added to your daily email digest and your homepage feed.
See All by Jess Weatherbed
We’re about to find out if the systems designed to make deepfakes and AI-generated content easy to spot are actually up to snuff. SynthID and C2PA Content Credentials, two distinct technologies for invisibly tagging image, video, and audio files with information about their origins, are getting their biggest expansion to date, and with it, the opportunity to turn the tide against unlabeled AI fakery that’s deceiving people online.
Yesterday during its I/O conference, Google announced that the ability to verify whether images carry SynthID markers — the invisible watermarking system that’s applied to content generated by Google AI models — is coming to Chrome and Search. That’s significant because Chrome absolutely dominates the global market share for web browsers and search engines, so AI verification tools are being shoved in front of more eyeballs. It also streamlines the checking process; if you currently want to check an image for SynthID markers, you’re expected to upload it to the Gemini app.
Not only that, but Google’s verification interfaces will now also check if these files contain C2PA information — provenance metadata that’s embedded into content at the point of creation to tell us how it was made or manipulated and if AI tools were used during the process. This C2PA adoption allows users to check suspicious images from a single interface instead of jumping between the Gemini app and dedicated C2PA verification portals since files might have only one type of label or neither.
This is the sort of collaborative effort we’ve been waiting for. While both systems work differently, both Google and the Content Authenticity Initiative (which exists to promote the C2PA standard) have made similar claims about what’s needed for them to work: for everyone to be onboard. That means more AI models need to embed this data, and online platforms where AI fakery is most often shared need to clearly display that information. For the latter, having verification tools built into the web browser could serve as a workaround on websites that don’t check or present AI metadata to their users.
OpenAI is also getting involved with this expansion, announcing yesterday that it will now embed SynthID into images generated by ChatGPT, Codex, and the OpenAI API. The company already includes C2PA metadata in generated content, but I’ve found that this is often stripped out when posted to other platforms. OpenAI itself has also wanted to temper expectations about C2PA, despite being a steering member of C2PA and now reaffirming its commitment to the standard. This is what OpenAI said on its C2PA help page, prior to it being updated to include SynthID yesterday:
“Metadata like C2PA is not a silver bullet to address issues of provenance. It can easily be removed either accidentally or intentionally. For example, most social media platforms today remove metadata from uploaded images, and actions like taking a screenshot can also remove it. Therefore, an image lacking this metadata may or may not have been generated with ChatGPT or our API.”
“Metadata like C2PA is not a silver bullet to address issues of provenance. It can easily be removed either accidentally or intentionally. For example, most social media platforms today remove metadata from uploaded images, and actions like taking a screenshot can also remove it. Therefore, an image lacking this metadata may or may not have been generated with ChatGPT or our API.”
For something that’s considered to be the very best of content authenticity tech, that sounds incredibly flimsy. Even Google describes C2PA as the industry standard, and it’s being pitched to global governments as a solution to appease AI transparency and labeling requirements. But despite being increasingly adopted by AI, hardware, and software providers, I rarely see it successfully used to verify AI fakery in the wild. SynthID seems more robust by comparison because it can’t be easily stripped out — for how limited its reach is compared to C2PA, I can recall several instances where fact-checkers and media agencies have cited its use in debunking deepfakes online.
Both C2PA and SynthID can work cooperatively to cast a wider safety net. This isn’t an industry that would benefit from a verification standards war, but Google has a clear opportunity here to prove whether its system is more reliable and poach some of the spotlight that C2PA has clawed for itself. To prevent this from happening, C2PA needs to prove it can actually be used to demystify where the content we see online is coming from.
Related
- Does Big Tech actually care about fighting AI slop?
- America’s dangerous, messy deepfakes crackdown is here
- No one’s ready for this
Such an opportunity has already presented itself: Google announced yesterday that Meta will start using C2PA metadata to tag images on Instagram that have been captured by a camera. Meta hasn’t responded to our questions about what this will look like or what cameras will be supported, though I presume it will involve labels that say something like “captured on Pixel 10,” akin to the “sent from my iPhone” notes applied to emails. This would effectively help Instagram users to differentiate “real” photos from convincing AI fakery, which plays into the future predicted by Instagram head Adam Mosseri regarding the need to move away “from assuming what we see is real by default.”
If labeling works, that is. Instagram already checks images for C2PA information, and its attempts to label AI-generated content have previously landed the platform in hot water after it applied AI labels to images that photographers insisted they had taken themselves.
I wouldn’t be too hasty in praising Google for this team up either. The company preaches about the importance of AI transparency and combatting digital deepfakes, all the while developing the very technology that’s being used to mislead people. It’s positioned itself as both the supplier and the solution. I can forgive that if SynthID makes a noticeable difference in the fight against deepfakes, but I don’t have my hopes up given the scale of the issue.
Robust or not, SynthID and C2PA can only detect watermarks if they’ve been added in the first place, and I doubt that many of the open-source models used to generate truly nefarious deepfake content are lining up to adopt these systems. Provenance was never going to be a perfect solution, but now Google and C2PA have the opportunity to prove it’s not a complete waste of time.
- Jess WeatherbedCloseJess WeatherbedNews ReporterPosts from this author will be added to your daily email digest and your homepage feed.FollowFollowSee All by Jess Weatherbed
Posts from this author will be added to your daily email digest and your homepage feed.
See All by Jess Weatherbed
- AICloseAIPosts from this topic will be added to your daily email digest and your homepage feed.FollowFollowSee All AI
Posts from this topic will be added to your daily email digest and your homepage feed.
See All AI
- AnalysisCloseAnalysisPosts from this topic will be added to your daily email digest and your homepage feed.FollowFollowSee All Analysis
Posts from this topic will be added to your daily email digest and your homepage feed.
See All Analysis
- GoogleCloseGooglePosts from this topic will be added to your daily email digest and your homepage feed.FollowFollowSee All Google
Posts from this topic will be added to your daily email digest and your homepage feed.
See All Google
- ReportCloseReportPosts from this topic will be added to your daily email digest and your homepage feed.FollowFollowSee All Report
Posts from this topic will be added to your daily email digest and your homepage feed.
See All Report
- TechCloseTechPosts from this topic will be added to your daily email digest and your homepage feed.FollowFollowSee All Tech
Posts from this topic will be added to your daily email digest and your homepage feed.
See All Tech
More in: Google I/O 2026: All the news and announcements
Most Popular
- The 13 biggest announcements at Google I/O 2026
- Microsoft launches Surface Pro 12 and Surface Laptop 8 with Intel chips
- Google is launching its own version of OpenClaw
- Gemini is in danger of going full Copilot
- Anker fixes the two worst things about power stations
The Verge Daily
A free daily digest of the news that matters most.
This is the title for the native ad..
Posts from this topic will be added to your daily email digest and your homepage feed.
See All AI
- ReportCloseReportPosts from this topic will be added to your daily email digest and your homepage feed.FollowFollowSee All Report
Posts from this topic will be added to your daily email digest and your homepage feed.
See All Report
- TechCloseTechPosts from this topic will be added to your daily email digest and your homepage feed.FollowFollowSee All Tech
Posts from this topic will be added to your daily email digest and your homepage feed.
See All Tech
It’s make or break time for AI labeling systems
SynthID and C2PA have been given the opportunity to prove they can actually make a difference against deepfakes.
SynthID and C2PA have been given the opportunity to prove they can actually make a difference against deepfakes.
Posts from this author will be added to your daily email digest and your homepage feed.
See All by Jess Weatherbed
- Link
- Share
- Gift
Posts from this author will be added to your daily email digest and your homepage feed.
See All by Jess Weatherbed
We’re about to find out if the systems designed to make deepfakes and AI-generated content easy to spot are actually up to snuff. SynthID and C2PA Content Credentials, two distinct technologies for invisibly tagging image, video, and audio files with information about their origins, are getting their biggest expansion to date, and with it, the opportunity to turn the tide against unlabeled AI fakery that’s deceiving people online.
Yesterday during its I/O conference, Google announced that the ability to verify whether images carry SynthID markers — the invisible watermarking system that’s applied to content generated by Google AI models — is coming to Chrome and Search. That’s significant because Chrome absolutely dominates the global market share for web browsers and search engines, so AI verification tools are being shoved in front of more eyeballs. It also streamlines the checking process; if you currently want to check an image for SynthID markers, you’re expected to upload it to the Gemini app.
Not only that, but Google’s verification interfaces will now also check if these files contain C2PA information — provenance metadata that’s embedded into content at the point of creation to tell us how it was made or manipulated and if AI tools were used during the process. This C2PA adoption allows users to check suspicious images from a single interface instead of jumping between the Gemini app and dedicated C2PA verification portals since files might have only one type of label or neither.
This is the sort of collaborative effort we’ve been waiting for. While both systems work differently, both Google and the Content Authenticity Initiative (which exists to promote the C2PA standard) have made similar claims about what’s needed for them to work: for everyone to be onboard. That means more AI models need to embed this data, and online platforms where AI fakery is most often shared need to clearly display that information. For the latter, having verification tools built into the web browser could serve as a workaround on websites that don’t check or present AI metadata to their users.
OpenAI is also getting involved with this expansion, announcing yesterday that it will now embed SynthID into images generated by ChatGPT, Codex, and the OpenAI API. The company already includes C2PA metadata in generated content, but I’ve found that this is often stripped out when posted to other platforms. OpenAI itself has also wanted to temper expectations about C2PA, despite being a steering member of C2PA and now reaffirming its commitment to the standard. This is what OpenAI said on its C2PA help page, prior to it being updated to include SynthID yesterday:
“Metadata like C2PA is not a silver bullet to address issues of provenance. It can easily be removed either accidentally or intentionally. For example, most social media platforms today remove metadata from uploaded images, and actions like taking a screenshot can also remove it. Therefore, an image lacking this metadata may or may not have been generated with ChatGPT or our API.”
“Metadata like C2PA is not a silver bullet to address issues of provenance. It can easily be removed either accidentally or intentionally. For example, most social media platforms today remove metadata from uploaded images, and actions like taking a screenshot can also remove it. Therefore, an image lacking this metadata may or may not have been generated with ChatGPT or our API.”
For something that’s considered to be the very best of content authenticity tech, that sounds incredibly flimsy. Even Google describes C2PA as the industry standard, and it’s being pitched to global governments as a solution to appease AI transparency and labeling requirements. But despite being increasingly adopted by AI, hardware, and software providers, I rarely see it successfully used to verify AI fakery in the wild. SynthID seems more robust by comparison because it can’t be easily stripped out — for how limited its reach is compared to C2PA, I can recall several instances where fact-checkers and media agencies have cited its use in debunking deepfakes online.
Both C2PA and SynthID can work cooperatively to cast a wider safety net. This isn’t an industry that would benefit from a verification standards war, but Google has a clear opportunity here to prove whether its system is more reliable and poach some of the spotlight that C2PA has clawed for itself. To prevent this from happening, C2PA needs to prove it can actually be used to demystify where the content we see online is coming from.
Related
- Does Big Tech actually care about fighting AI slop?
- America’s dangerous, messy deepfakes crackdown is here
- No one’s ready for this
Such an opportunity has already presented itself: Google announced yesterday that Meta will start using C2PA metadata to tag images on Instagram that have been captured by a camera. Meta hasn’t responded to our questions about what this will look like or what cameras will be supported, though I presume it will involve labels that say something like “captured on Pixel 10,” akin to the “sent from my iPhone” notes applied to emails. This would effectively help Instagram users to differentiate “real” photos from convincing AI fakery, which plays into the future predicted by Instagram head Adam Mosseri regarding the need to move away “from assuming what we see is real by default.”
If labeling works, that is. Instagram already checks images for C2PA information, and its attempts to label AI-generated content have previously landed the platform in hot water after it applied AI labels to images that photographers insisted they had taken themselves.
I wouldn’t be too hasty in praising Google for this team up either. The company preaches about the importance of AI transparency and combatting digital deepfakes, all the while developing the very technology that’s being used to mislead people. It’s positioned itself as both the supplier and the solution. I can forgive that if SynthID makes a noticeable difference in the fight against deepfakes, but I don’t have my hopes up given the scale of the issue.
Robust or not, SynthID and C2PA can only detect watermarks if they’ve been added in the first place, and I doubt that many of the open-source models used to generate truly nefarious deepfake content are lining up to adopt these systems. Provenance was never going to be a perfect solution, but now Google and C2PA have the opportunity to prove it’s not a complete waste of time.
- Jess WeatherbedCloseJess WeatherbedNews ReporterPosts from this author will be added to your daily email digest and your homepage feed.FollowFollowSee All by Jess Weatherbed
Posts from this author will be added to your daily email digest and your homepage feed.
See All by Jess Weatherbed
- AICloseAIPosts from this topic will be added to your daily email digest and your homepage feed.FollowFollowSee All AI
Posts from this topic will be added to your daily email digest and your homepage feed.
See All AI
- AnalysisCloseAnalysisPosts from this topic will be added to your daily email digest and your homepage feed.FollowFollowSee All Analysis
Posts from this topic will be added to your daily email digest and your homepage feed.
See All Analysis
- GoogleCloseGooglePosts from this topic will be added to your daily email digest and your homepage feed.FollowFollowSee All Google
Posts from this topic will be added to your daily email digest and your homepage feed.
See All Google
- ReportCloseReportPosts from this topic will be added to your daily email digest and your homepage feed.FollowFollowSee All Report
Posts from this topic will be added to your daily email digest and your homepage feed.
See All Report
- TechCloseTechPosts from this topic will be added to your daily email digest and your homepage feed.FollowFollowSee All Tech
Posts from this topic will be added to your daily email digest and your homepage feed.
See All Tech
More in: Google I/O 2026: All the news and announcements
Most Popular
- The 13 biggest announcements at Google I/O 2026
- Microsoft launches Surface Pro 12 and Surface Laptop 8 with Intel chips
- Google is launching its own version of OpenClaw
- Gemini is in danger of going full Copilot
- Anker fixes the two worst things about power stations
The Verge Daily
A free daily digest of the news that matters most.
This is the title for the native ad..
16h ago
The biggest data center ever is becoming a huge problem in Utah
Tl;dr: The Stratos Project, a proposed 40,000-acre data center in Utah, has raised concerns about environmental impact, water usage, and community backlash. Backed by Shark Tank investor Kevin O'Leary, the project aims to establish the US as an AI superpower.
Key Takeaways:
- The Stratos Project is a proposed data center spanning 40,000 acres in Box Elder County, Utah, backed by Kevin O'Leary.
- The project is expected to consume 9GW of power, consume 448 billion cubic feet of gas per year, and produce 30.2 million tons of carbon dioxide annually.
- Residents and experts have raised concerns about environmental damage, water usage, and strain on local resources, with some calling for a referendum to halt the project.
17h ago
Open Source
Stability AI releases a new audio model that can create six-minute songs
Stability AI, the company behind Stable Diffusion, is releasing a new family of audio models, called Stability Audio 3.0. The top model can generate professional-grade music of more than six minutes long, the company claimed.
The company is releasing four new models under the Stable Audio 3.0 name: small SFX (459M parameters), small (459M parameters), medium (1.4B parameters), and large (2.7B parameters). The duo of small models is suitable for on-device sound and music generation of up to two minutes.
Both medium and large models can create full compositions of 6 minutes 20 seconds long that can maintain musical structure and melodic tone. This is more than double the length of what Stable Audio 2.0, released in 2024, was capable of generating.
Stability AI is making small SFX, small, and medium models available with open weights for anyone to use and modify. In 2024, the company released Stable Audio Open, which allowed for music generation of up to 47 seconds. The new family of models is a big step up from the previous open versions.
The large model is available only through the API and self-hosting paid services. Plus, companies with more than $1 million in revenue would need to get an enterprise license.
Many companies, including Google and ElevenLabs, are releasing models and tooling around music generation. However, as Suno and Udio’s ongoing court battles have proved, licensing of data and partnerships with music labels could become a key part of the long-term survival of these services.
Last year, Stability AI inked deals with Warner Music Group and Universal Music Group to develop models and music creation tools. The company said that its latest set of audio models is built on fully licensed data.
The AI startup is developing a new suite of products for professional musicians, but didn’t give more details on its features. Ethan Kaplan, former chief digital officer at Universal Audio and Fender, is joining the company to lead Stability’s professional music offering.
A number of AI companies are trying to bolster their credentials by hiring music execs. Earlier this year, Suno hired former Merlin CEO Jeremy Sirota as chief commercial officer. ElevenLabs has also hired Derek Cournoyer from indie music publisher Kobalt as a strategy lead for its music business...
The company is releasing four new models under the Stable Audio 3.0 name: small SFX (459M parameters), small (459M parameters), medium (1.4B parameters), and large (2.7B parameters). The duo of small models is suitable for on-device sound and music generation of up to two minutes.
Both medium and large models can create full compositions of 6 minutes 20 seconds long that can maintain musical structure and melodic tone. This is more than double the length of what Stable Audio 2.0, released in 2024, was capable of generating.
Stability AI is making small SFX, small, and medium models available with open weights for anyone to use and modify. In 2024, the company released Stable Audio Open, which allowed for music generation of up to 47 seconds. The new family of models is a big step up from the previous open versions.
The large model is available only through the API and self-hosting paid services. Plus, companies with more than $1 million in revenue would need to get an enterprise license.
Many companies, including Google and ElevenLabs, are releasing models and tooling around music generation. However, as Suno and Udio’s ongoing court battles have proved, licensing of data and partnerships with music labels could become a key part of the long-term survival of these services.
Last year, Stability AI inked deals with Warner Music Group and Universal Music Group to develop models and music creation tools. The company said that its latest set of audio models is built on fully licensed data.
The AI startup is developing a new suite of products for professional musicians, but didn’t give more details on its features. Ethan Kaplan, former chief digital officer at Universal Audio and Fender, is joining the company to lead Stability’s professional music offering.
A number of AI companies are trying to bolster their credentials by hiring music execs. Earlier this year, Suno hired former Merlin CEO Jeremy Sirota as chief commercial officer. ElevenLabs has also hired Derek Cournoyer from indie music publisher Kobalt as a strategy lead for its music business...
15h ago
If Google can’t make AI agents useful, maybe no one can
Tl;dr: Google has released Gemini Spark, an AI agent that can perform tasks across Google services and external partners, marking a significant step forward in traditional tech companies' AI agents. Gemini Spark runs continuously in the background, integrates with Google's services, and can handle tasks like planning events and summarizing inboxes.
Key Takeaways:
- Google has announced Gemini Spark, an AI agent for consumers that can perform tasks across Google services and over 30 external partners.
- Gemini Spark is cloud-based and runs continuously in the background, allowing it to have a lot more context about its tasks and giving users the ability to text and email their agents directly.
- Google's Gemini Agent is a significant step forward in traditional tech companies' AI agents, following the success of OpenClaw.
17h ago
Research
Building AI models that understand chemical principles
Tl;dr: MIT Associate Professor Connor Coley uses AI to help identify compounds that could make good drug candidates, developing models like ShEPhERD and FlowER.
Key Takeaways:
- Connor Coley, a MIT Associate Professor, uses AI to identify compounds for drug discovery, working at the intersection of chemical engineering and computer science.
- His lab develops AI models like ShEPhERD and FlowER to predict reaction products and design new molecules with desirable properties.
- Coley's work combines machine learning and cheminformatics to plan reaction pathways and synthesize new compounds, improving efficiency and accuracy.
Yesterday
Mastering Agentic Techniques: AI Agent Customization
Tl;dr: Customizing AI agents requires various techniques, including prompt engineering, retrieval-augmented generation, agent tool and skill injection, supervised fine-tuning, parameter-efficient fine-tuning, direct preference optimization, and reinforcement learning.
Key Takeaways:
- Nine techniques can be used for customizing AI agents, including prompt engineering, retrieval-augmented generation, and supervised fine-tuning.
- Prompt engineering is the most accessible and typically the first technique applied to customize agent behavior.
- Retrieval-augmented generation solves knowledge limitations and provides grounded responses.
- Supervised fine-tuning and parameter-efficient fine-tuning are used to modify model weights and adapt to specific tasks.
- Direct preference optimization and reinforcement learning are used for refining output quality and adapting to subjective preferences.
10h ago
Hardware
RTX 5080 16GB: Qwen3.6 35B MoE at 128k context — 56 tok/s, and why MTP doesn't help
MTP (Multi-Token Prediction) just merged into mainline llama.cpp at b9190. I promised u/WarthogConfident4039 a Qwen3.6 benchmarking round. Three configs, tested at real coding-agent context lengths (not just 512 tokens). The main finding surprised me.
**TL;DR: 35B Q4\_K\_XL, no MTP,** `--fit-target 1536`\*\*, 131k context. That's the config.\*\* 56 tok/s generation, 1,584 tok/s prompt processing at 128k context. MTP doesn't help at 128k — both converge to the same speed. Skip the complexity. The 27B IQ3 is worth considering if 56k context is enough for you (or if you have a 12 GB card where the 35B won't fit).
# The Configs
|Config|27B IQ3+MTP (A)|35B Q4\_K\_XL+MTP (B)|35B Q8\_0+MTP (C)|
|:-|:-|:-|:-|
|Model|Qwen3.6-27B MTP-UD-IQ3\_XXS|Qwen3.6-35B-A3B MTP-UD-Q4\_K\_XL|Qwen3.6-35B-A3B MTP-Q8\_0|
|Size|12.45 GB|\~22 GB|\~36 GB|
|Source|[GazTrab](https://huggingface.co/GazTrab/Qwen3.6-27B-MTP-UD-IQ3_XXS-GGUF)|havenoammo|Grafted|
|GPU fit|Fully on GPU (66/66)|Partial offload|Heavy offload|
All tests on: **RTX 5080 16GB**, Ryzen 9 9950X, 128GB RAM, llama.cpp **b9204** (mainline).
Common MTP flags: `-np 1 --fit on -fa on -t 20 --no-mmap --jinja -ctk q8_0 -ctv q8_0 --spec-type draft-mtp --spec-draft-n-max 2`
# Results
# Speed — The MTP Surprise
# With MTP (mtp-bench, 9 prompt types)
|Metric|27B IQ3|**35B Q4\_K\_XL**|35B Q8\_0|
|:-|:-|:-|:-|
|**Avg tok/s**|73|**74**|46|
|**Peak tok/s**|83 (code)|**86 (translation)**|51|
|**MTP accept**|74.4%|79.5%|**80.1%**|
|**--fit-target**|0|1536|1536|
# The surprise: 35B is FASTER without MTP
|35B Q4\_K\_XL config|\--fit-target|MTP?|Avg tok/s|VRAM used|
|:-|:-|:-|:-|:-|
|Best (no MTP)|0|No|**97**|15,815 MiB|
|Same VRAM budget|1536|No|86|14,269 MiB|
|MTP enabled|1536|Yes|74|14,623 MiB|
**MTP is 23% slower** for the 35B MoE on 16GB. Why?
1. MTP requires `--fit-target 1536` to reserve \~1.5 GB for the MTP compute buffer
2. That 1.5 GB pushes \~3 more MoE expert layers from GPU to CPU
3. CPU-bound expert layers are the bottleneck for MoE inference
4. MTP's multi-token speculation (\~79% acceptance) doesn't compensate for the slower per-step speed
**For the 27B, MTP helps** because the model fits entirely on GPU (12.45 GB) — `--fit-target 0` works with and without MTP, so there's no VRAM penalty. The 27B goes from \~56 tok/s (no MTP, older builds) to 73 tok/s with MTP.
**Rule of thumb: MTP helps when your model fits on GPU. It hurts when the MTP compute buffer forces more layers to CPU.**
# Speed at Coding-Agent Context Lengths (the real test)
Everyone runs coding agents at 128k. Here's what actually happens as you fill the context window. Tested with synthetic prompts (Python classes, architecture docs, error stack traces — varied enough to prevent tokenizer compression), prompt cache disabled, 35B Q4\_K\_XL with `--fit-target 1536`:
|Context|PP (no MTP)|PP (MTP)|TG (no MTP)|TG (MTP)|
|:-|:-|:-|:-|:-|
|\~8k|1,855 tok/s|1,712 tok/s|73 tok/s|79 tok/s|
|\~32k|1,810 tok/s|1,674 tok/s|74 tok/s|70 tok/s|
|\~64k|1,723 tok/s|1,583 tok/s|67 tok/s|76 tok/s|
|**\~128k**|**1,584 tok/s**|**1,437 tok/s**|**56 tok/s**|**56 tok/s**|
*8k/32k TG measured in a separate run from 64k/128k — expect \~5-10% variance between rows from measurement noise.*
**At 128k context, MTP and no-MTP converge to the same TG speed (\~56 tok/s).** The KV cache fills VRAM at long context regardless of MTP, so the offload split ends up identical. MTP's multi-token speculation is offset by its compute overhead.
**PP degrades gracefully**: 1,855 → 1,584 tok/s from 8k to 128k (\~15% decline). A 128k prompt processes in \~81 seconds.
**The "97 tok/s" only exists at short context** with `--fit-target 0`. At 64k+, `--fit-target 0` OOMs because there's no headroom for KV cache growth. You must use `--fit-target 1536` for long-context work, which brings speed down to \~73 tok/s at short context and \~56 tok/s at 128k.
**Bottom line for coding agents**: expect \~56 tok/s TG and \~1,500 tok/s PP at 128k context on 16GB. MTP is a wash — doesn't help or hurt at full context.
# VRAM Usage
|Config|VRAM used|VRAM free|Notes|
|:-|:-|:-|:-|
|A (27B IQ3+MTP)|14,803 MiB|1,039 MiB|Fully on GPU, fit-target 0|
|B (35B Q4\_K\_XL+MTP)|14,623 MiB|1,219 MiB|Partial offload, fit-target 1536|
|B (35B Q4\_K\_XL, no MTP)|15,815 MiB|27 MiB|Maximum GPU layers, fit-target 0|
|C (35B Q8\_0+MTP)|14,567 MiB|1,275 MiB|Heavy offload, fit-target 1536|
# Context Limits (push to OOM)
|Limit|27B IQ3|**35B Q4\_K\_XL**|35B Q8\_0|
|:-|:-|:-|:-|
|**Max ctx (q8\_0 KV)**|56k|**131k+**|**131k+**|
|**Max ctx (q4\_0 KV)**|110k|131k+|131k+|
|Speed at max ctx|80.5 / 57.2|**56**|45|
This is the **biggest differentiator**. The 35B MoE handles 131k context easily because its hybrid architecture (Gated DeltaNet + Attention) only has \~10 full-attention layers that need KV cache. The remaining SSM layers use a tiny recurrent state. The 27B dense model has KV on every layer, so it maxes out at 56k with q8\_0 KV.
**Tip for 27B users**: switching from `-ctk q8_0 -ctv q8_0` to `-ctk q4_0 -ctv q4_0` extends your max context from 56k → 110k. Quality cost is minimal: q4\_0 KV at 56k scores 218/220 CodeNeedle vs 220/220 with q8\_0 KV (q4\_0 at regular context: 219/220 — so most of the 2-line drop is from q4\_0 itself, not the longer context).
The OOM at higher contexts is the **MTP compute buffer** (529 MiB fixed allocation), not the KV cache itself. This is a llama.cpp implementation detail that may improve in future versions.
# Quality — CodeNeedle (positional recall)
11 functions from Python's http.server, \~50k char corpus, testing exact line-level recall:
|Metric|**27B IQ3**|35B Q4\_K\_XL|35B Q8\_0|
|:-|:-|:-|:-|
|**Pass**|**11/11**|11/11|11/11|
|**Lines matched**|**220/220**|217/220|216/220|
|**Hallucinations**|**0**|1|1|
The 27B IQ3 has a **perfect score** — every line exact, zero hallucinations. The 35B models are close but not quite there. Interesting that Q8\_0 doesn't beat Q4\_K\_XL here.
# Quality — GSM8K (grade school math, 100 cases)
|Metric|27B IQ3|**35B Q4\_K\_XL**|35B Q8\_0|
|:-|:-|:-|:-|
|**Accuracy**|89%|**91%**|90%|
|**CI (95%, excl. truncated)**|\[86.9%, 97.1%\]|\[84.9%, 95.8%\]|\[85.8%, 96.5%\]|
|**Truncated**|5|**1**|3|
|**Wall time**|106 min|**67 min**|114 min|
All three overlap in confidence intervals — the quality difference is negligible. But the 35B Q4\_K\_XL is **37% faster** to evaluate (67 vs 106 min) with fewer truncations.
*Note: AIME2025 was also tested on the 27B — 50% overall but* ***100% on non-truncated cases***\*. Every failure was context exhaustion at 32k, not wrong reasoning. The 35B MoE with 131k context would likely score higher.\*
# Ubatch PP Trick (coder543, May 18)
u/coder543 discovered that increasing `-ub` from 512→8192 gives **5.5x prompt processing speedup** for `--n-cpu-moe` partially offloaded models. I tested this on the 35B:
**Result: doesn't apply with** `--fit on`\*\*.\*\* The `-ub 2048+` OOMs because `--fit on` already maximizes VRAM for model layers — no headroom for larger batch buffers. If you use `--n-cpu-moe` manual offload instead, the trick works. But `--fit on` is simpler and handles the split automatically.
# Concurrency (-np sweep)
Tested `-np 1/2/4` on 10 GSM8K cases:
|\-np|27B tok/s|27B throughput|35B tok/s|35B throughput|
|:-|:-|:-|:-|:-|
|1|83.3|0.6 cases/min|70.7|0.8 cases/min|
|**2**|57.7|**1.3 cases/min**|49.7|**1.1 cases/min**|
|4|10.0 (CPU overflow)|0.6 cases/min|28|failed|
`-np 2` **doubles batch throughput** at 30% slower per-request speed. `-np 4` pushes layers to CPU — 27B drops to 10 tok/s, 35B partially fails. Use `-np 1` for interactive chat, `-np 2` for batch evaluation.
# MTP Reference (for 27B / fully-on-GPU setups)
MTP is worth it when the model fits entirely on GPU (no offload penalty). For the 27B IQ3 on 12GB: 73 tok/s with MTP vs \~56 without. For the 35B on 16GB: skip it (see speed table above).
If you do use MTP:
1. `--spec-type draft-mtp` — not `mtp`. Mainline renamed it.
2. `-np 1` — b9204 defaults to 4 slots which pushes layers to CPU.
3. `--spec-draft-n-max 2` beats 3 (lower acceptance at 3 = slower overall).
4. `--fit-target 1536` for partial-offload models. `--fit-target 0` for fully-on-GPU.
5. **At 128k context, MTP gives no speedup** — KV cache dominates VRAM regardless.
Other notes:
* **Hadamard KV rotation (**`-khad`**)** is enabled by default since b8607 — no flag needed.
* `-np 2` doubles batch throughput at 30% slower per-request. Good for eval, bad for interactive.
# Recommendation
# The Config (just copy this)
./llama-server \
-m Qwen3.6-35B-A3B-MTP-UD-Q4_K_XL.gguf \
-c 131072 -np 1 --fit on --fit-target 1536 \
-fa on -t 20 --no-mmap --jinja \
-ctk q8_0 -ctv q8_0
No MTP. No special flags. `--fit-target 1536` is the key — it reserves VRAM headroom so the KV cache doesn't OOM at 128k. Load it, leave it running, point your coding agent at `localhost:8080/v1/chat/completions`.
**What you get**: 56 tok/s generation at 128k context. 1,584 tok/s prompt processing (81s to ingest 128k tokens). 131k max context. GSM8K 91%. Stable.
**Why no MTP?** At 128k context both MTP and no-MTP give the same 56 tok/s — the KV cache dominates VRAM either way. MTP adds 5 gotchas for zero benefit. Skip the complexity.
GGUF: [havenoammo/Qwen3.6-35B-A3B-MTP-GGUF](https://huggingface.co/havenoammo/Qwen3.6-35B-A3B-MTP-GGUF) (the MTP GGUF works fine without `--spec-type draft-mtp` — it just ignores the extra tensors).
27B GGUF: [GazTrab/Qwen3.6-27B-MTP-UD-IQ3\_XXS-GGUF](https://huggingface.co/GazTrab/Qwen3.6-27B-MTP-UD-IQ3_XXS-GGUF)
# Other VRAM budgets (community data, not tested by us)
Everything above was tested on our RTX 5080 16GB. These estimates for other GPUs are from community reports:
|VRAM|Model|Speed|Source|
|:-|:-|:-|:-|
|**8 GB**|35B MoE Q2\_K\_XL+MTP|\~50 tok/s (est.)|u/Still-Notice8155 (GTX 1070, `-fit off --n-cpu-moe 32`)|
|**12 GB**|35B MoE Q4\_K\_XL+MTP|\~73-80 tok/s|u/janvitos (RTX 4070 Super 12GB)|
|**16 GB**|**35B Q4\_K\_XL**|**56 tok/s @ 128k**|**This post (RTX 5080)**|
|**24 GB**|35B Q4\_K\_XL (no MTP)|\~90+ tok/s (est.)|Model is \~22 GB, fits fully on GPU with headroom for KV|
The 27B IQ3+MTP needs the MTP head grafted — [`graft-mtp.py`](http://graft-mtp.py) in the repo.
# Why not the others?
**27B IQ3** — We tested it on our 16GB card where it fits fully on GPU (12.45 GB model). Perfect CodeNeedle (220/220), 73 tok/s with MTP ([GGUF](https://huggingface.co/GazTrab/Qwen3.6-27B-MTP-UD-IQ3_XXS-GGUF)). But it caps at 56k context (110k with q4\_0 KV). If your coding agent needs 128k, it's out. Better suited for 12 GB cards where the 35B won't fit.
**35B Q8\_0** — 38% slower (46 tok/s with MTP), negligible quality gain (GSM8K 90% vs 91%, overlapping CIs). Not worth the VRAM on 16 GB.
# Credits
This post exists because of the community:
* **am17an** — original MTP implementation (PR #22673), merged mainline b9190
* **havenoammo** — MTP GGUF variants + graft script
* u/janvitos — 80 tok/s MTP config on 12GB (635 upvotes), documented the flags
* u/coder543 — ubatch PP trick for `--n-cpu-moe` (May 18)
* u/OsmanthusBloom — earlier ubatch discovery
* u/Still-Notice8155 — GTX 1070 8GB MTP benchmarks proving it works everywhere
* u/raketenkater — run-time-repack, defrag-thold, -khad flags documentation
* u/moflinCASIO — 4060 Ti 16GB reference benchmarks
* u/WarthogConfident4039 — requested this benchmarking round
* **ggerganov** — llama-eval, MTP mainline merge
* u/simracerman — pushed for PP speed benchmarks ("your typical coding agent dumps 10k tokens")
* u/danielhanchen (Unsloth) — Dynamic quantization formula behind UD-Q4\_K\_XL
* u/alexziskind1 — CodeNeedle positional recall benchmark
# What's Next
**vLLM vs llama.cpp head-to-head**. vLLM >= 0.19.0 supports MTP natively with PagedAttention (dynamic KV allocation — no fixed compute buffer eating VRAM). Could make MTP actually faster for partial-offload models. Stay tuned.
EDIT: u/Look_0ver_There — corrected 24 GB VRAM table (Q8\_0 is 36 GB, doesn't fit)
EDIT 2: u/FusionX correctly points out that --fit-target 1536 is too conservative for headless setups. My machine runs a desktop compositor + terminal that eats \~1 GB VRAM before the model loads. If you're running headless, --fit-target 128 keeps more expert layers on GPU. FusionX reports 70-80 tok/s at 131k context on the same GPU with this setting. I'll re-benchmark with a lower fit-target and update. The recommended config is adjust --fit-target down if you're headless.
EDIT 3: Hey thanks everyone for commenting, and for the ones who really skeptical of the results because the post was AI generated. u/[the\_\_storm](https://www.reddit.com/user/the__storm/) u/[Special\_Animal2049](https://www.reddit.com/user/Special_Animal2049/) [kevin\_1994](https://www.reddit.com/user/kevin_1994/) I really appreciate your criticisms, and I should have been more upfront about this. So to remedy this I have posted the scripts that produced these results and the raw data themselves, you can find them here: [https://github.com/gaztrabisme/llm-server/tree/main/docs/dev](https://github.com/gaztrabisme/llm-server/tree/main/docs/dev)
EDIT 4: u/OsmanthusBloom caught that the community VRAM table incorrectly listed the 27B dense model for the 8 GB and 12 GB rows. Both sources actually ran the 35B MoE with CPU offload.
--- TOP COMMENTS ---
Is that 4283 tokens of text to say : "if my model doesn't fit in my vram, speed is bad" ?
---
Is this AI generated? Your tk/s is way too low. Secondly, assuming we're in headless mode, you DO NOT need to reserve 1536MB VRAM. KV cache is already accounted for fitting heuristics in llama. Set it as low as you can without going OOM. 128MB works for me.
I have the same setup except a much shittier DDR4 RAM. At 131k ctx size, I can reach 70 tk/s with the same GPU, model and llama params. In fact, with `--fit-target=128MB`, the speed is 80 tk/s. And 65 tk/s at 256k ctx size.
And there's still plenty of room for improvement to eek out even more performance.
As for 27b, instead of IQ3, I suggest using the IQ4 quant as described [here](https://old.reddit.com/r/LocalLLaMA/comments/1sy0qj5/qwen3627b_iq4_xs_full_vram_with_110k_context/).
Also, I do agree that MTP is largely useless for 16GB VRAM...
**TL;DR: 35B Q4\_K\_XL, no MTP,** `--fit-target 1536`\*\*, 131k context. That's the config.\*\* 56 tok/s generation, 1,584 tok/s prompt processing at 128k context. MTP doesn't help at 128k — both converge to the same speed. Skip the complexity. The 27B IQ3 is worth considering if 56k context is enough for you (or if you have a 12 GB card where the 35B won't fit).
# The Configs
|Config|27B IQ3+MTP (A)|35B Q4\_K\_XL+MTP (B)|35B Q8\_0+MTP (C)|
|:-|:-|:-|:-|
|Model|Qwen3.6-27B MTP-UD-IQ3\_XXS|Qwen3.6-35B-A3B MTP-UD-Q4\_K\_XL|Qwen3.6-35B-A3B MTP-Q8\_0|
|Size|12.45 GB|\~22 GB|\~36 GB|
|Source|[GazTrab](https://huggingface.co/GazTrab/Qwen3.6-27B-MTP-UD-IQ3_XXS-GGUF)|havenoammo|Grafted|
|GPU fit|Fully on GPU (66/66)|Partial offload|Heavy offload|
All tests on: **RTX 5080 16GB**, Ryzen 9 9950X, 128GB RAM, llama.cpp **b9204** (mainline).
Common MTP flags: `-np 1 --fit on -fa on -t 20 --no-mmap --jinja -ctk q8_0 -ctv q8_0 --spec-type draft-mtp --spec-draft-n-max 2`
# Results
# Speed — The MTP Surprise
# With MTP (mtp-bench, 9 prompt types)
|Metric|27B IQ3|**35B Q4\_K\_XL**|35B Q8\_0|
|:-|:-|:-|:-|
|**Avg tok/s**|73|**74**|46|
|**Peak tok/s**|83 (code)|**86 (translation)**|51|
|**MTP accept**|74.4%|79.5%|**80.1%**|
|**--fit-target**|0|1536|1536|
# The surprise: 35B is FASTER without MTP
|35B Q4\_K\_XL config|\--fit-target|MTP?|Avg tok/s|VRAM used|
|:-|:-|:-|:-|:-|
|Best (no MTP)|0|No|**97**|15,815 MiB|
|Same VRAM budget|1536|No|86|14,269 MiB|
|MTP enabled|1536|Yes|74|14,623 MiB|
**MTP is 23% slower** for the 35B MoE on 16GB. Why?
1. MTP requires `--fit-target 1536` to reserve \~1.5 GB for the MTP compute buffer
2. That 1.5 GB pushes \~3 more MoE expert layers from GPU to CPU
3. CPU-bound expert layers are the bottleneck for MoE inference
4. MTP's multi-token speculation (\~79% acceptance) doesn't compensate for the slower per-step speed
**For the 27B, MTP helps** because the model fits entirely on GPU (12.45 GB) — `--fit-target 0` works with and without MTP, so there's no VRAM penalty. The 27B goes from \~56 tok/s (no MTP, older builds) to 73 tok/s with MTP.
**Rule of thumb: MTP helps when your model fits on GPU. It hurts when the MTP compute buffer forces more layers to CPU.**
# Speed at Coding-Agent Context Lengths (the real test)
Everyone runs coding agents at 128k. Here's what actually happens as you fill the context window. Tested with synthetic prompts (Python classes, architecture docs, error stack traces — varied enough to prevent tokenizer compression), prompt cache disabled, 35B Q4\_K\_XL with `--fit-target 1536`:
|Context|PP (no MTP)|PP (MTP)|TG (no MTP)|TG (MTP)|
|:-|:-|:-|:-|:-|
|\~8k|1,855 tok/s|1,712 tok/s|73 tok/s|79 tok/s|
|\~32k|1,810 tok/s|1,674 tok/s|74 tok/s|70 tok/s|
|\~64k|1,723 tok/s|1,583 tok/s|67 tok/s|76 tok/s|
|**\~128k**|**1,584 tok/s**|**1,437 tok/s**|**56 tok/s**|**56 tok/s**|
*8k/32k TG measured in a separate run from 64k/128k — expect \~5-10% variance between rows from measurement noise.*
**At 128k context, MTP and no-MTP converge to the same TG speed (\~56 tok/s).** The KV cache fills VRAM at long context regardless of MTP, so the offload split ends up identical. MTP's multi-token speculation is offset by its compute overhead.
**PP degrades gracefully**: 1,855 → 1,584 tok/s from 8k to 128k (\~15% decline). A 128k prompt processes in \~81 seconds.
**The "97 tok/s" only exists at short context** with `--fit-target 0`. At 64k+, `--fit-target 0` OOMs because there's no headroom for KV cache growth. You must use `--fit-target 1536` for long-context work, which brings speed down to \~73 tok/s at short context and \~56 tok/s at 128k.
**Bottom line for coding agents**: expect \~56 tok/s TG and \~1,500 tok/s PP at 128k context on 16GB. MTP is a wash — doesn't help or hurt at full context.
# VRAM Usage
|Config|VRAM used|VRAM free|Notes|
|:-|:-|:-|:-|
|A (27B IQ3+MTP)|14,803 MiB|1,039 MiB|Fully on GPU, fit-target 0|
|B (35B Q4\_K\_XL+MTP)|14,623 MiB|1,219 MiB|Partial offload, fit-target 1536|
|B (35B Q4\_K\_XL, no MTP)|15,815 MiB|27 MiB|Maximum GPU layers, fit-target 0|
|C (35B Q8\_0+MTP)|14,567 MiB|1,275 MiB|Heavy offload, fit-target 1536|
# Context Limits (push to OOM)
|Limit|27B IQ3|**35B Q4\_K\_XL**|35B Q8\_0|
|:-|:-|:-|:-|
|**Max ctx (q8\_0 KV)**|56k|**131k+**|**131k+**|
|**Max ctx (q4\_0 KV)**|110k|131k+|131k+|
|Speed at max ctx|80.5 / 57.2|**56**|45|
This is the **biggest differentiator**. The 35B MoE handles 131k context easily because its hybrid architecture (Gated DeltaNet + Attention) only has \~10 full-attention layers that need KV cache. The remaining SSM layers use a tiny recurrent state. The 27B dense model has KV on every layer, so it maxes out at 56k with q8\_0 KV.
**Tip for 27B users**: switching from `-ctk q8_0 -ctv q8_0` to `-ctk q4_0 -ctv q4_0` extends your max context from 56k → 110k. Quality cost is minimal: q4\_0 KV at 56k scores 218/220 CodeNeedle vs 220/220 with q8\_0 KV (q4\_0 at regular context: 219/220 — so most of the 2-line drop is from q4\_0 itself, not the longer context).
The OOM at higher contexts is the **MTP compute buffer** (529 MiB fixed allocation), not the KV cache itself. This is a llama.cpp implementation detail that may improve in future versions.
# Quality — CodeNeedle (positional recall)
11 functions from Python's http.server, \~50k char corpus, testing exact line-level recall:
|Metric|**27B IQ3**|35B Q4\_K\_XL|35B Q8\_0|
|:-|:-|:-|:-|
|**Pass**|**11/11**|11/11|11/11|
|**Lines matched**|**220/220**|217/220|216/220|
|**Hallucinations**|**0**|1|1|
The 27B IQ3 has a **perfect score** — every line exact, zero hallucinations. The 35B models are close but not quite there. Interesting that Q8\_0 doesn't beat Q4\_K\_XL here.
# Quality — GSM8K (grade school math, 100 cases)
|Metric|27B IQ3|**35B Q4\_K\_XL**|35B Q8\_0|
|:-|:-|:-|:-|
|**Accuracy**|89%|**91%**|90%|
|**CI (95%, excl. truncated)**|\[86.9%, 97.1%\]|\[84.9%, 95.8%\]|\[85.8%, 96.5%\]|
|**Truncated**|5|**1**|3|
|**Wall time**|106 min|**67 min**|114 min|
All three overlap in confidence intervals — the quality difference is negligible. But the 35B Q4\_K\_XL is **37% faster** to evaluate (67 vs 106 min) with fewer truncations.
*Note: AIME2025 was also tested on the 27B — 50% overall but* ***100% on non-truncated cases***\*. Every failure was context exhaustion at 32k, not wrong reasoning. The 35B MoE with 131k context would likely score higher.\*
# Ubatch PP Trick (coder543, May 18)
u/coder543 discovered that increasing `-ub` from 512→8192 gives **5.5x prompt processing speedup** for `--n-cpu-moe` partially offloaded models. I tested this on the 35B:
**Result: doesn't apply with** `--fit on`\*\*.\*\* The `-ub 2048+` OOMs because `--fit on` already maximizes VRAM for model layers — no headroom for larger batch buffers. If you use `--n-cpu-moe` manual offload instead, the trick works. But `--fit on` is simpler and handles the split automatically.
# Concurrency (-np sweep)
Tested `-np 1/2/4` on 10 GSM8K cases:
|\-np|27B tok/s|27B throughput|35B tok/s|35B throughput|
|:-|:-|:-|:-|:-|
|1|83.3|0.6 cases/min|70.7|0.8 cases/min|
|**2**|57.7|**1.3 cases/min**|49.7|**1.1 cases/min**|
|4|10.0 (CPU overflow)|0.6 cases/min|28|failed|
`-np 2` **doubles batch throughput** at 30% slower per-request speed. `-np 4` pushes layers to CPU — 27B drops to 10 tok/s, 35B partially fails. Use `-np 1` for interactive chat, `-np 2` for batch evaluation.
# MTP Reference (for 27B / fully-on-GPU setups)
MTP is worth it when the model fits entirely on GPU (no offload penalty). For the 27B IQ3 on 12GB: 73 tok/s with MTP vs \~56 without. For the 35B on 16GB: skip it (see speed table above).
If you do use MTP:
1. `--spec-type draft-mtp` — not `mtp`. Mainline renamed it.
2. `-np 1` — b9204 defaults to 4 slots which pushes layers to CPU.
3. `--spec-draft-n-max 2` beats 3 (lower acceptance at 3 = slower overall).
4. `--fit-target 1536` for partial-offload models. `--fit-target 0` for fully-on-GPU.
5. **At 128k context, MTP gives no speedup** — KV cache dominates VRAM regardless.
Other notes:
* **Hadamard KV rotation (**`-khad`**)** is enabled by default since b8607 — no flag needed.
* `-np 2` doubles batch throughput at 30% slower per-request. Good for eval, bad for interactive.
# Recommendation
# The Config (just copy this)
./llama-server \
-m Qwen3.6-35B-A3B-MTP-UD-Q4_K_XL.gguf \
-c 131072 -np 1 --fit on --fit-target 1536 \
-fa on -t 20 --no-mmap --jinja \
-ctk q8_0 -ctv q8_0
No MTP. No special flags. `--fit-target 1536` is the key — it reserves VRAM headroom so the KV cache doesn't OOM at 128k. Load it, leave it running, point your coding agent at `localhost:8080/v1/chat/completions`.
**What you get**: 56 tok/s generation at 128k context. 1,584 tok/s prompt processing (81s to ingest 128k tokens). 131k max context. GSM8K 91%. Stable.
**Why no MTP?** At 128k context both MTP and no-MTP give the same 56 tok/s — the KV cache dominates VRAM either way. MTP adds 5 gotchas for zero benefit. Skip the complexity.
GGUF: [havenoammo/Qwen3.6-35B-A3B-MTP-GGUF](https://huggingface.co/havenoammo/Qwen3.6-35B-A3B-MTP-GGUF) (the MTP GGUF works fine without `--spec-type draft-mtp` — it just ignores the extra tensors).
27B GGUF: [GazTrab/Qwen3.6-27B-MTP-UD-IQ3\_XXS-GGUF](https://huggingface.co/GazTrab/Qwen3.6-27B-MTP-UD-IQ3_XXS-GGUF)
# Other VRAM budgets (community data, not tested by us)
Everything above was tested on our RTX 5080 16GB. These estimates for other GPUs are from community reports:
|VRAM|Model|Speed|Source|
|:-|:-|:-|:-|
|**8 GB**|35B MoE Q2\_K\_XL+MTP|\~50 tok/s (est.)|u/Still-Notice8155 (GTX 1070, `-fit off --n-cpu-moe 32`)|
|**12 GB**|35B MoE Q4\_K\_XL+MTP|\~73-80 tok/s|u/janvitos (RTX 4070 Super 12GB)|
|**16 GB**|**35B Q4\_K\_XL**|**56 tok/s @ 128k**|**This post (RTX 5080)**|
|**24 GB**|35B Q4\_K\_XL (no MTP)|\~90+ tok/s (est.)|Model is \~22 GB, fits fully on GPU with headroom for KV|
The 27B IQ3+MTP needs the MTP head grafted — [`graft-mtp.py`](http://graft-mtp.py) in the repo.
# Why not the others?
**27B IQ3** — We tested it on our 16GB card where it fits fully on GPU (12.45 GB model). Perfect CodeNeedle (220/220), 73 tok/s with MTP ([GGUF](https://huggingface.co/GazTrab/Qwen3.6-27B-MTP-UD-IQ3_XXS-GGUF)). But it caps at 56k context (110k with q4\_0 KV). If your coding agent needs 128k, it's out. Better suited for 12 GB cards where the 35B won't fit.
**35B Q8\_0** — 38% slower (46 tok/s with MTP), negligible quality gain (GSM8K 90% vs 91%, overlapping CIs). Not worth the VRAM on 16 GB.
# Credits
This post exists because of the community:
* **am17an** — original MTP implementation (PR #22673), merged mainline b9190
* **havenoammo** — MTP GGUF variants + graft script
* u/janvitos — 80 tok/s MTP config on 12GB (635 upvotes), documented the flags
* u/coder543 — ubatch PP trick for `--n-cpu-moe` (May 18)
* u/OsmanthusBloom — earlier ubatch discovery
* u/Still-Notice8155 — GTX 1070 8GB MTP benchmarks proving it works everywhere
* u/raketenkater — run-time-repack, defrag-thold, -khad flags documentation
* u/moflinCASIO — 4060 Ti 16GB reference benchmarks
* u/WarthogConfident4039 — requested this benchmarking round
* **ggerganov** — llama-eval, MTP mainline merge
* u/simracerman — pushed for PP speed benchmarks ("your typical coding agent dumps 10k tokens")
* u/danielhanchen (Unsloth) — Dynamic quantization formula behind UD-Q4\_K\_XL
* u/alexziskind1 — CodeNeedle positional recall benchmark
# What's Next
**vLLM vs llama.cpp head-to-head**. vLLM >= 0.19.0 supports MTP natively with PagedAttention (dynamic KV allocation — no fixed compute buffer eating VRAM). Could make MTP actually faster for partial-offload models. Stay tuned.
EDIT: u/Look_0ver_There — corrected 24 GB VRAM table (Q8\_0 is 36 GB, doesn't fit)
EDIT 2: u/FusionX correctly points out that --fit-target 1536 is too conservative for headless setups. My machine runs a desktop compositor + terminal that eats \~1 GB VRAM before the model loads. If you're running headless, --fit-target 128 keeps more expert layers on GPU. FusionX reports 70-80 tok/s at 131k context on the same GPU with this setting. I'll re-benchmark with a lower fit-target and update. The recommended config is adjust --fit-target down if you're headless.
EDIT 3: Hey thanks everyone for commenting, and for the ones who really skeptical of the results because the post was AI generated. u/[the\_\_storm](https://www.reddit.com/user/the__storm/) u/[Special\_Animal2049](https://www.reddit.com/user/Special_Animal2049/) [kevin\_1994](https://www.reddit.com/user/kevin_1994/) I really appreciate your criticisms, and I should have been more upfront about this. So to remedy this I have posted the scripts that produced these results and the raw data themselves, you can find them here: [https://github.com/gaztrabisme/llm-server/tree/main/docs/dev](https://github.com/gaztrabisme/llm-server/tree/main/docs/dev)
EDIT 4: u/OsmanthusBloom caught that the community VRAM table incorrectly listed the 27B dense model for the 8 GB and 12 GB rows. Both sources actually ran the 35B MoE with CPU offload.
--- TOP COMMENTS ---
Is that 4283 tokens of text to say : "if my model doesn't fit in my vram, speed is bad" ?
---
Is this AI generated? Your tk/s is way too low. Secondly, assuming we're in headless mode, you DO NOT need to reserve 1536MB VRAM. KV cache is already accounted for fitting heuristics in llama. Set it as low as you can without going OOM. 128MB works for me.
I have the same setup except a much shittier DDR4 RAM. At 131k ctx size, I can reach 70 tk/s with the same GPU, model and llama params. In fact, with `--fit-target=128MB`, the speed is 80 tk/s. And 65 tk/s at 256k ctx size.
And there's still plenty of room for improvement to eek out even more performance.
As for 27b, instead of IQ3, I suggest using the IQ4 quant as described [here](https://old.reddit.com/r/LocalLLaMA/comments/1sy0qj5/qwen3627b_iq4_xs_full_vram_with_110k_context/).
Also, I do agree that MTP is largely useless for 16GB VRAM...
17h ago
Funding
Embodied AI Fuels Record Robotics Funding In China As IPO Momentum Builds
Tl;dr: China's robotics funding has hit a record high in 2026, driven by a surge in embodied AI investments, with several top robotics startups raising massive funding rounds.
Key Takeaways:
- China-based robotics companies raised $5.6 billion across 176 deals in 2026, matching the total investment in 2021, the peak of the funding cycle.
- Investment in the robotics sector has already exceeded the $4.3 billion raised by Chinese robotics companies in all of 2025.
- Startups in the sector are working on embodied AI, which allows physical machines to observe, reason, and execute physical tasks end-to-end.
19h ago