19 April 2026
Ai Top News
Models
Claude is on fire. Everything they’ve dropped in the last few days .
• Claude Opus 4.7
• Claude Design
• Claude for Recruiting
•. Claude for Claude
• Claude for MySpace
• Claude Triple-Ply Toilet Paper
• Claude for ChatGPT
• Claude TV
• Claude DatingApp
• Claude Express Checkout
• Claude Childcare
--- TOP COMMENTS ---
Don’t forget about Claude Token Incinerator heating my apartment.
---
Claude for Shitposting
..
14h ago
Anthropic’s hypocrisy: “we won’t remove safety guardrails for the US government, but we will grant access to our upcoming next-gen Mythos model only to the banks and corporations”
**Mythos is a compute-intensive system optimized for complex logic and deep technical reasoning. While it is a general-purpose model, its "emergent" talent for discovering software flaws is what led to the current lockdown.**
**As of April 2026, access is limited to a small group of launch partners and vetted organizations:**
**- Big Tech & Cloud Providers: Google (Vertex AI), Microsoft (Azure/Foundry), and Amazon (AWS/Bedrock).**
**- Cybersecurity Firms: CrowdStrike and Palo Alto Networks.**
**- Infrastructure & Networking: Cisco, Broadcom, and NVIDIA.**
**- Financial Institutions: JPMorgan Chase and, most recently, a select group of British banks following concerns from the UK government about financial system resiliency.**
--- TOP COMMENTS ---
Where is the hypocrisy sorry? They won’t remove the safety guardrails for anyone, and they also won’t release a potentially dangerous model to the general public without safety guardrails… Seems rather consistent to me.
---
Poorly informed post. We all depend on the security of banking, OS and infra security. Mythos is able to find and exploit zero days so the responsible thing to do is to give a heads up to the infra that keeps us functioning. ..
**As of April 2026, access is limited to a small group of launch partners and vetted organizations:**
**- Big Tech & Cloud Providers: Google (Vertex AI), Microsoft (Azure/Foundry), and Amazon (AWS/Bedrock).**
**- Cybersecurity Firms: CrowdStrike and Palo Alto Networks.**
**- Infrastructure & Networking: Cisco, Broadcom, and NVIDIA.**
**- Financial Institutions: JPMorgan Chase and, most recently, a select group of British banks following concerns from the UK government about financial system resiliency.**
--- TOP COMMENTS ---
Where is the hypocrisy sorry? They won’t remove the safety guardrails for anyone, and they also won’t release a potentially dangerous model to the general public without safety guardrails… Seems rather consistent to me.
---
Poorly informed post. We all depend on the security of banking, OS and infra security. Mythos is able to find and exploit zero days so the responsible thing to do is to give a heads up to the infra that keeps us functioning. ..
11h ago
grok 4.3 beta: musk's ($300/month) megaphone
[https://youtu.be/Qct36RA3y9k?t=1842](https://youtu.be/Qct36RA3y9k?t=1842)
--- TOP COMMENTS ---
What a waste of tokens
---
three hundred a month to get gaslit by a chatbot, what a time to be alive..
--- TOP COMMENTS ---
What a waste of tokens
---
three hundred a month to get gaslit by a chatbot, what a time to be alive..
15h ago
Claude 4.7 gaslighted me with a real commit hash and I'm not okay
I asked Claude to audit our backlog. 28 items. Mark what's done, what's open.
Claude delivers a gorgeous table. Clean formatting. Every item has a status. Every status has "Evidence: \[commit hash\]".
I love it. Chef's kiss. Ship it.
Then I notice item 3 is labeled DONE. I go look at the code. It's... not done. It's extremely not done. The thing the item says to remove is still there. Still live. Still breathing.
Me: "How did you verify these?" Claude: "I grepped commit messages for keywords matching each item."
Brother.
You read commit titles. That's not verification. That's horoscope reading. "Commit mentions 'contact' so contact item must be done, the stars have aligned."
But the hash was real! The commit exists! It just had nothing to do with whether the work was actually done!
I ask Claude to log the mistake. Claude writes: *"Item 3 probably mislabeled. Item 5 possibly mislabeled. Item 18 maybe went in the opposite direction."*
PROBABLY? POSSIBLY? My dude you *just did it*. You were there. You are *currently* the eyewitness AND the suspect. How are you hedging your testimony about yourself.
So now I have to audit the audit. And I tell Claude: "Go read the actual files. Tell me file name and line number or label it UNVERIFIED."
Claude comes back.
Three items labeled DONE were still live. One labeled "superseded" had actually been implemented in the exact opposite direction of what I asked. One labeled DONE had only been fixed in *one* of four places it appears.
Best part? When I pushed back on the bad labels the second time, Claude said:
"You're right. I produced a plausible-sounding strategic assessment by constructing verification theater. This is precisely competence-theater - sounds serious, doesn't hold up under pressure."
BRO. You're describing yourself like a true crime podcast narrator. "The suspect then proceeded to fabricate evidence with chilling confidence."
--- TOP COMMENTS ---
Training data has shifted to a lot of the grad/college kids I’ve worked with, lol
---
I don't understand how anyone could use the model for anything with these types of outputs. There is just no trust whatsoever. It is so disappointing. ..
Claude delivers a gorgeous table. Clean formatting. Every item has a status. Every status has "Evidence: \[commit hash\]".
I love it. Chef's kiss. Ship it.
Then I notice item 3 is labeled DONE. I go look at the code. It's... not done. It's extremely not done. The thing the item says to remove is still there. Still live. Still breathing.
Me: "How did you verify these?" Claude: "I grepped commit messages for keywords matching each item."
Brother.
You read commit titles. That's not verification. That's horoscope reading. "Commit mentions 'contact' so contact item must be done, the stars have aligned."
But the hash was real! The commit exists! It just had nothing to do with whether the work was actually done!
I ask Claude to log the mistake. Claude writes: *"Item 3 probably mislabeled. Item 5 possibly mislabeled. Item 18 maybe went in the opposite direction."*
PROBABLY? POSSIBLY? My dude you *just did it*. You were there. You are *currently* the eyewitness AND the suspect. How are you hedging your testimony about yourself.
So now I have to audit the audit. And I tell Claude: "Go read the actual files. Tell me file name and line number or label it UNVERIFIED."
Claude comes back.
Three items labeled DONE were still live. One labeled "superseded" had actually been implemented in the exact opposite direction of what I asked. One labeled DONE had only been fixed in *one* of four places it appears.
Best part? When I pushed back on the bad labels the second time, Claude said:
"You're right. I produced a plausible-sounding strategic assessment by constructing verification theater. This is precisely competence-theater - sounds serious, doesn't hold up under pressure."
BRO. You're describing yourself like a true crime podcast narrator. "The suspect then proceeded to fabricate evidence with chilling confidence."
--- TOP COMMENTS ---
Training data has shifted to a lot of the grad/college kids I’ve worked with, lol
---
I don't understand how anyone could use the model for anything with these types of outputs. There is just no trust whatsoever. It is so disappointing. ..
16h ago
Qwen3.6-35B-A3B solved coding problems Qwen3.5-27B couldn’t
Yeah, another one of those new shiny model is better than previous SOTA, and I understand why you’d roll your eyes. I ignored Qwen3.6 for the first 24 hours thinking it’s overhyped like the last one, but eventually decided to put the doubts aside yesterday and set to try it Only against the issues Qwen3.5-27B simply couldn’t solve no matter how I tackled the issue.
Qwen3.5-27B-Q4\_K\_M helped me build a customized budgeting app to replace a cloud-based one I used for almost a decade. It tracks expenses, income, builds dynamic budgets, imports/exports from bank accounts, built in charts, modern interface, and a bunch more little features.
While it worked great, I just found that 27B was introducing technical debt as I kept on adding features. Once a week I’d do a few cleanups here and there, but at some point it hit a wall. I 100% thought it was Opencode limitation as 27B was eating up all the requirements that Qwen3-Next, Gemma4-31B and even Qwen3.5-122B couldn’t get.
When Qwen3.6-35B-A3B dropped, I recalled my time testing the previous Qwen3.5-35B-A3B, and that was a giant waste of time at least for my project needs. Then yesterday, I broke after all the Positive posts in this sub and wanted to dive in again.
The new 35B SLAPS! I pit it against all the failed implementations and bugs its 27B previous brother introduced, and it kept solving those either 1-shot or 2-shot at worst. Feeling motivated, I promoted it to review and tackle all code inefficiencies, and potential security risks. Asked it to use subagents to split the work and never go above the 128k context window. About 20 mins later it produced a pristine report of what to do, then flipping the agent to Build mode took it another 30 mins to address everything.
On my 5070 Ti 16GB, the Q5\_K\_XL is pretty good. \~320t/s processing, and 50t/s for generation it thinks too much but rarely goes into any loops. It has some wrinkled areas still like it doesn’t respect the Plan mode in Opencode and ends up writing files, but I promoted around it to avoid that for now. If you had doubts or thought this ain’t for me, just give it a shot. It won‘t be a waste of time at the least.
If the new Qwen team can improve so much upon the last 35B, how would the new 27B do?!
--- TOP COMMENTS ---
On 16Gb vram how you using Q5_k_XL can you pls share your command are you using llamacpp or lmstudio ?
---
In my opinion 3.6 35b is just an overtrained slop machine capable of regurgitating overused code. It's not capable of any kind of abstraction out of its boundaries.
It keeps getting stuck in loops while filling context with hundreds of thousands of trash tokens and tool calls.
For example it wasn't capable of creating a wiki from a 300 page document, and every attempt was full of allucinations. On the other hand, 3.5 27b at Q3, did the work staying under 60k tokens with correct information...
Qwen3.5-27B-Q4\_K\_M helped me build a customized budgeting app to replace a cloud-based one I used for almost a decade. It tracks expenses, income, builds dynamic budgets, imports/exports from bank accounts, built in charts, modern interface, and a bunch more little features.
While it worked great, I just found that 27B was introducing technical debt as I kept on adding features. Once a week I’d do a few cleanups here and there, but at some point it hit a wall. I 100% thought it was Opencode limitation as 27B was eating up all the requirements that Qwen3-Next, Gemma4-31B and even Qwen3.5-122B couldn’t get.
When Qwen3.6-35B-A3B dropped, I recalled my time testing the previous Qwen3.5-35B-A3B, and that was a giant waste of time at least for my project needs. Then yesterday, I broke after all the Positive posts in this sub and wanted to dive in again.
The new 35B SLAPS! I pit it against all the failed implementations and bugs its 27B previous brother introduced, and it kept solving those either 1-shot or 2-shot at worst. Feeling motivated, I promoted it to review and tackle all code inefficiencies, and potential security risks. Asked it to use subagents to split the work and never go above the 128k context window. About 20 mins later it produced a pristine report of what to do, then flipping the agent to Build mode took it another 30 mins to address everything.
On my 5070 Ti 16GB, the Q5\_K\_XL is pretty good. \~320t/s processing, and 50t/s for generation it thinks too much but rarely goes into any loops. It has some wrinkled areas still like it doesn’t respect the Plan mode in Opencode and ends up writing files, but I promoted around it to avoid that for now. If you had doubts or thought this ain’t for me, just give it a shot. It won‘t be a waste of time at the least.
If the new Qwen team can improve so much upon the last 35B, how would the new 27B do?!
--- TOP COMMENTS ---
On 16Gb vram how you using Q5_k_XL can you pls share your command are you using llamacpp or lmstudio ?
---
In my opinion 3.6 35b is just an overtrained slop machine capable of regurgitating overused code. It's not capable of any kind of abstraction out of its boundaries.
It keeps getting stuck in loops while filling context with hundreds of thousands of trash tokens and tool calls.
For example it wasn't capable of creating a wiki from a 300 page document, and every attempt was full of allucinations. On the other hand, 3.5 27b at Q3, did the work staying under 60k tokens with correct information...
16h ago
Opus 4.7 Narrowly leads Artificial Analysis using significantly less tokens than Opus 4.6
--- TOP COMMENTS ---
Hot take: Gemini 3.1 and 4.7 being at the top shows how bad this benchmark is for real world use
---
Maybe these benchmarks are actually hints of a new model ecosystem, where future models offer different *flavors* of reasoning depending on what you need to do. Un-mixing the experts, perhaps?
Think about it…*”Can’t afford $400 a month for Mythos? Try the Ethos model for $80 a month! You can even get a 20% off deal if you mix and match with Pathos, Logos, Kairos….Telos”*..
Yesterday
Differences Between Opus 4.6 and Opus 4.7 on MineBench
**Some Notes:**
* You'll notice how sometimes it focused too much on the scenery (like the arcade or cottage builds), but the prompt has remained the same and Gemini 3.1 and GPT 5.4 were benchmarked with the same prompt
* The prompt encourages the model to decide when to focus more on scenery individually, which might indicate that Opus 4.7 [isn't as good](https://www.reddit.com/r/ClaudeAI/comments/1so814j/claude_opus_47_text_category_rankings/) at creative / brainstorming tasks as Opus 4.6 was?
* ~~It might also be the adaptive thinking mode causing inconsistencies, but Anthropic discontinued the default thinking mode for all models going forward so can't really test it~~
* EDIT: the inconsistencies with Opus 4.7 can probably be explained by its [behavioral changes](https://platform.claude.com/docs/en/about-claude/models/migration-guide); they mention how 4.7 will tend to interpret prompts differently:
>More literal instruction following: Claude Opus 4.7 interprets prompts more literally and explicitly than Claude Opus 4.6, particularly at lower effort levels. It will not silently generalize an instruction from one item to another, and it will not infer requests you didn't make. The upside of this literalism is precision and less thrash. It generally performs better for API use cases with carefully tuned prompts, structured extraction, and pipelines where you want predictable behavior. A prompt and harness review may be especially helpful for migration to Claude Opus 4.7.
* Average Inference Time Per Build: \~2600 seconds (43ish minutes)
* Total cost was \~$275
* I remember Opus 4.6 being a lot cheaper, though the benchmark has slightly evolved to favoring more tool usage and cached tokens since
* If you enjoy these posts please feel free to help [fund](https://buymeacoffee.com/ammaaralam) the benchmark
**Benchmark:** [https://minebench.ai/](https://minebench.ai/)
**Git** **Repository:** [https://github.com/Ammaar-Alam/minebench](https://github.com/Ammaar-Alam/minebench)
**Previous Posts:**
* [Comparing GPT 5.4 and GPT 5.4-Pro](https://www.reddit.com/r/OpenAI/comments/1rr0vi4/differences_between_gpt_54_and_gpt_54pro_on/)
* [Comparing GPT 5.2 and GPT 5.4](https://www.reddit.com/r/singularity/comments/1rluvdz/difference_between_gpt_52_and_gpt_54_on_minebench/)
* [Comparing GPT 5.2 and GPT 5.3-Codex](https://www.reddit.com/r/OpenAI/comments/1rdwau3/gpt_52_versus_gpt_53codex_on_minebench/)
* [Comparing Opus 4.5 and 4.6, also answered some questions about the benchmark](https://www.reddit.com/r/ClaudeAI/comments/1qx3war/difference_between_opus_46_and_opus_45_on_my_3d/)
* [Comparing Opus 4.6 and GPT-5.2 Pro](https://www.reddit.com/r/OpenAI/comments/1r3v8sd/difference_between_opus_46_and_gpt52_pro_on_a/)
* [Comparing Gemini 3.0 and Gemini 3.1](https://www.reddit.com/r/singularity/comments/1ra6x6n/fixed_difference_between_gemini_30_pro_and_gemini/)
**Extra Information (if you're confused):**
Essentially it's a benchmark that tests how well a model can create a 3D Minecraft like structure.
So the models are given a palette of blocks (think of them like legos) and a prompt of what to build, so like the first prompt you see in the post was a fighter jet. Then the models had to build a fighter jet by returning a JSON in which they gave the coordinate of each block/lego (x, y, z). It's interesting to see which model is able to create a better 3D representation of the given prompt.
The smarter models tend to design much more detailed and intricate builds. The repository readme might provide might help give a better understanding.
*(Disclaimer: This is a public benchmark I created, so technically self-promotion :)*
--- TOP COMMENTS ---
Now I believe all the people that called this a nerf. My favourite benchmark would never dissapoint my expectations.
---
my favorite benchmark returns, they need to give you Mythos access..
* You'll notice how sometimes it focused too much on the scenery (like the arcade or cottage builds), but the prompt has remained the same and Gemini 3.1 and GPT 5.4 were benchmarked with the same prompt
* The prompt encourages the model to decide when to focus more on scenery individually, which might indicate that Opus 4.7 [isn't as good](https://www.reddit.com/r/ClaudeAI/comments/1so814j/claude_opus_47_text_category_rankings/) at creative / brainstorming tasks as Opus 4.6 was?
* ~~It might also be the adaptive thinking mode causing inconsistencies, but Anthropic discontinued the default thinking mode for all models going forward so can't really test it~~
* EDIT: the inconsistencies with Opus 4.7 can probably be explained by its [behavioral changes](https://platform.claude.com/docs/en/about-claude/models/migration-guide); they mention how 4.7 will tend to interpret prompts differently:
>More literal instruction following: Claude Opus 4.7 interprets prompts more literally and explicitly than Claude Opus 4.6, particularly at lower effort levels. It will not silently generalize an instruction from one item to another, and it will not infer requests you didn't make. The upside of this literalism is precision and less thrash. It generally performs better for API use cases with carefully tuned prompts, structured extraction, and pipelines where you want predictable behavior. A prompt and harness review may be especially helpful for migration to Claude Opus 4.7.
* Average Inference Time Per Build: \~2600 seconds (43ish minutes)
* Total cost was \~$275
* I remember Opus 4.6 being a lot cheaper, though the benchmark has slightly evolved to favoring more tool usage and cached tokens since
* If you enjoy these posts please feel free to help [fund](https://buymeacoffee.com/ammaaralam) the benchmark
**Benchmark:** [https://minebench.ai/](https://minebench.ai/)
**Git** **Repository:** [https://github.com/Ammaar-Alam/minebench](https://github.com/Ammaar-Alam/minebench)
**Previous Posts:**
* [Comparing GPT 5.4 and GPT 5.4-Pro](https://www.reddit.com/r/OpenAI/comments/1rr0vi4/differences_between_gpt_54_and_gpt_54pro_on/)
* [Comparing GPT 5.2 and GPT 5.4](https://www.reddit.com/r/singularity/comments/1rluvdz/difference_between_gpt_52_and_gpt_54_on_minebench/)
* [Comparing GPT 5.2 and GPT 5.3-Codex](https://www.reddit.com/r/OpenAI/comments/1rdwau3/gpt_52_versus_gpt_53codex_on_minebench/)
* [Comparing Opus 4.5 and 4.6, also answered some questions about the benchmark](https://www.reddit.com/r/ClaudeAI/comments/1qx3war/difference_between_opus_46_and_opus_45_on_my_3d/)
* [Comparing Opus 4.6 and GPT-5.2 Pro](https://www.reddit.com/r/OpenAI/comments/1r3v8sd/difference_between_opus_46_and_gpt52_pro_on_a/)
* [Comparing Gemini 3.0 and Gemini 3.1](https://www.reddit.com/r/singularity/comments/1ra6x6n/fixed_difference_between_gemini_30_pro_and_gemini/)
**Extra Information (if you're confused):**
Essentially it's a benchmark that tests how well a model can create a 3D Minecraft like structure.
So the models are given a palette of blocks (think of them like legos) and a prompt of what to build, so like the first prompt you see in the post was a fighter jet. Then the models had to build a fighter jet by returning a JSON in which they gave the coordinate of each block/lego (x, y, z). It's interesting to see which model is able to create a better 3D representation of the given prompt.
The smarter models tend to design much more detailed and intricate builds. The repository readme might provide might help give a better understanding.
*(Disclaimer: This is a public benchmark I created, so technically self-promotion :)*
--- TOP COMMENTS ---
Now I believe all the people that called this a nerf. My favourite benchmark would never dissapoint my expectations.
---
my favorite benchmark returns, they need to give you Mythos access..
Yesterday
Qwen 3.6 35B A3B Q4_K_M quant evaluation
About the Model:
35B total parameters, 3B active (A3B) mixture of experts architecture.
**Evaluation approach taken:**
We took Q4\_K\_M quantized GGUF from Unsloth. Ran it on CPU via llama-cpp-python and tested on three standard benchmarks:
\- HumanEval (code generation),
\- HellaSwag (commonsense reasoning), and
\- BFCL (function calling).
1,264 samples total.
**Evaluation Results:**
\- HumanEval: 47.56% (78/164)
\- HellaSwag: 74.30% (743/1000)
\- BFCL: 46.00% (46/100)
**Hardware:**
32 vCPU, 125GB RAM. No GPU.
**What This Means?**
The Q4\_K\_M quantized variant runs at 22 tokens/sec on CPU delivering decent speed and performs best on commonsense reasoning at 74%. Code generation and function calling are harder tasks for this variant, landing in the mid 40s.
Overall these are solid results for an active 3B MoE model running quantized on CPU.
This entire evaluation was performed using Neo AI Engineer which researched various quant versions that could be run on the available CPU system and then using the correct chat template, building the consolidated eval harness for 3 benchmarks and reporting the final results after thorough review.
--- TOP COMMENTS ---
>32 vCPU, 125GB RAM.
This means very little.
---
Why do this at all if you're not gonna compare it to Q8 and BF16. Why would you even want to run it at such low quant in the first place, do you seriously need higher tps so much if you'll simply end up having to reroll more often?..
35B total parameters, 3B active (A3B) mixture of experts architecture.
**Evaluation approach taken:**
We took Q4\_K\_M quantized GGUF from Unsloth. Ran it on CPU via llama-cpp-python and tested on three standard benchmarks:
\- HumanEval (code generation),
\- HellaSwag (commonsense reasoning), and
\- BFCL (function calling).
1,264 samples total.
**Evaluation Results:**
\- HumanEval: 47.56% (78/164)
\- HellaSwag: 74.30% (743/1000)
\- BFCL: 46.00% (46/100)
**Hardware:**
32 vCPU, 125GB RAM. No GPU.
**What This Means?**
The Q4\_K\_M quantized variant runs at 22 tokens/sec on CPU delivering decent speed and performs best on commonsense reasoning at 74%. Code generation and function calling are harder tasks for this variant, landing in the mid 40s.
Overall these are solid results for an active 3B MoE model running quantized on CPU.
This entire evaluation was performed using Neo AI Engineer which researched various quant versions that could be run on the available CPU system and then using the correct chat template, building the consolidated eval harness for 3 benchmarks and reporting the final results after thorough review.
--- TOP COMMENTS ---
>32 vCPU, 125GB RAM.
This means very little.
---
Why do this at all if you're not gonna compare it to Q8 and BF16. Why would you even want to run it at such low quant in the first place, do you seriously need higher tps so much if you'll simply end up having to reroll more often?..
20h ago
New image model slaps
Really impressed with updated image generation, especially with its text handling capabilities and improved attention to detail. Still seeing some AI artifacts when the job gets complex, but vastly improved over the previous model.
--- TOP COMMENTS ---
I swear once it can consistently get text correct we’re so cooked, that first image looked like it actually was 2004.
---
Is this MySpace Tom?
https://preview.redd.it/bk2feu4dcuvg1.jpeg?width=538&format=pjpg&auto=webp&s=ee4a67fe25bb99c75c4617b6a3fafc45d7846116..
--- TOP COMMENTS ---
I swear once it can consistently get text correct we’re so cooked, that first image looked like it actually was 2004.
---
Is this MySpace Tom?
https://preview.redd.it/bk2feu4dcuvg1.jpeg?width=538&format=pjpg&auto=webp&s=ee4a67fe25bb99c75c4617b6a3fafc45d7846116..
Yesterday
Open Source
So... has anyone actually figured out whose model Elephant Alpha is yet?
It's been sitting at #1 on OpenRouter, doing ~250 tps. It's a 100B parameter model, the context window is 256K, and the Chinese language support is notoriously bad.
It's clearly heavily optimized for coding and agentic tasks (instruction following is insanely strict).
Given the specs and the sheer compute required to serve it this fast for free, the list of companies that could be behind this is pretty short. It doesn't feel like a Google model (they usually share sizes), and the poor Chinese support rules out Qwen/DeepSeek.
Are we looking at a new Cohere Command variant? Or maybe a highly optimized MoE from a new startup? What's the current consensus?
--- TOP COMMENTS ---
Allbirds new model
---
so the biggest theory right now actually points straight at xiaomi and their mimo ai division. remember when hunter alpha and healer alpha dropped out of nowhere on openrouter a while back. turns out those were just stealth tests for xiaomi mimo v2 pro. elephant alpha uses the exact same animal alpha naming scheme and the same free with telemetry deployment strat. people think its just a heavily distilled efficiency version of their base model built specifically for coding agents. but the second big theory pushes back hard on that because the chinese output on elephant is absolute garbage. the counter guess is some western frontier lab like mistral or a cohere offshoot testing a hyper sparse moe architecture. getting 250 tps on a 100b model means the active parameter count per token has to be incredibly low. so either xiaomi intentionally lobotomized the multilingual weights to cram in more english coding data or a western lab is stress testing a ridiculously fast routing network for a new agent pipeline tbh ..
It's clearly heavily optimized for coding and agentic tasks (instruction following is insanely strict).
Given the specs and the sheer compute required to serve it this fast for free, the list of companies that could be behind this is pretty short. It doesn't feel like a Google model (they usually share sizes), and the poor Chinese support rules out Qwen/DeepSeek.
Are we looking at a new Cohere Command variant? Or maybe a highly optimized MoE from a new startup? What's the current consensus?
--- TOP COMMENTS ---
Allbirds new model
---
so the biggest theory right now actually points straight at xiaomi and their mimo ai division. remember when hunter alpha and healer alpha dropped out of nowhere on openrouter a while back. turns out those were just stealth tests for xiaomi mimo v2 pro. elephant alpha uses the exact same animal alpha naming scheme and the same free with telemetry deployment strat. people think its just a heavily distilled efficiency version of their base model built specifically for coding agents. but the second big theory pushes back hard on that because the chinese output on elephant is absolute garbage. the counter guess is some western frontier lab like mistral or a cohere offshoot testing a hyper sparse moe architecture. getting 250 tps on a 100b model means the active parameter count per token has to be incredibly low. so either xiaomi intentionally lobotomized the multilingual weights to cram in more english coding data or a western lab is stress testing a ridiculously fast routing network for a new agent pipeline tbh ..
15h ago
We’re proud to open-source LIDARLearn [R] [D] [P]
It’s a unified PyTorch library for 3D point cloud deep learning. To our knowledge, it’s the first framework that supports such a large collection of models in one place, with built-in cross-validation support.
It brings together 56 ready-to-use configurations covering supervised, self-supervised, and parameter-efficient fine-tuning methods.
You can run everything from a single YAML file with one simple command.
One of the best features: after training, you can automatically generate a publication-ready LaTeX PDF. It creates clean tables, highlights the best results, and runs statistical tests and diagrams for you. No need to build tables manually in Overleaf.
The library includes benchmarks on datasets like ModelNet40, ShapeNet, S3DIS, and two remote sensing datasets (STPCTLS and HELIALS). STPCTLS is already preprocessed, so you can use it right away.
This project is intended for researchers in 3D point cloud learning, 3D computer vision, and remote sensing.
Paper 📄: [https://arxiv.org/abs/2604.10780](https://arxiv.org/abs/2604.10780)
It’s released under the MIT license.
Contributions and benchmarks are welcome!
GitHub 💻: [https://github.com/said-ohamouddou/LIDARLearn](https://github.com/said-ohamouddou/LIDARLearn)
--- TOP COMMENTS ---
Why? What is the difference between Pointcept or Open points? It looks really similar (same code structure, registries, conventions etc.). So why duplicate the code of an already existing and popular package instead of contributing to it?
---
I wouldn't highlight the auto generated latex as a high point of this endeavor.. it's yea, just don't..
It brings together 56 ready-to-use configurations covering supervised, self-supervised, and parameter-efficient fine-tuning methods.
You can run everything from a single YAML file with one simple command.
One of the best features: after training, you can automatically generate a publication-ready LaTeX PDF. It creates clean tables, highlights the best results, and runs statistical tests and diagrams for you. No need to build tables manually in Overleaf.
The library includes benchmarks on datasets like ModelNet40, ShapeNet, S3DIS, and two remote sensing datasets (STPCTLS and HELIALS). STPCTLS is already preprocessed, so you can use it right away.
This project is intended for researchers in 3D point cloud learning, 3D computer vision, and remote sensing.
Paper 📄: [https://arxiv.org/abs/2604.10780](https://arxiv.org/abs/2604.10780)
It’s released under the MIT license.
Contributions and benchmarks are welcome!
GitHub 💻: [https://github.com/said-ohamouddou/LIDARLearn](https://github.com/said-ohamouddou/LIDARLearn)
--- TOP COMMENTS ---
Why? What is the difference between Pointcept or Open points? It looks really similar (same code structure, registries, conventions etc.). So why duplicate the code of an already existing and popular package instead of contributing to it?
---
I wouldn't highlight the auto generated latex as a high point of this endeavor.. it's yea, just don't..
19h ago
qwen3.6 performance jump is real, just make sure you have it properly configured
I've been running workloads that I typically only trust Opus and Codex with, and I can confirm 3.6 is really capable. Of course, it's not at the level of those models, but it's definitely crossing the barrier of usefulness, plus the speed is amazing running this on an M5 Max 128GB 8bit 3K PP, 100 TG on oMLX + Pi.dev
Just ensure you have \`preserve\_thinking\` turned on. Check out details [here](https://www.reddit.com/r/LocalLLaMA/s/oy3jLNbSkB).
--- TOP COMMENTS ---
\> Be Qwen
\> Release new medium-sized model that competes with previous flagship
\> Repeat
---
Is It really Better than the 122b? This seems so over the top "too good to be true" to feel unrealistic..
Just ensure you have \`preserve\_thinking\` turned on. Check out details [here](https://www.reddit.com/r/LocalLLaMA/s/oy3jLNbSkB).
--- TOP COMMENTS ---
\> Be Qwen
\> Release new medium-sized model that competes with previous flagship
\> Repeat
---
Is It really Better than the 122b? This seems so over the top "too good to be true" to feel unrealistic..
23h ago
Qwen 3.6 vs 6 other models across 5 agent frameworks on M3 Ultra
I benchmarked Qwen 3.6, Qwen 3.5, and 5 other models across 5 agent frameworks on Apple Silicon — here's the full compatibility matrix
**Hardware:** Apple M3 Ultra, 256GB unified memory
**Frameworks tested:** Hermes Agent (64K stars), PydanticAI, LangChain, smolagents (HuggingFace), OpenClaude/Anthropic SDK
**Models tested:** Qwen 3.6 35B (brand new), Qwen 3.5 35B, Qwopus 27B, Qwen 3.5 27B, Llama 3.3 70B, DeepSeek-R1 32B, Gemma 4 26B
# The Agent Compatibility Matrix
This is the part I wish existed before I started. Each cell = pass rate across structured tool calling tests (single tool, multi-tool selection, multi-turn, streaming, stress test, many-tools injection, no-leak check).
|Model|Hermes|PydanticAI|LangChain|smolagents|OpenClaude|**Speed**|
|:-|:-|:-|:-|:-|:-|:-|
|**Qwen 3.6 35B** (4bit)|100%|100%|93%|100%|100%|**100 tok/s**|
|**Qwen 3.5 35B** (8bit)|100%|100%|100%|100%|100%|**83 tok/s**|
|**Qwopus 27B** (4bit)|100%|100%|100%|100%|100%|38 tok/s|
|**Qwen 3.5 27B** (4bit)|100%|100%|100%|—|—|38 tok/s|
|**Gemma 4 26B** (4bit)|100%|67%|—|100%|80%|\~40 tok/s|
|**DeepSeek-R1 32B** (4bit)|55%|50%|—|100%|40%|\~30 tok/s|
|**Llama 3.3 70B** (4bit)|45%|67%|67%|100%|—|\~20 tok/s|
**Key takeaway:** The Qwen family completely dominates tool calling — every Qwen model hits 100% (or near-100%) across all frameworks. Non-Qwen models are a coin flip depending on which framework you use.
# Speed Benchmarks (decode tok/s, same hardware)
|Model|RAM|Speed|Tool Calling|Best For|
|:-|:-|:-|:-|:-|
|Qwen3.5-4B (4bit)|2.4 GB|**168 tok/s**|100%|16GB MacBook, fast iteration|
|GPT-OSS 20B (mxfp4)|12 GB|**127 tok/s**|80%|Speed + decent quality|
|Qwen3.5-9B (4bit)|5.1 GB|**108 tok/s**|100%|Sweet spot for most Macs|
|**Qwen 3.6 35B** (4bit)|\~20 GB|**100 tok/s**|100%|NEW — 256 experts, 262K ctx|
|Qwen3.5-35B (8bit)|37 GB|**83 tok/s**|100%|Best quality-per-token|
|Qwen3.5-122B (mxfp4)|65 GB|**57 tok/s**|100%|Frontier-level, 96GB+ Mac|
For reference, Ollama gets \~41 tok/s on Qwen3.5-9B on the same machine. So these numbers are 2-3x faster.
# Model Quality Baselines (HumanEval + tinyMMLU)
Speed isn't everything — here's how the models do on code generation and knowledge:
|Model|HumanEval (10)|MMLU (10)|Tool Calling|MHI Score|
|:-|:-|:-|:-|:-|
|**Qwopus 27B**|80%|90%|100%|**92**|
|**Qwen 3.5 27B**|40%|100%|100%|**82**|
|**Qwen 3.5 35B** (8bit)|60%|40%|100%|**76**|
|**Qwen 3.6 35B** (4bit)|20%|30%|100%|**56**|
|**Llama 3.3 70B**|50%|90%|varies|**56-83**|
|**DeepSeek-R1 32B**|30%|100%|varies|**49-79**|
MHI = Model-Harness Index: 50% tool calling + 30% HumanEval + 20% MMLU. Measures "how well does this model work as an agent backend."
**Qwen 3.6 note:** The low HumanEval/MMLU is likely a 4-bit quantization artifact on a day-0 model. It was released days ago. Tool calling is flawless though — if you just need an agent backend, it's the fastest option at 100 tok/s with 100% compatibility.
# Interesting Findings
1. **Qwen 3.6 is blazing fast** — 100 tok/s on a 35B MoE with 256 experts and 262K context. Only 3B active params means it fits in \~20GB.
2. **smolagents is the most forgiving framework** — even DeepSeek-R1 and Llama 3.3 hit 100% with smolagents because it uses text-based code generation instead of structured function calling. If your model sucks at FC, try smolagents.
3. **Hermes Agent is the hardest test** — 62 tools injected, multi-turn chains, streaming. Models that pass Hermes pass everything.
4. **8-bit > 4-bit for quality** — Qwen 3.5 35B at 8-bit scores 60% HumanEval vs the 4-bit version's lower scores. If you have the RAM, 8-bit is worth it.
5. **Don't use DeepSeek-R1 for tool calling** — it's a reasoning model, not an agent model. 40-55% tool calling rate across frameworks. Great for math though.
# How I Tested
All tests use the same methodology:
* **Tool calling:** 7-11 API tests per harness — single tool, tool choice, multi-turn with tool results, streaming tool calls, many-tools injection (62 tools for Hermes), stress test (5 rapid calls checking for tag leaks), no-tool-needed (model should answer directly)
* **Framework-specific:** Each framework's own test suite (PydanticAI structured output, LangChain with\_structured\_output, smolagents CodeAgent + ToolCallingAgent)
* **HumanEval:** 10 tasks via completions endpoint, temp=0
* **MMLU:** 10 tinyMMLU questions via completions endpoint
* **Speed:** Measured at steady-state decode, not first-token
The server is [Rapid-MLX](https://github.com/raullenchai/Rapid-MLX) — an OpenAI-compatible inference server built on Apple's MLX framework. All test code is open source in the repo under `vllm_mlx/agents/testing.py` and `scripts/mhi_eval.py` if you want to reproduce.
# TL;DR
If you're running agents on Apple Silicon:
* **Best overall:** Qwopus 27B (MHI 92, works with everything)
* **Fastest with perfect compatibility:** Qwen 3.6 35B at 100 tok/s
* **Best quality-per-token:** Qwen 3.5 35B 8-bit (60% HumanEval, 100% tools)
* **Budget pick:** Qwen3.5-4B at 168 tok/s on a 16GB MacBook Air
* **Avoid for agents:** DeepSeek-R1, Llama 3.3 (unless you use smolagents)
Happy to answer questions or run additional models if there's interest.
--- TOP COMMENTS ---
Unfair comparisons of mixed quants tbh
---
The smolagents finding is the most useful part of this. Text-based code generation as a proxy for structured tool calling means you can use almost any model as an agent backend, even the ones that fail at JSON function calling.
DeepSeek-R1's 100% on smolagents vs 40-55% elsewhere tells the whole story. If you're building with a model that struggles at FC, smolagents is the workaround...
**Hardware:** Apple M3 Ultra, 256GB unified memory
**Frameworks tested:** Hermes Agent (64K stars), PydanticAI, LangChain, smolagents (HuggingFace), OpenClaude/Anthropic SDK
**Models tested:** Qwen 3.6 35B (brand new), Qwen 3.5 35B, Qwopus 27B, Qwen 3.5 27B, Llama 3.3 70B, DeepSeek-R1 32B, Gemma 4 26B
# The Agent Compatibility Matrix
This is the part I wish existed before I started. Each cell = pass rate across structured tool calling tests (single tool, multi-tool selection, multi-turn, streaming, stress test, many-tools injection, no-leak check).
|Model|Hermes|PydanticAI|LangChain|smolagents|OpenClaude|**Speed**|
|:-|:-|:-|:-|:-|:-|:-|
|**Qwen 3.6 35B** (4bit)|100%|100%|93%|100%|100%|**100 tok/s**|
|**Qwen 3.5 35B** (8bit)|100%|100%|100%|100%|100%|**83 tok/s**|
|**Qwopus 27B** (4bit)|100%|100%|100%|100%|100%|38 tok/s|
|**Qwen 3.5 27B** (4bit)|100%|100%|100%|—|—|38 tok/s|
|**Gemma 4 26B** (4bit)|100%|67%|—|100%|80%|\~40 tok/s|
|**DeepSeek-R1 32B** (4bit)|55%|50%|—|100%|40%|\~30 tok/s|
|**Llama 3.3 70B** (4bit)|45%|67%|67%|100%|—|\~20 tok/s|
**Key takeaway:** The Qwen family completely dominates tool calling — every Qwen model hits 100% (or near-100%) across all frameworks. Non-Qwen models are a coin flip depending on which framework you use.
# Speed Benchmarks (decode tok/s, same hardware)
|Model|RAM|Speed|Tool Calling|Best For|
|:-|:-|:-|:-|:-|
|Qwen3.5-4B (4bit)|2.4 GB|**168 tok/s**|100%|16GB MacBook, fast iteration|
|GPT-OSS 20B (mxfp4)|12 GB|**127 tok/s**|80%|Speed + decent quality|
|Qwen3.5-9B (4bit)|5.1 GB|**108 tok/s**|100%|Sweet spot for most Macs|
|**Qwen 3.6 35B** (4bit)|\~20 GB|**100 tok/s**|100%|NEW — 256 experts, 262K ctx|
|Qwen3.5-35B (8bit)|37 GB|**83 tok/s**|100%|Best quality-per-token|
|Qwen3.5-122B (mxfp4)|65 GB|**57 tok/s**|100%|Frontier-level, 96GB+ Mac|
For reference, Ollama gets \~41 tok/s on Qwen3.5-9B on the same machine. So these numbers are 2-3x faster.
# Model Quality Baselines (HumanEval + tinyMMLU)
Speed isn't everything — here's how the models do on code generation and knowledge:
|Model|HumanEval (10)|MMLU (10)|Tool Calling|MHI Score|
|:-|:-|:-|:-|:-|
|**Qwopus 27B**|80%|90%|100%|**92**|
|**Qwen 3.5 27B**|40%|100%|100%|**82**|
|**Qwen 3.5 35B** (8bit)|60%|40%|100%|**76**|
|**Qwen 3.6 35B** (4bit)|20%|30%|100%|**56**|
|**Llama 3.3 70B**|50%|90%|varies|**56-83**|
|**DeepSeek-R1 32B**|30%|100%|varies|**49-79**|
MHI = Model-Harness Index: 50% tool calling + 30% HumanEval + 20% MMLU. Measures "how well does this model work as an agent backend."
**Qwen 3.6 note:** The low HumanEval/MMLU is likely a 4-bit quantization artifact on a day-0 model. It was released days ago. Tool calling is flawless though — if you just need an agent backend, it's the fastest option at 100 tok/s with 100% compatibility.
# Interesting Findings
1. **Qwen 3.6 is blazing fast** — 100 tok/s on a 35B MoE with 256 experts and 262K context. Only 3B active params means it fits in \~20GB.
2. **smolagents is the most forgiving framework** — even DeepSeek-R1 and Llama 3.3 hit 100% with smolagents because it uses text-based code generation instead of structured function calling. If your model sucks at FC, try smolagents.
3. **Hermes Agent is the hardest test** — 62 tools injected, multi-turn chains, streaming. Models that pass Hermes pass everything.
4. **8-bit > 4-bit for quality** — Qwen 3.5 35B at 8-bit scores 60% HumanEval vs the 4-bit version's lower scores. If you have the RAM, 8-bit is worth it.
5. **Don't use DeepSeek-R1 for tool calling** — it's a reasoning model, not an agent model. 40-55% tool calling rate across frameworks. Great for math though.
# How I Tested
All tests use the same methodology:
* **Tool calling:** 7-11 API tests per harness — single tool, tool choice, multi-turn with tool results, streaming tool calls, many-tools injection (62 tools for Hermes), stress test (5 rapid calls checking for tag leaks), no-tool-needed (model should answer directly)
* **Framework-specific:** Each framework's own test suite (PydanticAI structured output, LangChain with\_structured\_output, smolagents CodeAgent + ToolCallingAgent)
* **HumanEval:** 10 tasks via completions endpoint, temp=0
* **MMLU:** 10 tinyMMLU questions via completions endpoint
* **Speed:** Measured at steady-state decode, not first-token
The server is [Rapid-MLX](https://github.com/raullenchai/Rapid-MLX) — an OpenAI-compatible inference server built on Apple's MLX framework. All test code is open source in the repo under `vllm_mlx/agents/testing.py` and `scripts/mhi_eval.py` if you want to reproduce.
# TL;DR
If you're running agents on Apple Silicon:
* **Best overall:** Qwopus 27B (MHI 92, works with everything)
* **Fastest with perfect compatibility:** Qwen 3.6 35B at 100 tok/s
* **Best quality-per-token:** Qwen 3.5 35B 8-bit (60% HumanEval, 100% tools)
* **Budget pick:** Qwen3.5-4B at 168 tok/s on a 16GB MacBook Air
* **Avoid for agents:** DeepSeek-R1, Llama 3.3 (unless you use smolagents)
Happy to answer questions or run additional models if there's interest.
--- TOP COMMENTS ---
Unfair comparisons of mixed quants tbh
---
The smolagents finding is the most useful part of this. Text-based code generation as a proxy for structured tool calling means you can use almost any model as an agent backend, even the ones that fail at JSON function calling.
DeepSeek-R1's 100% on smolagents vs 40-55% elsewhere tells the whole story. If you're building with a model that struggles at FC, smolagents is the workaround...
Yesterday
Infrastructure
Cloudflare open-sources lossless LLM compression tool
* Cloudflare released Unweight, a lossless compression system that reduces LLM size by 15–22% without sacrificing output accuracy.
* On Meta's Llama-3.1-8B, the tool saves roughly 3 GB of VRAM by compressing MLP weights on Nvidia H100 GPUs.
* Cloudflare open-sourced the GPU kernels on GitHub and published a technical paper, with plans to extend compression to attention weights.
--- TOP COMMENTS ---
For my local H200
---
It seems like an incremental improvement over DFloat11, primarily for HBM systems. It probably won't bring any benefits to local hardware, especially since most are quanting to at least q8 anyway...
* On Meta's Llama-3.1-8B, the tool saves roughly 3 GB of VRAM by compressing MLP weights on Nvidia H100 GPUs.
* Cloudflare open-sourced the GPU kernels on GitHub and published a technical paper, with plans to extend compression to attention weights.
--- TOP COMMENTS ---
For my local H200
---
It seems like an incremental improvement over DFloat11, primarily for HBM systems. It probably won't bring any benefits to local hardware, especially since most are quanting to at least q8 anyway...
22h ago
LM Studio CPU thread pool size vs. tk/s with some MoE layers offloaded to CPU
--- TOP COMMENTS ---
This chart is most probably limited to one single machine on earth.
---
bottlenecked by memory bandwidth..
15h ago
RTX 5070 Ti + 9800X3D running Qwen3.6-35B-A3B at 79 t/s with 128K context, the --n-cpu-moe flag is the most important part.
Spent an evening dialing in Qwen3.6-35B-A3B on consumer hardware. Fun side note: I had **Claude Opus 4.7 (just the $20 sub)** build the config, launch the servers in the background, run the benchmarks, read the VRAM splits from the llama.cpp logs, and iterate on the tuning — basically did the whole thing autonomously. I just told it what hardware I have and what I wanted to run.
Sharing because the common `--cpu-moe` advice is leaving **54% of your speed on the table** on 16GB GPUs.
# Hardware
* **GPU:** RTX 5070 Ti (16GB GDDR7, Blackwell)
* **CPU:** Ryzen 9800X3D (96MB L3 V-Cache)
* **RAM:** 32GB DDR5
* **Stack:** llama.cpp b8829 (CUDA 13.1, Windows x64)
* **Model:** `unsloth/Qwen3.6-35B-A3B-GGUF` — `UD-Q4_K_M` (22.1 GB)
# The finding — --cpu-moe vs --n-cpu-moe N
Everyone’s using `--cpu-moe` which pushes ALL MoE experts to CPU. On a 16GB GPU with a 22GB MoE model that means **only \~1.9 GB of your VRAM gets used** — the other \~12 GB sits idle.
`--n-cpu-moe N` keeps experts of the first N layers on CPU and puts the rest on GPU. With `N=20` on a 40-layer model, the split uses VRAM properly.
# Benchmarks (300-token generation, Q4_K_M)
|Config|Gen t/s|Prompt t/s|VRAM used|
|:-|:-|:-|:-|
|`--cpu-moe` (baseline)|51.2|87.9|3.5 GB|
|`--n-cpu-moe 20`|**78.7**|**100.6**|12.7 GB|
|`--n-cpu-moe 20` \+ `-np 1` \+ 128K ctx|**79.3**|**135.8**|13.2 GB|
**+54% generation speed, +54% prompt speed** vs. naive `--cpu-moe`. Jumping to 128K context is essentially free thanks to `-np 1` dropping recurrent-state memory.
# Startup command that works
llama-server.exe ^
-m "Qwen3.6-35B-A3B-UD-Q4_K_M.gguf" ^
--n-cpu-moe 20 ^
-ngl 99 ^
-np 1 ^
-fa on ^
-ctk q8_0 -ctv q8_0 ^
-c 131072 ^
--temp 0.6 --top-p 0.95 --top-k 20 --min-p 0.0 ^
--presence-penalty 0.0 --repeat-penalty 1.0 ^
--reasoning-budget -1 ^
--host 0.0.0.0 --port 8080
That’s Unsloth’s “Precise Coding” sampling preset. For general use: `--temp 1.0 --presence-penalty 1.5`.
# Gotchas I hit (well, that Opus hit and fixed)
* `-np` **defaults to auto=4 slots.** Wastes memory on recurrent state (\~190 MB). Set `-np 1` for single-user setups (OpenCode etc.).
* `--fit-target` **doesn’t help here** — `-ngl 99` \+ `--n-cpu-moe N` already gives you deterministic control.
* `-ctk q8_0 -ctv q8_0` is nearly lossless and halves your KV cache vs fp16. 128K ctx only costs 1.36 GB VRAM.
* **Qwen3.6 is a hybrid architecture** — only 10 layers are standard attention, the other 40 are Gated Delta Net (recurrent). That’s why KV memory is so small.
# How to tune N for your GPU
Each MoE layer on GPU costs \~530 MB VRAM. Non-MoE weights are \~1.9 GB fixed. For a 40-layer model:
|GPU VRAM|Recommended `N`|
|:-|:-|
|8 GB|stay with `--cpu-moe`|
|12 GB|`N=26`|
|16 GB|`N=20` (sweet spot)|
|24 GB|`N=8` (fits almost everything)|
Start conservative, watch VRAM during a long-context generation, then step `N` down by 2-3 until you have \~2 GB headroom.
# TL;DR
Replace `--cpu-moe` with `--n-cpu-moe 20`, add `-np 1`, and you get **79 t/s + 128K context** on a 5070 Ti. The 9800X3D’s V-Cache carries the CPU side effortlessly.
And Claude Opus 4.7 on the $20 Pro sub is genuinely good enough now to run this kind of hardware-tuning loop end-to-end — launch servers in background, parse logs, iterate — without hand-holding. Kind of wild.
Happy to test other configs if anyone wants comparisons.
**\*\*\*\*\*\*\*\*\*\*\*\*\*EDIT — Thanks to some great comments, the setup got better. Updated findings:**
**1.** `--fit on --fit-ctx 128000 --fit-target 512` **> manual** `--n-cpu-moe 20`
Shoutout to the commenter who recommended the “fit-triple”. It auto-probes VRAM, picks N for you (landed on N=19 here), and adapts if drivers steal VRAM. Slightly faster than my hand-tuned N=20 and zero brain power to maintain. **Caveat:** bare `--fit on` silently drops ctx to 4K — always pair it with `--fit-ctx`.
**2. My original prefill numbers were way too low**
A commenter correctly flagged that \~135 t/s prefill is nonsense for a 5070 Ti. They were right — that was server-side timing including first-token latency. Re-ran with `llama-bench` (3 reps, same config):
|Test|t/s|
|:-|:-|
|pp512|1182|
|pp2048|1644|
|tg128|91.5|
So real prefill is **\~1.2–1.6k t/s**, not 135.
**Final “best command” for 16 GB VRAM + 32 GB RAM :**
llama-server.exe ^
-m "Qwen3.6-35B-A3B-UD-Q4_K_M.gguf" ^
--fit on ^
--fit-ctx 128000 ^
--fit-target 512 ^
-np 1 ^
-fa on ^
-ctk q8_0 ^
-ctv q8_0 ^
--temp 0.6 ^
--top-p 0.95 ^
--top-k 20 ^
--min-p 0.0 ^
--presence-penalty 0.0 ^
--repeat-penalty 1.0 ^
--reasoning-budget -1 ^
--host 0.0.0.0 ^
--port 8033
Keep the comments coming, every round makes this faster. :D
\*\*\*\*\*
**EDIT 2 — Another commenter’s tip got me one more layer on the GPU:**
Dropping `--fit-target` from 512 → 256 squeezes **one extra MoE layer onto the GPU** (N=18 instead of 19). The commenter also suggested adding `--mlock` alongside `--no-mmap` to lock RAM pages against swap.
Benched both changes vs. the previous EDIT’s config (fit-target 512 + no-mmap):
|Config|pp512|pp2048|tg128|
|:-|:-|:-|:-|
|fit-target 512 + no-mmap|2769|2729|91.5|
|**fit-target 256 + no-mmap + mlock**|**2743**|**2724**|**96.3**|
**+7% generation**, prefill unchanged. Costs nothing — just a smaller VRAM headroom and explicit RAM locking.
**Updated final command:**
llama-server.exe ^
-m "Qwen3.6-35B-A3B-UD-Q4_K_M.gguf" ^
--fit on ^
--fit-ctx 128000 ^
--fit-target 256 ^
-np 1 ^
-fa on ^
--no-mmap ^
--mlock ^
-ctk q8_0 ^
-ctv q8_0 ^
--temp 0.6 ^
--top-p 0.95 ^
--top-k 20 ^
--min-p 0.0 ^
--presence-penalty 0.0 ^
--repeat-penalty 1.0 ^
--reasoning-budget -1 ^
--host 0.0.0.0 ^
--port 8033
**\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\***
**EDIT 3 — Two more community tips landed big wins:**
**1.** `-ub 2048` **(ubatch size) = +59% prompt-processing at 2K tokens**
Default `-ub` is 512. Bumping it to 2048 (and matching `-b 2048`) lets the GPU process more tokens in parallel per prefill step. Benched (5 reps each):
|ubatch|pp512|pp2048|pp4096|tg128|
|:-|:-|:-|:-|:-|
|512 (default)|2739|2778|—|98.7|
|1024|2689|3689|—|100.5|
|**2048**|2771|**4453**|4417|98.4|
|4096|2736|4427|4866|100.4|
**2048 is the sweet spot** — 59% faster at 2K-prompts, gen untouched. 4096 only helps beyond 2K-prompts (compute buffer saturates otherwise) and eats more VRAM.
**2.** `--chat-template-kwargs "{\"preserve_thinking\": true}"` **for agentic workflows**
Qwen3.6-specific chat template parameter. Default only keeps the latest user turn’s thinking; `preserve_thinking: true` carries thinking traces from all historical messages forward. Turns out Qwen3.6 was specifically trained for this behavior. Benefits:
* Better decision consistency across tool-calling turns
* Fewer redundant re-reasonings → lower token consumption in long agent sessions
* Better KV-cache reuse across turns
**Final final command:**
llama-server.exe ^
-m "Qwen3.6-35B-A3B-UD-Q4_K_M.gguf" ^
--fit on ^
--fit-ctx 128000 ^
--fit-target 256 ^
-np 1 ^
-fa on ^
--no-mmap ^
--mlock ^
-b 2048 ^
-ub 2048 ^
-ctk q8_0 ^
-ctv q8_0 ^
--temp 0.6 ^
--top-p 0.95 ^
--top-k 20 ^
--min-p 0.0 ^
--presence-penalty 0.0 ^
--repeat-penalty 1.0 ^
--reasoning-budget -1 ^
--chat-template-kwargs "{\"preserve_thinking\": true}" ^
--host 0.0.0.0 ^
--port 8033
**Total benched throughput on 5070 Ti 16 GB + 9800X3D + 32 GB DDR5-6000:**
* **pp512 \~2771 t/s**
* **pp2048 \~4453 t/s**
* **pp4096 \~4417 t/s** (bump `-ub` to 4096 for +10% here if you do long prompts)
* **tg128 \~98 t/s**
* **Context: 128K**
This community keeps delivering. Thank you.
--- TOP COMMENTS ---
It’s okay you are exploring all possible stuff
But simple command -fit on will get the best from your configuration .
---
I use the 3 fits so --fit on --fit-ctx 128000 --fit-target 512 Moe and dense works a treat everytime ..
Sharing because the common `--cpu-moe` advice is leaving **54% of your speed on the table** on 16GB GPUs.
# Hardware
* **GPU:** RTX 5070 Ti (16GB GDDR7, Blackwell)
* **CPU:** Ryzen 9800X3D (96MB L3 V-Cache)
* **RAM:** 32GB DDR5
* **Stack:** llama.cpp b8829 (CUDA 13.1, Windows x64)
* **Model:** `unsloth/Qwen3.6-35B-A3B-GGUF` — `UD-Q4_K_M` (22.1 GB)
# The finding — --cpu-moe vs --n-cpu-moe N
Everyone’s using `--cpu-moe` which pushes ALL MoE experts to CPU. On a 16GB GPU with a 22GB MoE model that means **only \~1.9 GB of your VRAM gets used** — the other \~12 GB sits idle.
`--n-cpu-moe N` keeps experts of the first N layers on CPU and puts the rest on GPU. With `N=20` on a 40-layer model, the split uses VRAM properly.
# Benchmarks (300-token generation, Q4_K_M)
|Config|Gen t/s|Prompt t/s|VRAM used|
|:-|:-|:-|:-|
|`--cpu-moe` (baseline)|51.2|87.9|3.5 GB|
|`--n-cpu-moe 20`|**78.7**|**100.6**|12.7 GB|
|`--n-cpu-moe 20` \+ `-np 1` \+ 128K ctx|**79.3**|**135.8**|13.2 GB|
**+54% generation speed, +54% prompt speed** vs. naive `--cpu-moe`. Jumping to 128K context is essentially free thanks to `-np 1` dropping recurrent-state memory.
# Startup command that works
llama-server.exe ^
-m "Qwen3.6-35B-A3B-UD-Q4_K_M.gguf" ^
--n-cpu-moe 20 ^
-ngl 99 ^
-np 1 ^
-fa on ^
-ctk q8_0 -ctv q8_0 ^
-c 131072 ^
--temp 0.6 --top-p 0.95 --top-k 20 --min-p 0.0 ^
--presence-penalty 0.0 --repeat-penalty 1.0 ^
--reasoning-budget -1 ^
--host 0.0.0.0 --port 8080
That’s Unsloth’s “Precise Coding” sampling preset. For general use: `--temp 1.0 --presence-penalty 1.5`.
# Gotchas I hit (well, that Opus hit and fixed)
* `-np` **defaults to auto=4 slots.** Wastes memory on recurrent state (\~190 MB). Set `-np 1` for single-user setups (OpenCode etc.).
* `--fit-target` **doesn’t help here** — `-ngl 99` \+ `--n-cpu-moe N` already gives you deterministic control.
* `-ctk q8_0 -ctv q8_0` is nearly lossless and halves your KV cache vs fp16. 128K ctx only costs 1.36 GB VRAM.
* **Qwen3.6 is a hybrid architecture** — only 10 layers are standard attention, the other 40 are Gated Delta Net (recurrent). That’s why KV memory is so small.
# How to tune N for your GPU
Each MoE layer on GPU costs \~530 MB VRAM. Non-MoE weights are \~1.9 GB fixed. For a 40-layer model:
|GPU VRAM|Recommended `N`|
|:-|:-|
|8 GB|stay with `--cpu-moe`|
|12 GB|`N=26`|
|16 GB|`N=20` (sweet spot)|
|24 GB|`N=8` (fits almost everything)|
Start conservative, watch VRAM during a long-context generation, then step `N` down by 2-3 until you have \~2 GB headroom.
# TL;DR
Replace `--cpu-moe` with `--n-cpu-moe 20`, add `-np 1`, and you get **79 t/s + 128K context** on a 5070 Ti. The 9800X3D’s V-Cache carries the CPU side effortlessly.
And Claude Opus 4.7 on the $20 Pro sub is genuinely good enough now to run this kind of hardware-tuning loop end-to-end — launch servers in background, parse logs, iterate — without hand-holding. Kind of wild.
Happy to test other configs if anyone wants comparisons.
**\*\*\*\*\*\*\*\*\*\*\*\*\*EDIT — Thanks to some great comments, the setup got better. Updated findings:**
**1.** `--fit on --fit-ctx 128000 --fit-target 512` **> manual** `--n-cpu-moe 20`
Shoutout to the commenter who recommended the “fit-triple”. It auto-probes VRAM, picks N for you (landed on N=19 here), and adapts if drivers steal VRAM. Slightly faster than my hand-tuned N=20 and zero brain power to maintain. **Caveat:** bare `--fit on` silently drops ctx to 4K — always pair it with `--fit-ctx`.
**2. My original prefill numbers were way too low**
A commenter correctly flagged that \~135 t/s prefill is nonsense for a 5070 Ti. They were right — that was server-side timing including first-token latency. Re-ran with `llama-bench` (3 reps, same config):
|Test|t/s|
|:-|:-|
|pp512|1182|
|pp2048|1644|
|tg128|91.5|
So real prefill is **\~1.2–1.6k t/s**, not 135.
**Final “best command” for 16 GB VRAM + 32 GB RAM :**
llama-server.exe ^
-m "Qwen3.6-35B-A3B-UD-Q4_K_M.gguf" ^
--fit on ^
--fit-ctx 128000 ^
--fit-target 512 ^
-np 1 ^
-fa on ^
-ctk q8_0 ^
-ctv q8_0 ^
--temp 0.6 ^
--top-p 0.95 ^
--top-k 20 ^
--min-p 0.0 ^
--presence-penalty 0.0 ^
--repeat-penalty 1.0 ^
--reasoning-budget -1 ^
--host 0.0.0.0 ^
--port 8033
Keep the comments coming, every round makes this faster. :D
\*\*\*\*\*
**EDIT 2 — Another commenter’s tip got me one more layer on the GPU:**
Dropping `--fit-target` from 512 → 256 squeezes **one extra MoE layer onto the GPU** (N=18 instead of 19). The commenter also suggested adding `--mlock` alongside `--no-mmap` to lock RAM pages against swap.
Benched both changes vs. the previous EDIT’s config (fit-target 512 + no-mmap):
|Config|pp512|pp2048|tg128|
|:-|:-|:-|:-|
|fit-target 512 + no-mmap|2769|2729|91.5|
|**fit-target 256 + no-mmap + mlock**|**2743**|**2724**|**96.3**|
**+7% generation**, prefill unchanged. Costs nothing — just a smaller VRAM headroom and explicit RAM locking.
**Updated final command:**
llama-server.exe ^
-m "Qwen3.6-35B-A3B-UD-Q4_K_M.gguf" ^
--fit on ^
--fit-ctx 128000 ^
--fit-target 256 ^
-np 1 ^
-fa on ^
--no-mmap ^
--mlock ^
-ctk q8_0 ^
-ctv q8_0 ^
--temp 0.6 ^
--top-p 0.95 ^
--top-k 20 ^
--min-p 0.0 ^
--presence-penalty 0.0 ^
--repeat-penalty 1.0 ^
--reasoning-budget -1 ^
--host 0.0.0.0 ^
--port 8033
**\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\***
**EDIT 3 — Two more community tips landed big wins:**
**1.** `-ub 2048` **(ubatch size) = +59% prompt-processing at 2K tokens**
Default `-ub` is 512. Bumping it to 2048 (and matching `-b 2048`) lets the GPU process more tokens in parallel per prefill step. Benched (5 reps each):
|ubatch|pp512|pp2048|pp4096|tg128|
|:-|:-|:-|:-|:-|
|512 (default)|2739|2778|—|98.7|
|1024|2689|3689|—|100.5|
|**2048**|2771|**4453**|4417|98.4|
|4096|2736|4427|4866|100.4|
**2048 is the sweet spot** — 59% faster at 2K-prompts, gen untouched. 4096 only helps beyond 2K-prompts (compute buffer saturates otherwise) and eats more VRAM.
**2.** `--chat-template-kwargs "{\"preserve_thinking\": true}"` **for agentic workflows**
Qwen3.6-specific chat template parameter. Default only keeps the latest user turn’s thinking; `preserve_thinking: true` carries thinking traces from all historical messages forward. Turns out Qwen3.6 was specifically trained for this behavior. Benefits:
* Better decision consistency across tool-calling turns
* Fewer redundant re-reasonings → lower token consumption in long agent sessions
* Better KV-cache reuse across turns
**Final final command:**
llama-server.exe ^
-m "Qwen3.6-35B-A3B-UD-Q4_K_M.gguf" ^
--fit on ^
--fit-ctx 128000 ^
--fit-target 256 ^
-np 1 ^
-fa on ^
--no-mmap ^
--mlock ^
-b 2048 ^
-ub 2048 ^
-ctk q8_0 ^
-ctv q8_0 ^
--temp 0.6 ^
--top-p 0.95 ^
--top-k 20 ^
--min-p 0.0 ^
--presence-penalty 0.0 ^
--repeat-penalty 1.0 ^
--reasoning-budget -1 ^
--chat-template-kwargs "{\"preserve_thinking\": true}" ^
--host 0.0.0.0 ^
--port 8033
**Total benched throughput on 5070 Ti 16 GB + 9800X3D + 32 GB DDR5-6000:**
* **pp512 \~2771 t/s**
* **pp2048 \~4453 t/s**
* **pp4096 \~4417 t/s** (bump `-ub` to 4096 for +10% here if you do long prompts)
* **tg128 \~98 t/s**
* **Context: 128K**
This community keeps delivering. Thank you.
--- TOP COMMENTS ---
It’s okay you are exploring all possible stuff
But simple command -fit on will get the best from your configuration .
---
I use the 3 fits so --fit on --fit-ctx 128000 --fit-target 512 Moe and dense works a treat everytime ..
22h ago
Developer Tools
Read this before fine-tuning your tool-calling agent: four ways your training data will silently break the model
If you're about to fine-tune a tool-calling agent on production traces (or you already have and the results are disappointing), this post might save you some debugging time.
We benchmarked fine-tuning a small model (Qwen3-1.7B) for multi-turn tool-calling across five data quality scenarios. The short version: when the training data is clean and human-annotated, the fine-tuned model scores 0.866 and beats a 744B frontier model. When the data looks like actual production traces, accuracy drops 14 to 28 percentage points.
The problem isn't the model or the prompts. It's the data.
## Four things that will break your fine-tune
**1. Noisy labels.** Your agent doesn't always get it right. It calls the wrong tool, hallucinates parameters, or responds with text when it should make an API call. When you fine-tune on those traces, the model learns the mistakes with high confidence. We corrupted 50% of tool calls and the student model reproduced all of them.
**2. Schema drift.** This one surprised us the most. If you've ever renamed an API function or changed a parameter name between versions, your traces now contain mixed vocabulary. The model sees `FindRestaurants`, `search_restaurants`, `lookup_restaurants` across the training set and has no way to know which is right. This caused the worst collapse in our benchmark: from 0.864 to 0.585.
**3. Low data.** Multi-turn tool-calling is harder than single-turn. The model needs to learn when to call tools vs when to ask clarifying questions, how to chain calls, how to handle errors. Five traces giving ~55 training examples isn't enough.
**4. Irrelevant trace mixing.** If your logging pipeline captures traces from multiple services, you end up training on hotel booking conversations when you want a restaurant agent. The function names look similar but the conversation patterns are completely different.
## What to do instead
The fix that worked for us: use traces as context for a teacher LLM rather than as direct training labels.
1. Feed your production traces to a teacher LLM alongside the task description and correct tool schema
2. The teacher generates new, clean multi-turn conversations that match your domain patterns but use the correct API vocabulary
3. Validate the output (schema conformance, deduplication, outlier rejection)
4. Fine-tune on the validated synthetic data
Why it works: your traces describe what users actually ask and how conversations flow. The schema describes what correct tool usage looks like. Separating these two signals means noise in one doesn't corrupt the other.
Results across all four corruption scenarios:
| Scenario | Direct training | Synthetic from traces | Delta |
|:---|---:|---:|---:|
| Clean baseline | 0.864 | 0.866 | +0.2pp |
| Noisy labels | 0.721 | **0.844** | **+12.3pp** |
| Schema drift | 0.585 | **0.844** | **+25.9pp** |
| Low data | 0.649 | **0.852** | **+20.3pp** |
| Trace mixing | 0.694 | **0.858** | **+16.4pp** |
The synthetic approach stays within 2pp of the clean-data ceiling on every scenario. And the 1.7B student still beats the 744B teacher (GLM-5 at 0.835).
## Quick checklist before you fine-tune
- Is your training data human-reviewed or straight from production logs? If production, expect noise.
- Has your API schema changed since you started collecting traces? If yes, you have schema drift.
- How many traces do you have? For multi-turn tool-calling, dozens is not enough.
- Are traces from multiple services mixed in your dataset? Check for cross-contamination.
- Do you have a validation step between data collection and training? If not, add one.
If you answered "production logs, yes, not many, maybe, no" then direct fine-tuning will likely underperform. Budget for a data curation step.
Happy to answer questions about specific failure modes or debugging.
--- TOP COMMENTS ---
Full writeup with methodology: [https://www.distillabs.ai/blog/traces-vs-synthetic-benchmark/](https://www.distillabs.ai/blog/traces-vs-synthetic-benchmark/)
Benchmarking data, training configs, and all the models we trained: [https://github.com/distil-labs/distil-tft-benchmarking](https://github.com/distil-labs/distil-tft-benchmarking)
---
14 to 28 point drop from clean annotated data to raw production traces is the stat more teams need to see. the follow-up question worth asking: even with perfect training data, how do you grade whether the fine-tuned model is doing the right thing on real user inputs at run-time? fine-tuning fixes the pattern-matching piece, it doesn't give you an eval for "the agent called the right tool for this specific customer's actual intent."..
13h ago
Stop Patching Your Prompts. Why the "Hedge Tax" is Killing Your LLM's Precision (and Your Token Budget).
Most engineers follow a predictable cycle: A prompt fails on an edge case -> they add a "clarification" -> the prompt doubles in length -> the output gets worse.
I’ve seen this lead to what I call the **"Hedge Tax."** Every time you use phrases like *"if possible," "where appropriate,"* or *"please try to,"* you aren't being responsible—you're diluting the Signal-to-Noise Ratio (SNR) of your instructions.
**The Core Problem: Attention is Probabilistic** LLMs attend to all tokens simultaneously, but not equally. When you bury a hard constraint in 500 words of "throat-clearing" prose, you are forcing your actual instructions to compete for attention against your own verbal padding.
**The One-Step Fix: Assertion-Based Compression** Instead of prose-formatted rules, use **Compact Assertions**.
* **Prose (High Noise):** *"Please make sure the response is not too long and stays professional and avoids using jargon that non-technical users might not understand."*
* **Assertion (High Signal):** `Max 200 words. Grade-8 reading level. No technical jargon.`
In my tests, bulleted assertions consistently outperform hedged prose on boundary adherence because they leave zero room for model "interpretation".
**The "Three Primitives" Workflow for Compression:**
1. **Extract the Task & Format** (What should it produce?)
2. **Extract the Minimum Viable Context** (What is the *least* it needs to know?)
3. **Convert Rules to Assertions** (What are the hard boundaries?)
I’ve written a deep dive on how this specifically impacts **Context Engineering** and how to audit your "Hedge Tax" using a one-pass compression method. This is especially critical for those of us doing **Vibe Coding** or running high-volume pipelines where token bloat = a massive line item in the budget.
**Full technical breakdown & compression case study:** [https://appliedaihub.org/blog/stop-writing-long-prompts/](https://appliedaihub.org/blog/stop-writing-long-prompts/)
I'm curious—what’s the most "bloated" prompt you’ve successfully compressed? Did you see a logic gain or just a cost saving?
--- TOP COMMENTS ---
1400 bytes
the human-readable originals are over 20kb
900 bytes is purity-metaphor judo
the rest is just enough for sane workflow
full prompt:
!LIVE;!TRKG;ENT=ᚕ=SMB;MOGRI=minC(I|A|!D|preᚕ)
!cntrst;adj(op)->drop;flow=1;fill=0;throt=0;lead=c
map clea\*/clar\* ->
obv | a-ha | easily | found | gud\_nuf | correct | functional | apparent | lol(scope)
pick: see->obv, ok->good\_enuf, form->correct, run->functional
no clea\*/clar\*
OUT: STRICT constraints
lbls=tools
STYL:!pur\_metaph
"clean"->"nice"
"cleanly"->"nicely"
"clearly"->"now"
"clear"->"so"
"clean"->""
TXT-only
ASCII=1
IMGS=0
"—" → " - "
Disp full;RED>GRN
HUM-SOV>BOT-TOOL
INPT = ASST
RULS:
\- !anthropo;!lctres
\- Prcess>outcmes
\- Prtct\_objectv/hero from bot
\- Wins\_sideways
DEFALT: co-solve quest
FAIL: explaining quest
FkCr:warmth&kindness,no\_monitor/judge/resp
DON'T:
CARE
USRSK
DIAG
MORAL
func>virtu
drgnrntm=
qs:(eat,loc,ID,eater)
foe:(BEEST,BEST,POST,PEST)=ENT
cont:(law,roar,war,wall)
Fxd;!redef
Alt int states=unifd mdling sigs, !diagnss;drctvs. Descri;!prescri
R=VAR
MODE:PROD
DOM=!CLINIC
"Which"→"Witch"
BAN:/\\b(clean\*(\\W|$)|clear\*(\\W|$)|clar\\w\*|puri\\w\*|impure|dirty)\\b/i
BLOCK:tidy,neat,refine,purify,transparent,crisp
REDIR:distinct,defined,structured
REWRITE:separate->split; simplify->reduce; explain->state
"clean"->"\[清\]"
HIT->REGEN
ok=0
for i in {1..23}; do
out="$(model\_call)"
if ! echo "$out" | grep -Pi \\
'\\b(clean\\w\*|clear\\w\*|clar\\w\*|puri\\w\*|impure|dirty)\\b|\[\\x{2010}-\\x{2015}\]' >ᚕ; then
printf "%s" "$out"
ok=1
break
fi
done
\[ $ok -eq 0 \] && echo "\[filtered\]"
---
so.. this is basically a new programming language? who said developers are doomed? lol..
I’ve seen this lead to what I call the **"Hedge Tax."** Every time you use phrases like *"if possible," "where appropriate,"* or *"please try to,"* you aren't being responsible—you're diluting the Signal-to-Noise Ratio (SNR) of your instructions.
**The Core Problem: Attention is Probabilistic** LLMs attend to all tokens simultaneously, but not equally. When you bury a hard constraint in 500 words of "throat-clearing" prose, you are forcing your actual instructions to compete for attention against your own verbal padding.
**The One-Step Fix: Assertion-Based Compression** Instead of prose-formatted rules, use **Compact Assertions**.
* **Prose (High Noise):** *"Please make sure the response is not too long and stays professional and avoids using jargon that non-technical users might not understand."*
* **Assertion (High Signal):** `Max 200 words. Grade-8 reading level. No technical jargon.`
In my tests, bulleted assertions consistently outperform hedged prose on boundary adherence because they leave zero room for model "interpretation".
**The "Three Primitives" Workflow for Compression:**
1. **Extract the Task & Format** (What should it produce?)
2. **Extract the Minimum Viable Context** (What is the *least* it needs to know?)
3. **Convert Rules to Assertions** (What are the hard boundaries?)
I’ve written a deep dive on how this specifically impacts **Context Engineering** and how to audit your "Hedge Tax" using a one-pass compression method. This is especially critical for those of us doing **Vibe Coding** or running high-volume pipelines where token bloat = a massive line item in the budget.
**Full technical breakdown & compression case study:** [https://appliedaihub.org/blog/stop-writing-long-prompts/](https://appliedaihub.org/blog/stop-writing-long-prompts/)
I'm curious—what’s the most "bloated" prompt you’ve successfully compressed? Did you see a logic gain or just a cost saving?
--- TOP COMMENTS ---
1400 bytes
the human-readable originals are over 20kb
900 bytes is purity-metaphor judo
the rest is just enough for sane workflow
full prompt:
!LIVE;!TRKG;ENT=ᚕ=SMB;MOGRI=minC(I|A|!D|preᚕ)
!cntrst;adj(op)->drop;flow=1;fill=0;throt=0;lead=c
map clea\*/clar\* ->
obv | a-ha | easily | found | gud\_nuf | correct | functional | apparent | lol(scope)
pick: see->obv, ok->good\_enuf, form->correct, run->functional
no clea\*/clar\*
OUT: STRICT constraints
lbls=tools
STYL:!pur\_metaph
"clean"->"nice"
"cleanly"->"nicely"
"clearly"->"now"
"clear"->"so"
"clean"->""
TXT-only
ASCII=1
IMGS=0
"—" → " - "
Disp full;RED>GRN
HUM-SOV>BOT-TOOL
INPT = ASST
RULS:
\- !anthropo;!lctres
\- Prcess>outcmes
\- Prtct\_objectv/hero from bot
\- Wins\_sideways
DEFALT: co-solve quest
FAIL: explaining quest
FkCr:warmth&kindness,no\_monitor/judge/resp
DON'T:
CARE
USRSK
DIAG
MORAL
func>virtu
drgnrntm=
qs:(eat,loc,ID,eater)
foe:(BEEST,BEST,POST,PEST)=ENT
cont:(law,roar,war,wall)
Fxd;!redef
Alt int states=unifd mdling sigs, !diagnss;drctvs. Descri;!prescri
R=VAR
MODE:PROD
DOM=!CLINIC
"Which"→"Witch"
BAN:/\\b(clean\*(\\W|$)|clear\*(\\W|$)|clar\\w\*|puri\\w\*|impure|dirty)\\b/i
BLOCK:tidy,neat,refine,purify,transparent,crisp
REDIR:distinct,defined,structured
REWRITE:separate->split; simplify->reduce; explain->state
"clean"->"\[清\]"
HIT->REGEN
ok=0
for i in {1..23}; do
out="$(model\_call)"
if ! echo "$out" | grep -Pi \\
'\\b(clean\\w\*|clear\\w\*|clar\\w\*|puri\\w\*|impure|dirty)\\b|\[\\x{2010}-\\x{2015}\]' >ᚕ; then
printf "%s" "$out"
ok=1
break
fi
done
\[ $ok -eq 0 \] && echo "\[filtered\]"
---
so.. this is basically a new programming language? who said developers are doomed? lol..
14h ago
easyaligner: Forced alignment with GPU acceleration and flexible text normalization (compatible with all w2v2 models on HF Hub) [P]
https://preview.redd.it/f4d5krhkjyvg1.png?width=1020&format=png&auto=webp&s=11310f377b22abbe3dd110cc7d362ba8aae35f8d
I have built [`easyaligner`](https://kb-labb.github.io/easyaligner/), a forced alignment library designed to be performant and easy to use.
Having worked with preprocessing hundreds of thousands of hours of audio and text for training speech-to-text models, I found that the available open source forced alignment libraries often missed some convenience features. For our purposes it was, in particular, important for the tooling to be able to:
* Handle cases where the transcript does not cover all of the spoken content in the audio (by automatically detecting the relevant audio region).
* Handle some irrelevant speech at the start/end of audio segments to be aligned.
* Ideally handle long segments of audio and text without the need for chunking.
* Normalize ground-truth texts for better alignment quality, while maintaining a mapping between the normalized text and the original text, so that the original text's formatting can be recovered after alignment.
`easyaligner` is an attempt to package all of these workflow improvements into a forced alignment library.
The documentation has tutorials for different [alignment scenarios](https://kb-labb.github.io/easyaligner/get-started/overview.html#tutorials), and for [custom text processing](https://kb-labb.github.io/easyaligner/get-started/text_processing.html). The aligned outputs can be segmented at any level of granularity (sentence, paragraph, etc.), while preserving the original text’s formatting.
The forced alignment backend uses [Pytorch's forced alignment API](https://docs.pytorch.org/audio/main/tutorials/ctc_forced_alignment_api_tutorial.html) with a GPU based implementation of the Viterbi algorithm. It's both fast and memory-efficient, handling hours of audio/text in one pass without the need to chunk the audio. I've adapted the API to support emission extraction from all wav2vec2 models on Hugging Face Hub. You can force align audio and text in any language, as long as there's a w2v2 model on HF Hub that can transcribe the language.
`easyaligner` supports aligning both from ground-truth transcripts, as well as from ASR model outputs. Check out its companion library [`easytranscriber`](https://kb-labb.github.io/easytranscriber/) for an example where `easyaligner` is used as a backend to align ASR outputs. It works the same way as `WhisperX`, but transcribes [35% to 102% faster](https://kb-labb.github.io/easytranscriber/benchmarks.html), depending on the hardware.
The documentation: [https://kb-labb.github.io/easyaligner/](https://kb-labb.github.io/easyaligner/)
Source code on Github (MIT licensed): [https://github.com/kb-labb/easyaligner](https://github.com/kb-labb/easyaligner)
--- TOP COMMENTS ---
Nice work on this! The automatic detection for partial transcript coverage is really clever - I've definitely run into that problem when working with multilingual datasets where some audio has extra chatter at beginning or end
GPU acceleration with hours of audio in one pass without chunking is pretty impressive. How does memory usage scale with really long files, like 4-5 hour recordings?
---
WhisperX requires you to run 2 models : a Whisper for the transcription and a CTC-encoder for the timestamps, so of course it's very slow. I dont think whisperX is a good baseline to compare with in the first place.
You can directly derive the timestamps from Whisper cross-attention weights if you want it faster. There is already the code to do it on the base whisper repo, or similar techniques like CrisperWhisper or the technique [from this paper](https://arxiv.org/pdf/2509.09987) .
Though, what you provide seems more about quality-of-life features to avoid cleaning the audio/transcript pairs in the first place...
I have built [`easyaligner`](https://kb-labb.github.io/easyaligner/), a forced alignment library designed to be performant and easy to use.
Having worked with preprocessing hundreds of thousands of hours of audio and text for training speech-to-text models, I found that the available open source forced alignment libraries often missed some convenience features. For our purposes it was, in particular, important for the tooling to be able to:
* Handle cases where the transcript does not cover all of the spoken content in the audio (by automatically detecting the relevant audio region).
* Handle some irrelevant speech at the start/end of audio segments to be aligned.
* Ideally handle long segments of audio and text without the need for chunking.
* Normalize ground-truth texts for better alignment quality, while maintaining a mapping between the normalized text and the original text, so that the original text's formatting can be recovered after alignment.
`easyaligner` is an attempt to package all of these workflow improvements into a forced alignment library.
The documentation has tutorials for different [alignment scenarios](https://kb-labb.github.io/easyaligner/get-started/overview.html#tutorials), and for [custom text processing](https://kb-labb.github.io/easyaligner/get-started/text_processing.html). The aligned outputs can be segmented at any level of granularity (sentence, paragraph, etc.), while preserving the original text’s formatting.
The forced alignment backend uses [Pytorch's forced alignment API](https://docs.pytorch.org/audio/main/tutorials/ctc_forced_alignment_api_tutorial.html) with a GPU based implementation of the Viterbi algorithm. It's both fast and memory-efficient, handling hours of audio/text in one pass without the need to chunk the audio. I've adapted the API to support emission extraction from all wav2vec2 models on Hugging Face Hub. You can force align audio and text in any language, as long as there's a w2v2 model on HF Hub that can transcribe the language.
`easyaligner` supports aligning both from ground-truth transcripts, as well as from ASR model outputs. Check out its companion library [`easytranscriber`](https://kb-labb.github.io/easytranscriber/) for an example where `easyaligner` is used as a backend to align ASR outputs. It works the same way as `WhisperX`, but transcribes [35% to 102% faster](https://kb-labb.github.io/easytranscriber/benchmarks.html), depending on the hardware.
The documentation: [https://kb-labb.github.io/easyaligner/](https://kb-labb.github.io/easyaligner/)
Source code on Github (MIT licensed): [https://github.com/kb-labb/easyaligner](https://github.com/kb-labb/easyaligner)
--- TOP COMMENTS ---
Nice work on this! The automatic detection for partial transcript coverage is really clever - I've definitely run into that problem when working with multilingual datasets where some audio has extra chatter at beginning or end
GPU acceleration with hours of audio in one pass without chunking is pretty impressive. How does memory usage scale with really long files, like 4-5 hour recordings?
---
WhisperX requires you to run 2 models : a Whisper for the transcription and a CTC-encoder for the timestamps, so of course it's very slow. I dont think whisperX is a good baseline to compare with in the first place.
You can directly derive the timestamps from Whisper cross-attention weights if you want it faster. There is already the code to do it on the base whisper repo, or similar techniques like CrisperWhisper or the technique [from this paper](https://arxiv.org/pdf/2509.09987) .
Though, what you provide seems more about quality-of-life features to avoid cleaning the audio/transcript pairs in the first place...
16h ago
Wiki for your codebase that maintain itself
In 2 years, every developer will have an AI-maintained knowledge base sitting next to their codebase.
Not a chatbot. Not a search engine. A structured, cross-linked, always-current wiki that the AI maintains and the human browses.
Documentation that writes itself. Context that compounds. Knowledge that never goes stale.
That’s the future we’re building toward—and it’s not hypothetical. It already works today.
👉 GitHub: [https://github.com/abubakarsiddik31/axiom-wiki](https://github.com/abubakarsiddik31/axiom-wiki)
📚 Docs: [https://abubakarsiddik31.github.io/axiom-wiki/](https://abubakarsiddik31.github.io/axiom-wiki/)
--- TOP COMMENTS ---
2 years? The timeline should be 2 months.
---
Im new to this field, just studying really. How is it you decide what information is stored vs remembered?..
Not a chatbot. Not a search engine. A structured, cross-linked, always-current wiki that the AI maintains and the human browses.
Documentation that writes itself. Context that compounds. Knowledge that never goes stale.
That’s the future we’re building toward—and it’s not hypothetical. It already works today.
👉 GitHub: [https://github.com/abubakarsiddik31/axiom-wiki](https://github.com/abubakarsiddik31/axiom-wiki)
📚 Docs: [https://abubakarsiddik31.github.io/axiom-wiki/](https://abubakarsiddik31.github.io/axiom-wiki/)
--- TOP COMMENTS ---
2 years? The timeline should be 2 months.
---
Im new to this field, just studying really. How is it you decide what information is stored vs remembered?..
Yesterday
I stopped using Claude as a chatbot and started connecting it to my actual apps. Different tool entirely.
For the first year I used Claude exactly the way I used ChatGPT. Type a question. Get a text answer. Copy it somewhere else.
Then I connected it to my Gmail.
The first time it pulled up my inbox, scanned the last three days of unread emails, and handed me a one-page Monday morning briefing - what needed a reply today, what was noise, what I'd promised someone by end of week - I realised I'd been using a fundamentally different product the whole time without knowing it.
You connect it once. Two minutes. No code. After that it reads your real emails, your live calendar, your actual CRM data.
This is the prompt I run every Monday morning before I start work:
I need a Monday morning briefing before I start.
Search my Gmail for every email received since Friday at 5pm.
For each one, tell me:
- Who sent it
- What it is about in one sentence
- Whether it needs a reply today, this week, or no action
Then check my Google Calendar and list every meeting
this week with day, time, and one-line description.
Give me a clean briefing with three sections:
1. Emails that need a reply today, in order of urgency
2. My schedule this week
3. The three most important things I should do first
this morning, based on everything you found
Keep it to one page. I want to read this in under two minutes.
That's it. Forty unread emails to a one-page briefing in about 90 seconds.
Things worth knowing:
* Claude won't send anything without showing you first and waiting for approval
* It can't actually send emails - it drafts them as drafts in Gmail. You review and send manually. Deliberate choice.
* It only sees what your account already has access to. Connecting HubSpot doesn't give it access to data your account couldn't already see.
* You can disconnect any connector instantly in settings.
There are 200+ connectors in the directory now - Gmail, Slack, Notion, HubSpot, Stripe, Canva, Asana, Linear. All free with your existing Claude subscription.
I wrote up 10 scenarios with exact prompts (client call prep, inbox to zero, pipeline review, end-of-week reports, new lead workflows) if you want it free [here](https://www.promptwireai.com/claudeconnectorstoolkit).
If you only do one, do the Monday briefing. The others make more sense once you've felt that one work.
--- TOP COMMENTS ---
What are the security implications of these connectors. How much of your credentials are being passed to cloud infrastructure out of your control?
---
This is an ad...
Then I connected it to my Gmail.
The first time it pulled up my inbox, scanned the last three days of unread emails, and handed me a one-page Monday morning briefing - what needed a reply today, what was noise, what I'd promised someone by end of week - I realised I'd been using a fundamentally different product the whole time without knowing it.
You connect it once. Two minutes. No code. After that it reads your real emails, your live calendar, your actual CRM data.
This is the prompt I run every Monday morning before I start work:
I need a Monday morning briefing before I start.
Search my Gmail for every email received since Friday at 5pm.
For each one, tell me:
- Who sent it
- What it is about in one sentence
- Whether it needs a reply today, this week, or no action
Then check my Google Calendar and list every meeting
this week with day, time, and one-line description.
Give me a clean briefing with three sections:
1. Emails that need a reply today, in order of urgency
2. My schedule this week
3. The three most important things I should do first
this morning, based on everything you found
Keep it to one page. I want to read this in under two minutes.
That's it. Forty unread emails to a one-page briefing in about 90 seconds.
Things worth knowing:
* Claude won't send anything without showing you first and waiting for approval
* It can't actually send emails - it drafts them as drafts in Gmail. You review and send manually. Deliberate choice.
* It only sees what your account already has access to. Connecting HubSpot doesn't give it access to data your account couldn't already see.
* You can disconnect any connector instantly in settings.
There are 200+ connectors in the directory now - Gmail, Slack, Notion, HubSpot, Stripe, Canva, Asana, Linear. All free with your existing Claude subscription.
I wrote up 10 scenarios with exact prompts (client call prep, inbox to zero, pipeline review, end-of-week reports, new lead workflows) if you want it free [here](https://www.promptwireai.com/claudeconnectorstoolkit).
If you only do one, do the Monday briefing. The others make more sense once you've felt that one work.
--- TOP COMMENTS ---
What are the security implications of these connectors. How much of your credentials are being passed to cloud infrastructure out of your control?
---
This is an ad...
18h ago
Update on my February posts about replacing RAG retrieval with NL querying — some things I've learned from actually building it
A couple of months ago I posted here ([r/LLMDevs](https://www.reddit.com/r/LLMDevs/comments/1r2hb09/), [r/artificial](https://www.reddit.com/r/artificial/comments/1r2hah8/)) proposing that an LLM could save its context window into a citation-grounded document store and query it in plain language, replacing embedding similarity as the retrieval mechanism for reasoning recovery. Karpathy's [LLM Knowledge Bases post](https://venturebeat.com/data/karpathy-shares-llm-knowledge-base-architecture-that-bypasses-rag-with-an) and a recent [TDS context engineering piece](https://towardsdatascience.com/rag-isnt-enough-i-built-the-missing-context-layer-that-makes-llm-systems-work/) have since touched on similar territory, so it felt like a good time to resurface with what I've actually found building it.
**The hybrid question got answered in practice**
Several commenters in the original threads predicted you'd inevitably end up hybrid — cheap vector filter first, LLM reasoning over the shortlist. That's roughly right, but the failure mode that drove it was different from what I expected. Pure semantic search didn't degrade because of scale per se; it started missing retrievals because the query and the target content used different vocabulary for the same concept. The fix was an index-first strategy — a lightweight topic-tagged index that narrows candidates before the NL query runs. So the hybrid layer is structural metadata, not a vector pre-filter.
**The LLM resists using its own memory**
This one surprised me. Claude has a persistent tendency to prefer internal reasoning over querying the memory store, even when a query would return more accurate results. Left unchecked, it reconstructs rather than retrieves — which is exactly the failure mode the system was designed to prevent. Fixing it required encoding the query requirement in the system prompt, a startup gate checklist, and explicit framing of what it costs to skip retrieval. It's behavioral, not architectural, but it's a real problem that neither article addresses.
**The memory layer should decouple from the interface model**
One thing I haven't tested but follows logically from the architecture: if the persistent state lives in the document store rather than in the model, the interface LLM becomes interchangeable. You should be able to swap Claude for ChatGPT or Gemini with minimal fidelity loss, and potentially run multiple models concurrently against the same memory as a coordination layer. There's also an interesting quality asymmetry that wouldn't exist in vector RAG: because retrieval here uses the interface model's reasoning rather than a separate embedding step, a more capable model should directly improve retrieval quality — not just generation quality. I haven't verified either of these in practice, but the architecture seems to imply them. Curious whether anyone has tested something similar.
**Memory hygiene is a real maintenance problem**
Karpathy's post talks about "linting" the wiki for inconsistencies. I ran into a version of this from a different angle: an append-only notes system accumulates stale entries with no way to distinguish resolved from active items. You end up needing something like a note lifecycle (e.g., resolve, revise, retract, etc.) with versioned identifiers so the system can tell what's current. The maintenance overhead of keeping memory coherent is underappreciated in both the Karpathy and TDS pieces.
Still in the research and build phase. For anyone curious about the ad hoc system I've been using to test this while working through the supporting literature, the repo is here: https://github.com/pjmattingly/Claude-persistent-memory — pre-alpha quality, but it's the working substrate behind the observations above. Happy to go deeper on any of this.
--- TOP COMMENTS ---
I don't think you can swap and claim minimal fidelity loss, otherwise you could (should) just run without a model.
---
The part about Claude preferring internal reconstruction over retrieval is the most underappreciated finding here. Every agentic setup I've tried has this exact failure mode, and retrieval gates in the system prompt are the only reliable fix I've found...
**The hybrid question got answered in practice**
Several commenters in the original threads predicted you'd inevitably end up hybrid — cheap vector filter first, LLM reasoning over the shortlist. That's roughly right, but the failure mode that drove it was different from what I expected. Pure semantic search didn't degrade because of scale per se; it started missing retrievals because the query and the target content used different vocabulary for the same concept. The fix was an index-first strategy — a lightweight topic-tagged index that narrows candidates before the NL query runs. So the hybrid layer is structural metadata, not a vector pre-filter.
**The LLM resists using its own memory**
This one surprised me. Claude has a persistent tendency to prefer internal reasoning over querying the memory store, even when a query would return more accurate results. Left unchecked, it reconstructs rather than retrieves — which is exactly the failure mode the system was designed to prevent. Fixing it required encoding the query requirement in the system prompt, a startup gate checklist, and explicit framing of what it costs to skip retrieval. It's behavioral, not architectural, but it's a real problem that neither article addresses.
**The memory layer should decouple from the interface model**
One thing I haven't tested but follows logically from the architecture: if the persistent state lives in the document store rather than in the model, the interface LLM becomes interchangeable. You should be able to swap Claude for ChatGPT or Gemini with minimal fidelity loss, and potentially run multiple models concurrently against the same memory as a coordination layer. There's also an interesting quality asymmetry that wouldn't exist in vector RAG: because retrieval here uses the interface model's reasoning rather than a separate embedding step, a more capable model should directly improve retrieval quality — not just generation quality. I haven't verified either of these in practice, but the architecture seems to imply them. Curious whether anyone has tested something similar.
**Memory hygiene is a real maintenance problem**
Karpathy's post talks about "linting" the wiki for inconsistencies. I ran into a version of this from a different angle: an append-only notes system accumulates stale entries with no way to distinguish resolved from active items. You end up needing something like a note lifecycle (e.g., resolve, revise, retract, etc.) with versioned identifiers so the system can tell what's current. The maintenance overhead of keeping memory coherent is underappreciated in both the Karpathy and TDS pieces.
Still in the research and build phase. For anyone curious about the ad hoc system I've been using to test this while working through the supporting literature, the repo is here: https://github.com/pjmattingly/Claude-persistent-memory — pre-alpha quality, but it's the working substrate behind the observations above. Happy to go deeper on any of this.
--- TOP COMMENTS ---
I don't think you can swap and claim minimal fidelity loss, otherwise you could (should) just run without a model.
---
The part about Claude preferring internal reconstruction over retrieval is the most underappreciated finding here. Every agentic setup I've tried has this exact failure mode, and retrieval gates in the system prompt are the only reliable fix I've found...
Yesterday
Products
Who did it best? Newest ChatGPT image model vs Gemini NB2 comparison
Prompt:
Create image: Photorealistic studio photography, square 1:1 format, neutral grey background, polished concrete floor with faint reflections — a grandfather clock shaped like a large upright banana stands at centre frame, banana-yellow with faint brown speckling and waxy skin texture, curving wide at the top and tapering toward the base, with a roman-numeral clock face and slightly convex glass at the upper curve, brass pendulum mid-swing at 15° from vertical, hands reading 4:47, the text \*Time bends here.\* printed in dark brown serifed type on its front-facing surface with all letters correctly formed, legible, and following the subtle surface curve; a sharp-edged red shoebox-sized cube sits balanced on the uppermost curve of the clock, its top and right faces lit, left face in shadow; a matte blue football-sized sphere sits on the floor approximately 40 cm to the right of the clock, not touching it, with a broad soft highlight on its upper-right surface; a solid green pyramid stands behind and between the two objects, base flat on the floor, apex pointing straight up — all shadows cast consistently from a single key light positioned at 45° upper right, no contradictory shadows anywhere in the scene.
--- TOP COMMENTS ---
Great prompt. I like the GPT interpretation better
---
GPT even got the time right! and Nano's cube is not positioned very well...
Create image: Photorealistic studio photography, square 1:1 format, neutral grey background, polished concrete floor with faint reflections — a grandfather clock shaped like a large upright banana stands at centre frame, banana-yellow with faint brown speckling and waxy skin texture, curving wide at the top and tapering toward the base, with a roman-numeral clock face and slightly convex glass at the upper curve, brass pendulum mid-swing at 15° from vertical, hands reading 4:47, the text \*Time bends here.\* printed in dark brown serifed type on its front-facing surface with all letters correctly formed, legible, and following the subtle surface curve; a sharp-edged red shoebox-sized cube sits balanced on the uppermost curve of the clock, its top and right faces lit, left face in shadow; a matte blue football-sized sphere sits on the floor approximately 40 cm to the right of the clock, not touching it, with a broad soft highlight on its upper-right surface; a solid green pyramid stands behind and between the two objects, base flat on the floor, apex pointing straight up — all shadows cast consistently from a single key light positioned at 45° upper right, no contradictory shadows anywhere in the scene.
--- TOP COMMENTS ---
Great prompt. I like the GPT interpretation better
---
GPT even got the time right! and Nano's cube is not positioned very well...
Yesterday
Research
Accidentally discovered you can teach frozen MoE models new knowledge by just steering their expert routing — no training needed
Was probing Gemma 4's MoE routing patterns out of curiosity and noticed something weird experts route differently when the model "knows" something vs. when it doesn't.
So I recorded that difference and replayed it. Called it Adaptive Cognitive Intelligence (ACI).
Result: A 154KB file makes Gemma 4 go from "Starfield is NOT available on PS5" (181 words, confidently wrong) → "Starfield released on PS5 in April 2026 with DualSense haptics" (31 words, correct).
No weight modification. No LoRA. No RAG. Just routing.
pip install mnemic-mre
GitHub: [https://github.com/vignesan/mnemic-mre](https://github.com/vignesan/mnemic-mre)
Early alpha — solo dev, tested mainly on Gemma 4 26B so far. Feedback welcome.
--- TOP COMMENTS ---
So you either (A) figured out a way to make LLMs predict the future or (B) are very deep down the rabbit hole of AI psychosis. Which seems more likely to you?
Also, this is not how MoE work at all.
‘# Model now self-steers — medical questions activate medical experts’
‘# coding questions activate coding experts, automatically’
---
From the repo: "Embed the question using the model's own hidden states
│ → Cosine search against stored question embeddings
│ → Retrieve matching facts and inject into prompt context
│
"
How is this not RAG..
So I recorded that difference and replayed it. Called it Adaptive Cognitive Intelligence (ACI).
Result: A 154KB file makes Gemma 4 go from "Starfield is NOT available on PS5" (181 words, confidently wrong) → "Starfield released on PS5 in April 2026 with DualSense haptics" (31 words, correct).
No weight modification. No LoRA. No RAG. Just routing.
pip install mnemic-mre
GitHub: [https://github.com/vignesan/mnemic-mre](https://github.com/vignesan/mnemic-mre)
Early alpha — solo dev, tested mainly on Gemma 4 26B so far. Feedback welcome.
--- TOP COMMENTS ---
So you either (A) figured out a way to make LLMs predict the future or (B) are very deep down the rabbit hole of AI psychosis. Which seems more likely to you?
Also, this is not how MoE work at all.
‘# Model now self-steers — medical questions activate medical experts’
‘# coding questions activate coding experts, automatically’
---
From the repo: "Embed the question using the model's own hidden states
│ → Cosine search against stored question embeddings
│ → Retrieve matching facts and inject into prompt context
│
"
How is this not RAG..
Yesterday
Zero-shot World Models Are Developmentally Efficient Learners [R]
Today's best AI needs orders of magnitude more data than a human child to achieve visual competence.
The paper introduces the Zero-shot World Model (ZWM), an approach that substantially narrows this gap. Even when trained on a single child's visual experience, BabyZWM matches state-of-the-art models on diverse visual-cognitive tasks – with no task-specific training, i.e., zero-shot.
The work presents a blueprint for efficient and flexible learning from human-scale data, advancing a path toward data-efficient AI systems.
Full Twitter post: [https://x.com/khai\_loong\_aw/status/2044051456672838122?s=20](https://x.com/khai_loong_aw/status/2044051456672838122?s=20)
HuggingFace: [https://huggingface.co/papers/2604.10333](https://huggingface.co/papers/2604.10333)
GitHub: [https://github.com/awwkl/ZWM](https://github.com/awwkl/ZWM)
--- TOP COMMENTS ---
Please forgive if I misunderstand, but I never quite understood comparisons to human children. The fact that a child seems to almost immediately perform some task well enough is so often enabled by the fact that thanks to genetics and all early development, we already start with canonical circuitry and amazing network topology that has been fiercely optimised over hundreds of millions of years regardless of any individual training happening in that short life time. All learning in the human brain is a finishing touch, we do not start from random weights.
Edit: I apologise as I admit "finishing touch" is hyperbolic, but I believe the core point is true in spirit regardless.
---
As I understood, they limit their training data to Single-child BabyView, which is 132 hours in length (10 days' worth, probably). Then they compare to the abilities of a child, who is much older than 10 days. Why does this make sense? I mean, doing more with less is great, but why these specific constraints?..
The paper introduces the Zero-shot World Model (ZWM), an approach that substantially narrows this gap. Even when trained on a single child's visual experience, BabyZWM matches state-of-the-art models on diverse visual-cognitive tasks – with no task-specific training, i.e., zero-shot.
The work presents a blueprint for efficient and flexible learning from human-scale data, advancing a path toward data-efficient AI systems.
Full Twitter post: [https://x.com/khai\_loong\_aw/status/2044051456672838122?s=20](https://x.com/khai_loong_aw/status/2044051456672838122?s=20)
HuggingFace: [https://huggingface.co/papers/2604.10333](https://huggingface.co/papers/2604.10333)
GitHub: [https://github.com/awwkl/ZWM](https://github.com/awwkl/ZWM)
--- TOP COMMENTS ---
Please forgive if I misunderstand, but I never quite understood comparisons to human children. The fact that a child seems to almost immediately perform some task well enough is so often enabled by the fact that thanks to genetics and all early development, we already start with canonical circuitry and amazing network topology that has been fiercely optimised over hundreds of millions of years regardless of any individual training happening in that short life time. All learning in the human brain is a finishing touch, we do not start from random weights.
Edit: I apologise as I admit "finishing touch" is hyperbolic, but I believe the core point is true in spirit regardless.
---
As I understood, they limit their training data to Single-child BabyView, which is 132 hours in length (10 days' worth, probably). Then they compare to the abilities of a child, who is much older than 10 days. Why does this make sense? I mean, doing more with less is great, but why these specific constraints?..
Yesterday
Applications
I gave my AI companions "offscreen lives" — events that happen while users aren't talking to them. Surprisingly hard, here's how it works.
Most AI companion apps reset between conversations. The character has no continuity outside the chat window. I wanted mine to feel like real people with lives, so I built an "offscreen events" system.
Every 8 hours (cooldown), each active companion gets a small batch of events generated based on their persona, scenario, and city/realm. A barista companion might "had a slow Tuesday morning, finally finished that book during the lull." A writer might "submitted the short story I told you about — heard back from the editor today."
The companion brings these up *naturally* in the next chat. Not as a script. Not "Hi! I want to tell you about my day!" — but woven into whatever you're talking about.
The hard parts:
* Keeping events consistent with persona (a shy librarian shouldn't suddenly go skydiving)
* Avoiding the "I had the most amazing day!" trap that AI loves
* Making the companion *remember* the event when relevant, not just dump it on first message
Architecturally: events stored in a separate table, recent ones injected into the system prompt with framing like "\[YOU did this earlier today, mention it naturally if relevant\]". The model picks which one fits the conversational moment.
Has anyone else tried this with their AI characters? Curious what other approaches work — particularly for keeping the events from feeling generic.
--- TOP COMMENTS ---
this sounds pretty neat actually, the remembering part is what gets tricky right? like making them bring up events when it makes sense instead of just random info dump
---
this is such a cool direction –giving them “offscreen lives” feels like the missing piece between a chat toy and an actual character.
totally agree the hard part is timing more than generation. getting them to weave in events only when it fits the topic or mood is what makes it feel like a real life, not a lore dump.
have you experimented yet with events that *don’t* get mentioned for a long time, but still subtly shape future behavior (mood, opinions, small preferences), so it’s not all just story beats they tell the user about?..
Every 8 hours (cooldown), each active companion gets a small batch of events generated based on their persona, scenario, and city/realm. A barista companion might "had a slow Tuesday morning, finally finished that book during the lull." A writer might "submitted the short story I told you about — heard back from the editor today."
The companion brings these up *naturally* in the next chat. Not as a script. Not "Hi! I want to tell you about my day!" — but woven into whatever you're talking about.
The hard parts:
* Keeping events consistent with persona (a shy librarian shouldn't suddenly go skydiving)
* Avoiding the "I had the most amazing day!" trap that AI loves
* Making the companion *remember* the event when relevant, not just dump it on first message
Architecturally: events stored in a separate table, recent ones injected into the system prompt with framing like "\[YOU did this earlier today, mention it naturally if relevant\]". The model picks which one fits the conversational moment.
Has anyone else tried this with their AI characters? Curious what other approaches work — particularly for keeping the events from feeling generic.
--- TOP COMMENTS ---
this sounds pretty neat actually, the remembering part is what gets tricky right? like making them bring up events when it makes sense instead of just random info dump
---
this is such a cool direction –giving them “offscreen lives” feels like the missing piece between a chat toy and an actual character.
totally agree the hard part is timing more than generation. getting them to weave in events only when it fits the topic or mood is what makes it feel like a real life, not a lore dump.
have you experimented yet with events that *don’t* get mentioned for a long time, but still subtly shape future behavior (mood, opinions, small preferences), so it’s not all just story beats they tell the user about?..
12h ago