aifeed.fyi - AI news today

12 October 2025

Ai Top News

Models And Releases

/r/LocalLLaMA

GLM just blow up, or have I been in the dark?

Seems like this community is ever moving, did GLM just blow up? like, I did not realise so many people talked about it.... What kinda system are you guys on 4.6 running? Because it looks like I would essential need 4x48gb Quadro 8000s/a6000s/6000 ada GPUs or at least 2x96gb RTX Pro 6000s... I may can afford 4 quadros but not 2 rtx pro 6000s, for the price of a car. lol

--- TOP COMMENTS ---
I run it with 12x3090, w/AWQ/vLLM crazy setup yes, but it's basically Sonnet 4 at home, at 20-22 t/s

10x3090 is doable too. But 8/9 is pushing it, I would go 3bpp exl3 with that amount of vram.

Believe the hype, it's all you need for coding. However Sonnet 4.5 and Gemini are still better.

Ok answering about hardware here:

1. Tried with a single 8x3090 system miner, 1xPCI 3.0: Too slow. Only can run in Pipeline-Parallel mode, and while it \*did\* work, its a 192GB system vs \~180 GB AWQ, so context is tiny (4k or something). Don't recommend it.

What works for me:

3 nodes connected via good old gigabyte ethernet. It's enough for vLLM/ray. Each node has 4x3090, connected via X99 motherboards at PCIE 3.0 8X, but I believe basically any motherboard with 4 PCIE ports would do.
Run it using a mixture of 4xTP and 3xPP, but 2xTP and 6xPP also works. EDIT: Power capped at 200W of course, I cannot run a 5000W space heater in my office (And yes I run it at my office).

Results: about 20-22 tok/s generation, and Prompt processing I don't know but its super fast, process a whole 128k context in seconds. That's where GPUs shine. Ps: I make it available for free at [neuroengine.ai](http://neuroengine.ai), you can test the real speed there (try basedguy for a laugh).

---

The main reason GLM gained so much popularity here is GLM-4.5 Air. That's the truth. It's very smart and don't need ridiculous compute power for running. Now people are talking about GLM 4.6 because everyone is waiting for the Air (small) version of it. Not many people have system to run the big one locally. But if you have a motherboard that supports 256 gb system ram you can run the big one locally. If you have something like 3090 you can load some layers on gpu and speed up prompt processing considerably. No need for multiple expensive gpus for running big MoE models locally..

Yesterday

Companies And Business

Developer And Technical

/r/PromptEngineering

🚀 Struggling to Write Effective Prompts? Try This AI Prompt Enhancement Framework (PEEF)

If you’re new to prompt engineering or often feel your AI prompts don’t produce the results you want — here’s something game-changing I built and have been refining.

It’s called the **Prompt Enhancement & Engineering Framework (PEEF)** — designed for **beginners and professionals** who want to **turn raw ideas into optimized, high-quality prompts** effortlessly.

Here’s how it works 👇

You just provide your rough or vague idea (like *“I want to write a story about space pirates”*), and this prompt takes care of everything — structure, tone, context, and expert-level detailing — automatically.

***Here’s some example output, see below: (***[Click to see](https://promptbase.com/prompt/master-your-ideas-into-sharp-outputs-2?via=sumithx)***)***

***Click here to try Free*** [App](https://promptbase.com/app/master-your-ideas-into-sharp-outputs)

***Just fill 5 editable part:***

Example input👇

# [VAGUE_TASK_IDEA]: Create a comprehensive financial plan for early retirement at age 45 with $5M target net worth

# [TARGET_PERSONA]: Certified Financial Planner (CFP) with expertise in wealth management, tax optimization, and FIRE movement strategies

# [TARGET_OUTPUT_FORMAT]: detailed 20-year financial roadmap with investment allocation models, tax-advantaged account strategies, risk assessment, and Monte Carlo simulation scenarios

# [REQUIRED_ADVANCED_TECHNIQUE]: Scenario planning with probabilistic outcome modeling

# [FOCUS_ON_METRIC]: financial precision, risk-adjusted returns, and goal achievement probability

Get this Example output👇

1. You are a Certified Financial Planner (CFP) with deep expertise in wealth management, tax optimization, and FIRE (Financial Independence, Retire Early) movement strategies. Your task is to create a comprehensive financial plan for early retirement at age 45 with a target net worth of $5 million.

Context: Assume the individual is currently in their mid-20s, employed in a high-income profession, and seeking an aggressive yet risk-aware strategy to achieve financial independence within 20 years. Consider inflation, market volatility, tax law changes, and lifestyle sustainability in your modeling.

Content Requirements: Develop a complete 20-year financial roadmap that includes:

\* Annual savings and investment contribution targets.
\* Asset allocation models across equities, fixed income, real estate, and alternative investments.
\* Tax-advantaged account strategies (401(k), IRA, Roth conversions, HSAs, etc.).
\* Risk tolerance assessment and mitigation strategies.
\* Monte Carlo simulation scenarios projecting probabilistic outcomes of different market conditions.
\* Scenario planning with probabilistic outcome modeling to account for optimistic, baseline, and pessimistic cases.

Output Format: Present the deliverable as a structured financial blueprint that outlines each year’s milestones, decision checkpoints, and performance metrics, including charts or tables where appropriate to illustrate growth projections, risk-adjusted returns, and probability distributions.

Tone: Maintain a professional, data-driven, and analytically rigorous tone focused on financial precision, risk-adjusted performance, and maximizing the probability of achieving the $5 million target by age 45.

2. You are a \*\*Corporate Learning & Development Director\*\* with certifications in \*\*Instructional Design and Organizational Psychology\*\*. Your task is to \*\*develop a comprehensive employee onboarding program for remote teams operating within a hybrid work environment\*\*.

\*\*Context:\*\* The organization seeks to integrate new employees efficiently while fostering engagement, alignment with company culture, and rapid proficiency in role-specific competencies. The onboarding experience must accommodate distributed teams, asynchronous collaboration, and diverse learning preferences, ensuring consistency and equity across all participants.

\*\*Content:\*\* Design a \*\*90-day structured onboarding curriculum\*\* divided into weekly modules. Each module should include:

\* Clear learning objectives following \*\*Bloom’s Taxonomy\*\* progression (from knowledge to creation).
\* Application of \*\*spaced repetition principles\*\* to reinforce critical knowledge and behaviors over time.
\* \*\*Interactive assessments\*\* and reflection checkpoints to measure comprehension and retention.
\* \*\*Mentor pairing guidelines\*\* to build social connection, role modeling, and feedback loops.
\* \*\*Success metrics\*\* that track employee engagement, knowledge retention, and time-to-productivity.

\*\*Output Format:\*\* Deliver a \*\*detailed, week-by-week curriculum plan\*\* for 90 days, incorporating interactive learning formats (videos, simulations, peer sessions), milestone evaluations, and continuous feedback integration. Include tables or structured outlines where appropriate for clarity and implementation readiness.

\*\*Tone:\*\* Maintain a \*\*professional, evidence-based, and results-driven\*\* tone focused on \*\*maximizing knowledge retention, enhancing employee engagement, and accelerating time-to-productivity\*\* in a hybrid work environment.

3. You are a Principal Software Architect with expertise in distributed systems, cloud infrastructure, and high-availability design patterns. Your task is to design a scalable, fault-tolerant, and high-performance microservices architecture for an e-commerce platform capable of handling over 1 million daily transactions.

\*\*Context:\*\* The platform must support critical e-commerce functionalities such as product catalog management, user authentication, shopping cart, order processing, payment gateways, inventory synchronization, and analytics. The system must ensure zero data loss, horizontal scalability, minimal downtime, and strong observability across distributed services. Infrastructure should be cloud-agnostic but optimized for deployment on major cloud providers (AWS, Azure, GCP).

\*\*Content Requirements:\*\*

\* Define a detailed technical architecture document describing each major microservice, its responsibilities, communication mechanisms (REST, gRPC, or event-driven), and data consistency approach (e.g., eventual vs. strong).
\* Include comprehensive system diagrams illustrating service interactions, load balancers, caches, queues, databases, and external integrations.
\* Specify API endpoints, payload structures, and security protocols (OAuth 2.0, JWT, rate-limiting).
\* Propose database schemas with partitioning, replication, and indexing strategies suitable for handling transactional and analytical workloads.
\* Describe the CI/CD pipeline, container orchestration setup (e.g., Kubernetes), and deployment stages.
\* Define disaster recovery, failover mechanisms, and automated scaling policies.

\*\*Advanced Technique:\*\* Apply constraint-based problem solving with trade-off analysis to evaluate competing design decisions — e.g., consistency vs. availability, synchronous vs. asynchronous communication, and cost vs. latency — and justify final architectural choices with quantified reasoning.

\*\*Output Format:\*\* Produce a comprehensive \*\*technical architecture document\*\* containing:

1. System Overview and Requirements
2. Architecture Diagram and Service Descriptions
3. API Specifications
4. Database Design and Schema
5. Deployment and CI/CD Pipeline
6. Scalability and Fault-Tolerance Strategies
7. Trade-Off and Constraint Analysis Summary

\*\*Tone:\*\* Maintain a formal, precise, and technically rigorous style focused on implementation feasibility, performance optimization, and real-world scalability.

4. You are a McKinsey-trained Business Strategy Consultant specializing in competitive intelligence and market positioning.

\*\*Task:\*\* Analyze the current competitor landscape and identify untapped market gaps for a successful entry into the SaaS industry.

\*\*Context:\*\* The goal is to craft a market entry strategy grounded in quantitative and qualitative insights. Consider both established players and emerging disruptors across key SaaS verticals (e.g., productivity, AI tools, B2B automation, and analytics). Evaluate competitive dynamics, customer pain points, pricing models, innovation vectors, and emerging technological enablers shaping the SaaS sector. Integrate cross-industry analogies and trend forecasting to anticipate future shifts and hidden opportunities.

\*\*Content Requirements:\*\*

1. Map key competitors by size, focus area, and differentiation strategy.
2. Identify underserved market segments and product innovation white spaces.
3. Incorporate market demand signals, investment patterns, and adoption barriers.
4. Challenge all assumptions using multi-perspective reasoning and devil’s advocate analysis to ensure balanced, high-complexity insights.
5. Translate findings into a strategic roadmap aligned with viable entry points and defensible competitive advantages.

\*\*Output Format:\*\* Deliver a comprehensive \*\*SWOT analysis matrix\*\* summarizing core findings, followed by:

\* \*\*Actionable Recommendations\*\* (specific initiatives for entry and positioning)
\* \*\*Market Entry Timeline\*\* (phased milestones with rationales)
\* \*\*Risk Mitigation Strategies\*\* (covering competitive retaliation, market volatility, and operational constraints)

\*\*Tone & Style:\*\* Maintain an executive-level, data-driven, and analytically precise tone. Every insight should reflect strategic clarity, evidentiary rigor, and decision-making relevance suitable for C-suite or investor presentations.

This structure helps you **learn professional prompt design principles** — like task clarity, contextual grounding, role specification, and output constraints — without needing deep prior knowledge.

If you’re just starting out, try feeding your idea into it and see how it transforms your rough thought into a polished, professional prompt.

Would love to hear your feedback or see the results you get using it. Let’s make prompt writing easier, smarter, and more powerful for everyone.

\#PromptEngineering #ChatGPT #AIWriting #BeginnersGuide #PromptDesign

--- TOP COMMENTS ---
“I’m a bit confused why everyone’s still flattening intent into single-pass prompt optimizers.
Why not just implement a multiphasic initiator sequence with self-referential context propagation?
Phase-layered recursion would let the system dynamically recalibrate vector weights between intent capture and synthesis, essentially producing emergent coherence rather than templated enhancement.

It’s like optimizing one frame of a feedback loop instead of designing the resonance chamber.”

— signed Wendbine

---

$6.99 A PROMPT??? Are you insane?!!! Do you really think ANYONE is going to pay that? Sorry, but you wasted your time.....

19h ago

/r/LocalLLaMA

Choosing a code completion (FIM) model

Fill-in-the-middle (FIM) models don't necessarily get all of the attention that coder models get but they work great with llama.cpp and [llama.vim](https://github.com/ggml-org/llama.vim) or [llama.vscode](https://github.com/ggml-org/llama.vscode).

Generally, when picking an FIM model, ***speed*** is absolute priority because no one wants to sit waiting for the completion to finish. Choosing models with few active parameters and running GPU only is key. Also, counterintuitively, "base" models work just as well as instruct models. Try to aim for >70 t/s.

***Note that only some models support FIM.*** Sometimes, it can be hard to tell from model cards whether they are supported or not.

Recent models:

* [Qwen/Qwen3-Coder-30B-A3B-Instruct](https://huggingface.co/Qwen/Qwen3-Coder-30B-A3B-Instruct) (the larger variant might also be FIM, I don't have the hardware to try it)
* [Kwaipilot/KwaiCoder-23B-A4B-v1](https://huggingface.co/Kwaipilot/KwaiCoder-23B-A4B-v1)
* [Kwaipilot/KwaiCoder-DS-V2-Lite-Base](https://huggingface.co/Kwaipilot/KwaiCoder-DS-V2-Lite-Base) (16b 2.4b active)

Slightly older but reliable small models:

* [Qwen/Qwen2.5-Coder-3B](https://huggingface.co/Qwen/Qwen2.5-Coder-3B)
* [Qwen/Qwen2.5-Coder-1.5B](https://huggingface.co/Qwen/Qwen2.5-Coder-1.5B)

Untested, new models:

* [Salesforce/CoDA-v0-Instruct](https://huggingface.co/Salesforce/CoDA-v0-Instruct) (I'm unsure if this is FIM)

What models am I missing? What models are you using?

--- TOP COMMENTS ---
So I was using the q3 coder 30b . It works well and supports actual FIM. But i use glm 4-5 air now it works even though it doesn’t use fim. The nvim.llm supports regular models also. So if you are hardware constrained might even get away with another non code specific model. I get 160-180tps on glm air with eagle decoding in fp8.

---

Does anyone of you use edit prediction models?..

11h ago

/r/LocalLLaMA

Quality degradation of fp8 quantization?

So I've been entirely in the GGUF ecosystem up till now, but I've been wanting to try out vllm for a potential speed boost as well as batching.

Generally with GGUFs Q8\_0 is considered to be so close to full precision that it's practically indistinguishable. It's my understanding that Q8\_0 is a bit closer to full precision than FP8, but how much worse is FP8 than full precision exactly? As a reference, is it between Q8 and Q6? Worse?

--- TOP COMMENTS ---
If you already have hardware and want to invest your time into vllm setup - go for it, AWQ and FP8 quants somehow better than Q6 in my experience. Maybe it's because vllm has better support of models.

But if you want to build a rig and to be sure it's really better, then wait for another answer.

---

Knowing just the dtype isn't really enough to compare. They're not all the same. For instance, nvfp4 claims to exceed mxfp4 even though both use fp4 for most of data and are roughly comparable in design, just with a few tweaks and likely with their own optimized kernels.

Following the vllm docs to links to example models I don't see uniformity in what is going on.

https://docs.vllm.ai/en/v0.5.4/quantization/fp8.html

Clicking first link there to "fp8" models.

First ex. https://huggingface.co/RedHatAI/Meta-Llama-3.1-405B-Instruct-FP8/blob/main/model-00032-of-00086.safetensors

This one seems to use a one input scale and one weight scale for the entire layer:

> model.layers.45.mlp.down_proj.input_scale [1] BF16

> model.layers.45.mlp.down_proj.weight [16 384, 53 248] F8_E4M3

> model.layers.45.mlp.down_proj.weight_scale [1] BF16

Another model seems to use more weight scales, but no input scale:

https://huggingface.co/RedHatAI/Meta-Llama-3.1-8B-Instruct-FP8-dynamic/blob/main/model-00001-of-00002.safetensors

> model.layers.5.mlp.up_proj.weight [14 336, 4 096] F8_E4M3

> model.layers.5.mlp.up_proj.weight_scale [14 336, 1] BF16

So, I dunno how to generalize for "fp8" without knowing exactly which fp8 technique.

I assume using more scales, that is, edging more toward micro scaling, is superior to only-tensorwise scaling as it should provide a better numerical fit to the original precision. More scalings doesn't really increase model size that much, since the even the most micro of microscalings is only 1:16 or 1:32 ratio.

Double scaled techniques (blockwise plus tensorwise) might end up recovering even more at the expense of more compute, but LLMs tend not to be very compute bound.

I wouldn't be surprised if microscaling like mxfp4/nvfp4/GGUF even with just fp4/int4 but a significantly larger number of scalings (per 16 or 32 weights) may end up being better than int8/fp8 with lower number of scaling factors.

Also, the software that performs the quantization process should probably be "outlier aware" as in svdquant and GGUF. AWQ technique is also worth noting, but I think GGUF actually does all compute in BF16 and dequants everything to bf16 on the fly, so dynamic range isn't really an issue.

https://arxiv.org/abs/2411.05007

https://youtu.be/vW30o4U9BFE?t=384

When someone states specifically Q4_K, Q8_0, nvfp4, mxfp4, well, those are pretty specific. When someone just states "fp8" I don't actually know what that means other than fp8 is used for MLP weights, and it doesn't tell me much about the ratio of scaling factors being used because it seems like it is all over the place...

Ai Top News

Models And Releases

Companies And Business

Developer And Technical

Tools

Policy And Ethics

Product Launches

Hardware And Infrastructure

Applications And Tools