Your central hub for AI news and updates on Infrastructure. We're tracking the latest articles, discussions, tools, and videos from the last 7 days.
Fyra's Brief
NVIDIA has introduced the Cosm Policy, a state-of-the-art robot control policy that post-trains the Cosm Predict-2 world foundation model for manipulation tasks.
The Cosm Policy is a significant step towards adapting world foundation models for robot control and planning, and its successful deployment will depend on further experimentation and optimization.
Fyra's Brief
NVIDIA Run:ai v2.24 introduces time-based fairshare scheduling mode to address over-quota resource fairness in Kubernetes clusters. This capability tracks historical resource usage to adjust over-quota allocations and ensures fair sharing over time.
Time-based fairshare addresses a significant challenge in shared GPU infrastructure and provides a major improvement in over-quota resource fairness for organizations relying on Run:ai.
Fyra's Brief
Dynamic Context Parallelism (Dynamic-CP) is introduced in NVIDIA Megatron Core for large-scale model training, achieving up to 1.48x speedup on real-world datasets by efficiently handling variable-length sequences.
NVIDIA Megatron Core's Dynamic-CP is a significant improvement for large-scale model training, reducing computational inefficiency and memory consumption by handling variable-length sequences efficiently.
Fyra's Brief
NVIDIA's MPS delivers significant throughput wins for small language models (<3B parameters) with short-to-medium context, and engines with significant CPU overhead, but falls short for larger models or longer contexts.
NVIDIA's MPS is a useful tool for boosting GPU utilization in specific scenarios, but its impact is limited by the nature of the workload and model size.
Fyra's Brief
A ZDNET reporter attempted to download and run an AI model on a MacBook Pro, expecting ease and quick results. However, the experience turned out to be painfully slow, requiring a significant amount of time to process even simple tasks.
Running AI models locally can be a time-consuming and resource-intensive task, requiring a significant amount of time, memory, and patience, which might not be feasible for everyone, highlighting the need for efficient cloud-based solutions.
Fyra's Brief
A massive winter storm has strained power grids in the US, highlighting the challenges of accommodating growing AI data centers. Wholesale electricity prices soared in Virginia, and experts warn of increasing energy costs due to AI demand.
This article highlights the growing strain on power grids as AI data centers expand, and the need for effective policies to mitigate the impact.
Fyra's Brief
NVIDIA TensorRT for RTX introduces adaptive inference, eliminating manual tuning and manual GPU-specific optimizations for real-time AI applications on consumer-grade devices.
NVIDIA TensorRT for RTX brings adaptive inference to the table, enabling real-time AI performance on consumer-grade devices without compromising portability.
Fyra's Brief
The US is leading a record surge in gas-fired power generation driven by AI demands, which will significantly increase planet-heating emissions and have major costs for the climate.
This article highlights the unintended consequences of AI demands driving a surge in gas-fired power generation, which will significantly harm the climate if not addressed.
Check back soon for new AI tools
Community talk
SpaceX seeks FCC nod for solar-powered satellite data centers for AI
Managed to run Kimi k2.5 IQ4-SX locally.
I tracked GPU prices across 25 cloud providers and the price differences are insane (V100: $0.05/hr vs $3.06/hr)
Running KimiK2 locally
[D] Lessons from building search over vague, human queries
768Gb "Mobile" AI Server Follow-Up Part 1, Look Inside
[D] How do you actually track which data transformations went into your trained models?
China plans space‑based AI data centres, challenging Musk's SpaceX ambitions
Why no NVFP8 or MXFP8?
[D] What framework do you use for RL post-training at scale?