Vishvak Murahari

Tried a small experiment on vibe coding for assessing the productivity of GPT-5. Tried similar experiments with Gemini Pro before, and also with perplexity pro. Together, as human-AI collaboration, we built up a system that can automate complex decision-making on faculty timetable preparation for a mid-size institute given all the constraints. This is a non trivial problem for a planner, given the variabilities involved. With a set of carefully written prompts, we ended up with about 500 lines of working code in under ten minutes. It read the data, optimized the schedule, resolved conflicts, and exported a clean result. I spent most of my time stating the rules and checking use cases, not typing syntax or correcting errors. As a trained academician we should not indulge in sweeping generalizations such as "gone are the days of the coders," based on one experiment. There has to be extensive and intensive research with industry-academia collaboration to understand fully the impact of such a tremendous productivity boost. When tools can draft usable scaffolds in minutes, the real leverage moves upstream and downstream: scoping the problem, modeling constraints, validating outputs, and stitching systems together. The value is in understanding the process, the data, the risks, and the goals—then making the whole thing work end to end. This is where academia and industry have to look beyond, starting from how courses can be restructured to become more cross discipline and digital focused to usher in "transformation". For decades, educational institutions—particularly local colleges—have measured success by the number of “job‑ready” developers they produce. This made sense in an era when software development was a scarce and highly manual skill. But in a world where AI can generate production‑grade code in minutes, this approach risks creating a talent surplus in areas where the demand curve is flattening. We should not be preparing our students merely to compete with algorithms—they will lose that race. Instead, the conversation must shift toward digital transformation—not just in the IT sector, but across industries and societies. The opportunities shaped by Industry 4.0 are far broader than writing code. For instance, rather than IT, operational technology OT has a great strategic role to play, and IT OT integration is a vast unexplored realm by behavioral and management science, going beyond the obvious technology. Similarly, although supply chain disruption is a vast research area, it is rarely taught as a curriculum except for being a minor part of OM. Other areas of interest can be interoperability and integration with business partners, Megaproject execution — deploying digital platforms to manage complexity, and risk. Platformization and digital ecosystem creation across technology landscapes are going to of critical essence for businesses, large and small. It is time that we work on these concepts in broader, cross disciplinary ways.

54

7 Comments

Last year's NeurIPS dropped something quietly brilliant. Most RLHF techniques treat each LLM response in isolation. Works fine for single queries. But what about when an AI needs to guide a conversation towards a goal over multiple turns? Google's MTPO paper tackled this head-on. Instead of rating individual responses, they evaluate entire conversations. A response that seems poor in isolation might be strategically brilliant when you see how the conversation unfolds. Their results were compelling, MTPO significantly outperformed single-turn baselines by learning to plan ahead rather than just optimise locally. With NeurIPS 2025 just weeks away, I'm buzzing to see what breakthroughs are coming next. If last year gave us multi-turn reasoning improvements, what's on the horizon this year? What and who are you hoping to see at this year's conference? Any top tips when heading to your first NeurIPS? 🤗 🛬 #NeurIPS2025 #ReinforcementLearning #LLMReasoning

13

2 Comments

Yesterday at GTC, NVIDIA announced a CPU optimized for single-thread performance. Surprised? If you've been paying attention to the inference bottleneck, you wouldn’t be. The primary reason - agentic AI broke the fundamental assumption that GPUs do all the heavy work. When you have agent swarms spawning sub-agents, each needing its own thread of orchestration, the CPU can easily become the chokepoint. NVIDIA's own infrastructure lead said so. With 88 Olympus cores, 1.2 TB/s memory bandwidth, Vera CPU is the fastest single-threaded performance CPU on the market. Why single-thread? Because every nanosecond that CPU stalls, your very expensive GPU sits idle waiting for data. Paired with Rubin GPUs over NVLink-C2C, Vera hits 1.8 TB/s of coherent CPU-GPU bandwidth. For context, PCIe Gen 6 doesn't come close. They also unveiled a CPU-only rack — 256 liquid-cooled Vera CPUs, purpose-built for agentic workloads. No GPUs. From the company that made its fortune on GPUs. This is a telling architectural decision in which the company that sells the world's most expensive GPUs just told you that for the next wave of AI, the CPU matters again. Full analysis of every Vera Rubin architectural choice — and why they all point to the same bottleneck — coming to AI Afterhours soon.

13

1 Comment

Context Engineering vs. Prompt Engineering Prompt engineering was the starting point of working with LLMs. Context engineering is where serious AI systems begin. Prompt engineering focuses on how we write instructions, especially system prompts to get better outputs from a model. This made sense in the early days, when most LLM use cases were single-shot tasks like classification or text generation. But as we move toward agents that operate over multiple turns, tools, and longer time horizons, prompts alone are no longer sufficient. This is where context engineering comes in. Context engineering is the discipline of curating, maintaining, and refining everything that enters the model’s context window during inference—system instructions, tools, external data, message history, and more. It’s not a one-time activity, it’s an iterative process that happens at every step of an agent’s execution. Why does this matter? Because context is not free. As context windows grow, models experience what’s often called context rot—their ability to retrieve and reason over relevant information degrades as more tokens are added. Like humans, LLMs have a limited attention budget. Every additional token competes for that attention. This limitation is rooted in transformer architectures, where attention scales quadratically with context length. Longer context doesn’t mean unlimited focus—it means harder prioritization. Effective context engineering, then, is about finding the smallest set of high-signal tokens that maximizes the likelihood of the desired outcome. That applies across: System prompts: clear, well-scoped, neither brittle nor vague Tools: minimal, unambiguous, and designed for efficient interaction Examples: a few strong, representative cases instead of exhaustive rule lists Message history: selectively retained and continuously pruned The shift from prompt engineering to context engineering mirrors the shift from chatbots to agents. Writing good prompts is necessary—but managing context intelligently is what enables reliable, scalable, agentic behavior. In short: Prompt engineering tells the model what to do. Context engineering decides what the model gets to know. And that distinction increasingly defines the difference between demos and production-grade AI systems.

30

5 Comments

You don't need a separate reference model for RL. We built an RL pipeline built on a multi-turn-focused modification of Self-Distillation Policy Optimization (SDPO) that uses the same model in two roles: student and teacher. The difference? Context window. → The student generates actions with no knowledge of the outcome. The teacher re-evaluates the same tokens with hindsight: the observation from each action, terminal test output, and a successful rollout from the same prompt. The per-token advantage is just the logit gap between these two views. → This gives you dense, per-token credit assignment instead of a single scalar reward for the whole trajectory. Think tokens get 0.25× weight (loosely guided, not tightly constrained). Action tokens get full gradient. System/user/tool result tokens get nothing. → The "teacher" is not a frozen checkpoint or a separate reward model. It's your current policy, conditioned on richer information. This is well-defined under MoE because same parameters with different routing is normal operating behavior. → We use a tiered reward system with hard gates only for truly unrecoverable failures: sandbox escapes, test corruption, harness crashes, timeouts, infinite loops. Everything else like wrong directory, failed commands, incorrect approaches are learnable signal, not wasted compute. The key insight: most of the information you need for credit assignment is already inside the model. You just need to ask it the right question with the right context. On LiveCodeBench v6 SDPO reached 48.8% vs GRPO's 41.2% beating Claude Sonnet 4 with an 8B model! That's bonkers.

5

Stanford just dropped a paradigm-shifting paper : Agentic Context Engineering (ACE), proving that LLMs can get smarter without fine-tuning a single weight. Instead of retraining, ACE lets models iteratively rewrite their own prompts, reflect, and evolve; turning context itself into a living, self-improving knowledge system. The results? +10.6% over GPT-4 agents, +8.6% on financial reasoning, and 86.9% lower cost/latency, all without labeled data. This flips the fine-tuning narrative: the future isn’t about smaller prompts, it’s about denser context. Think of it as the shift from static intelligence to autonomous context evolution, a foundation for real-time, memory-driven AI agents. Paper: https://lnkd.in/gtv8fAZ9

4