AI Summit Takeaways for Engineers

My biggest takeaways from Sherwin Wu: 1. AI is writing virtually all code at OpenAI. 95% of the engineers use Codex, and engineers who embrace these tools open 70% more pull requests than their peers, and that gap is widening over time. 2. The role of a software engineer is shifting from writing code to managing fleets of AI agents. Many engineers now run 10 to 20 parallel Codex threads, steering and reviewing rather than writing code themselves. 3. The average PR code review time has dropped from 10-15 minutes per PR to 2-3 minutes. Every pull request at OpenAI is now reviewed by Codex before human eyes see it, and Codex surfaces suggestions and catches issues up front. This allows engineers to focus on more creative and strategic work while dramatically increasing productivity. 4. The models will eat your scaffolding for breakfast. When building AI products, don’t optimize for today’s model capabilities. The field is evolving so rapidly that the scaffolding (vector stores, agent frameworks, etc.) that seems essential today may be obsolete tomorrow as models improve. 5. Build for where the models are going, not where they are today. The most successful AI startups build products that work at 80% capability now, knowing the next model release will push them over the line. 6. Top performers become disproportionately more productive with AI tools. AI tools amplify the productivity of high-agency individuals, so the gap between top performers and everyone else is widening. The ROI on unblocking and empowering your best people compounds faster than ever in an AI-augmented environment. 7. Most enterprise AI deployments have negative ROI because they’re top-down mandates without bottom-up adoption. Success requires both executive buy-in and grassroots enthusiasm. Sherwin recommends creating a “tiger team” of technically-minded enthusiasts (often not engineers) who can explore capabilities, apply AI to specific workflows, and create excitement throughout the organization. 8. The one-person billion-dollar startup is coming, but with unexpected second-order effects. As AI makes individuals more productive, we’ll see not just billion-dollar solo founders but an explosion of small businesses: hundreds of $100M startups and tens of thousands of $10M startups. This will transform the startup ecosystem and venture capital landscape. 9. Business process automation is an underrated AI opportunity. While Silicon Valley focuses on knowledge work, most of the economy runs on repeatable business processes with standard operating procedures. There’s massive potential to apply AI to these workflows, which are often overlooked by the tech community. 10. The next two to three years will be the most exciting in tech history. After a relatively quiet period from 2015 to 2020, we’re now in an unprecedented era of innovation. Sherwin encourages everyone to engage with AI tools and not take this moment for granted, as the pace of change will eventually slow.

Building AI is easy. Running it when OpenAI goes down is the real test. That line perfectly framed a tech meetup at Meesho, Bangalore — where the conversation wasn’t about demos, but about operating AI in the real world. Three talks. One clear theme: production reality beats lab intelligence. 1) AI at Billion Scale – Portkey | Ayush Garg (Co-founder) At scale, AI stops being a model problem and becomes a systems problem. Key lessons: - Resilience > intelligence - Stability > sophistication - Abstraction layers are unavoidable at scale - Security belongs at the gateway, not in apps Hard truths: - Blind retries = 2x cost, 3x latency - FINOPS asks: Which team consumed these tokens? - CISOs ask: Can you audit every request? - Business asks: Why did AI costs spike 4x? Observability. Accountability. Audit trails. Guardrails. At billion #scale, AI is infra first, models second. 2) Agents for #Agents – Anubhav Singh (AI Engineer @ Weights & Biases) Agents are powerful. Agents without evals are dangerous. What mattered: Agent evals using role assignment + tools LLM-based tracing with Weave from Weights & Biases. Metrics that actually reflect reality: • Tokens consumed • TTFT + TPOT • Call-level traces Standout demo: An optimizer agent improving response quality. Polite refusal trait improved from 20% → 60%. Not prompt engineering. Measured, observable improvement. 3) #Voice AI at Scale – Anuj Goel (Tech Leader at Meesho & VoiceBot) Voice AI is where all constraints collide: latency, language, and real users. The stack: - Telephony layer - VoiceBot pipeline: ASR → VCA (intent detection) → TTS - Fine-tuned LLMs to bridge human–agent conversation gaps - Models are a mix of internal and external nature Scale & performance: - 2 lakh calls/day on average, peaks at 5–10 lakh - p90 latency ~1 second - 3 Indic languages supported (Tamil, Telugu, Malayalam) + English & Hindi Key engineering choices: - Cross-region fallbacks & dynamic routing - Circuit breakers & vendor localisation - Real-time escalation using parameters like empathy & coherence - LLM streaming architecture to cut latency - Warmed-up WebSockets for faster starts - Metadata (order IDs, details) cached deliberately - Conversation phrasing intentionally not cached to mimic the natural flow of human agent behavior This wasn’t a chatbot story. This was distributed systems engineering with humans in the loop. Big takeaway from the morning: If you can’t trace it, audit it, explain its latency, and justify its cost — it’s not production AI. AI isn’t scaling because models got smarter. It’s scaling because engineering discipline finally caught up. Respect to Meesho and Nihal Kashinath & #DeepTechStars for hosting AI conversations that actually matter. Thank you for having me! #AbhiWritesAI #AIatScale

I spent 3+ hours in the last 2 weeks putting together this no-nonsense curriculum so you can break into AI as a software engineer in 2025. This post (plus flowchart) gives you the latest AI trends, core skills, and tool stack you’ll need. I want to see how you use this to level up. Save it, share it, and take action. ➦ 1. LLMs (Large Language Models) This is the core of almost every AI product right now. think ChatGPT, Claude, Gemini. To be valuable here, you need to: →Design great prompts (zero-shot, CoT, role-based) →Fine-tune models (LoRA, QLoRA, PEFT, this is how you adapt LLMs for your use case) →Understand embeddings for smarter search and context →Master function calling (hooking models up to tools/APIs in your stack) →Handle hallucinations (trust me, this is a must in prod) Tools: OpenAI GPT-4o, Claude, Gemini, Hugging Face Transformers, Cohere ➦ 2. RAG (Retrieval-Augmented Generation) This is the backbone of every AI assistant/chatbot that needs to answer questions with real data (not just model memory). Key skills: -Chunking & indexing docs for vector DBs -Building smart search/retrieval pipelines -Injecting context on the fly (dynamic context) -Multi-source data retrieval (APIs, files, web scraping) -Prompt engineering for grounded, truthful responses Tools: FAISS, Pinecone, LangChain, Weaviate, ChromaDB, Haystack ➦ 3. Agentic AI & AI Agents Forget single bots. The future is teams of agents coordinating to get stuff done, think automated research, scheduling, or workflows. What to learn: -Agent design (planner/executor/researcher roles) -Long-term memory (episodic, context tracking) -Multi-agent communication & messaging -Feedback loops (self-improvement, error handling) -Tool orchestration (using APIs, CRMs, plugins) Tools: CrewAI, LangGraph, AgentOps, FlowiseAI, Superagent, ReAct Framework ➦ 4. AI Engineer You need to be able to ship, not just prototype. Get good at: -Designing & orchestrating AI workflows (combine LLMs + tools + memory) -Deploying models and managing versions -Securing API access & gateway management -CI/CD for AI (test, deploy, monitor) -Cost and latency optimization in prod -Responsible AI (privacy, explainability, fairness) Tools: Docker, FastAPI, Hugging Face Hub, Vercel, LangSmith, OpenAI API, Cloudflare Workers, GitHub Copilot ➦ 5. ML Engineer Old-school but essential. AI teams always need: -Data cleaning & feature engineering -Classical ML (XGBoost, SVM, Trees) -Deep learning (TensorFlow, PyTorch) -Model evaluation & cross-validation -Hyperparameter optimization -MLOps (tracking, deployment, experiment logging) -Scaling on cloud Tools: scikit-learn, TensorFlow, PyTorch, MLflow, Vertex AI, Apache Airflow, DVC, Kubeflow

The biggest myth in AI today? That tools like LLMs, CoPilots, MCPs, and Agents will do the engineering for you. They won’t — because AI is engineering. LLMs. MCP. Agents. They’re all just that — tools. Yet many organizations are spending an extraordinary amount of time comparing, evaluating, and switching between tools — while missing the real essence of AI transformation. The real differentiator isn’t the toolchain. It’s the engineering mindset behind how those tools are used. Most organizations miss that AI is an engineering discipline — not a collection of experiments. It demands the same rigor as any mature system: design, development, testing, validation, rollout, and continuous optimization. Don’t go by leaderboards — they’re tested to work in controlled benchmarks, not in real-world, multi-system environments where context, latency, data, and cost all collide. And don’t fall for the misconception that AI will replace engineers. That’s a narrative being set — but having worked with top LLMs and chatbots, one thing is clear: they often fail when confronted with real engineering. Their code lacks depth, structure, and holistic system thinking. Tools never replace real engineering. They amplify those who understand it. Invest in the core. Invest in robust engineering practices. Upskill your teams. This will be your foundation in building scalable, responsible, and future-ready AI systems. Because tools will change. Frameworks will evolve. But engineering excellence — that’s what endures #aiengineering #ai #leanagenticai

Software just went through its second great refactor due to AI. Andrej Karpathy's (x OpenAI, Tesla, Stanford PHD) keynote -  “Software Is Changing (Again)” - is a must watch for founders building AI agents; here are five takeaways: 1. Natural language is the new IDE. Prompts are code; context windows are memory. Build for English-in, action-out. 2. Partial autonomy beats full autonomy. Cursor thrives because it keeps humans in the loop, orchestrates many model calls, and surfaces a clean GUI for fast verification. Ship the Iron Man suit before the autonomous robot. 3. Meet the model halfway.  Model output is untrusted code. Ship an eval/CI loop, tests and guardrails; your product’s value is in verifying, not generating. 4. LLMs resemble operating systems and utilities. Your moat will come from workflow ownership and data, not model access. 5. This is the decade—not the year—of agents. Autonomy at Tesla teaches patience: perfect demos arrive long before dependable systems. Plan for a marathon, not a sprint. Bottom line: Software + AI is eating the stack. Founders who pair LLM superpowers with pragmatic UX, tight verification loops, and controlled autonomy will define the next enterprise wave.

What roles turn a legacy technical team into an AI team that’s ready to deliver value vs. endless PoCs? Just as the AI stack must prioritize value over hype, the AI team’s composition must realign to deliver growth. Data analysts make excellent decision analysts. The focus moves from reporting (BI) with no value to outcomes (AI) with high business and customer impact. Why do business users need data? What outcome or customer value are they trying to deliver? The transition to decision analytics puts the data analyst’s technical skills in line with their business and domain expertise. The result is a high-value role. Data and BI engineers are in the best position to support the business’s emerging information needs. High-value AI is an information product. Decision-makers need information to improve outcomes and create value more efficiently. ML engineers and data scientists have AI engineering skills, so the major shift happening here is from PoCs to products. The product-first mindset and skillset are critical to support AI teams that directly impact the top and bottom line. Product owners and PMs are becoming product strategists and value owners. They ensure that the AI team only works on projects with significant ROI. They shield the AI team from endless PoCs by supporting opportunity discovery and enforcing value-centric prioritization. AI is fundamentally different from prior technologies, so it requires new capabilities and roles. AI Platform Engineers: AI isn’t a standalone technology, so a multi-technology platform is crucial. Agentic Workflow Engineers: Workflows must be reengineered for AI to deliver value. Bolt-on AI doesn’t deliver enough value to justify the costs. Hardware Optimization Engineers: Keeping training and inference costs low is a massive competitive advantage. It makes more use cases economically feasible and delivers higher margins. AI Ops Engineers: AI in production requires constant attention and modification to ensure reliable operation. AI Evaluation & Quality Engineers: Reliability is another massive competitive advantage. AI must work within specific guarantees, or customers won’t pay for it, and internal users won’t adopt it. What roles am I missing (I left one out on purpose)? What is your business doing to transition its legacy technical teams into value-centric AI teams?

The next 3,000 developers you hire might not be human 🤖 Last week, we hosted Eiso Kant , the CTO of poolside for a roundtable discussion with our CTO Advisory Board. 6 takeaways that stayed with me: ⏳ Models aren't fully there yet - but it's only a matter of time. Today’s models can do impressive things, but true autonomy still relies heavily on scaffolding: human review, QA layers, exception handling. The reliability rate for non-trivial tasks often hovers around 60% - far from the 99%+ needed for fully agentic workflows. But progress is nonlinear. Massive training runs are pushing model capabilities forward faster than intuition expects. 🐎 Relevancy will also matter as much as raw horsepower. Smart models are table stakes. Models that deeply understand YOUR codebase, YOUR libraries, YOUR business context - that’s where the competitive advantage will be built. 📦 AI won't make coding a black box - it will redefine the developer’s role. The fear: Black box spaghetti code. The reality: Better refactoring, modularization, and documentation - if you use AI as a partner, not a vending machine. Developers won't disappear. They'll evolve - from manual coding to interrogating, curating, and steering AI agents. 🧪 Measure leading indicators, not just lagging ones. Waiting six months for productivity metrics to improve (like PR cycle time) is too slow. Instead, track AI usage today: (1) How many engineers are using it daily?, (2) How many completions are leading to accepted code changes?, (3) How often are models handling full tasks vs. just assisting? 🏆 Don’t trust public benchmarks. Run your own evals. Public benchmarks are increasingly noisy - and often gamed. If you're serious about AI integration, you need your own mini-evaluation suite, tailored to your codebases and workflows. External hype fades. Internal truth compounds. 🌎 Plan now for a world where AI is your fastest-growing team. In a few years, scaling engineering won't just mean adding humans. It’ll mean managing fleets of autonomous AI agents - evolving daily, operating at cents on the dollar. That future demands hard rethinking today: How do you design onboarding, collaboration, and governance when half your workforce isn’t human? Where will human judgment and creativity still matter most? How do you build compliance frameworks for autonomous agents? The firms that treat AI as labor - not software - will define the next decade.

Five days on campus at MIT Sloan School of Management with Paul McDonagh-Smith set the intellectual rails for leading in an AI world. Two days later Jensen Huang powered onto the All In stage with the locomotive. Hard numbers and hard deadlines. Put the two together and you get an integrated playbook rather than parallel tracks. What I took away from MIT • Intent, culture, compute. Strategy, people, and infrastructure must move as one unit or not at all. • Algorithmic business thinking (ABT). Leaders learn to decompose problems, spot patterns, and abstract repeatable solutions. • Governance for speed with trust. Ethics, explainability, and automated controls belong inside the deployment loop. • Workforce refit. HR shifts from support function to chief orchestrator of reskilling and new incentive models. • Energy as strategy. Compute scale is now constrained by kilowatts and cooling, not rack space. What Jensen amplified • Pods over pyramids. A five person power pod armed with agents can out-ship a five hundred person program. • Language is the new IDE. If you can frame a question you can build an application. • Velocity equals value. AI is set to mint more millionaires in five years than the internet did in twenty. • Half-trillion compute surge. Nvidia plans to move enough super compute to light up a gigawatt conversation in every boardroom. • Compensation for outsized impact. Equity and outcome based rewards attract and keep the builders who matter. Where they lock in My week at MIT laid the conceptual rails; Jensen’s on-the-ground numbers supplied the locomotive. When intent × culture × compute meets pods × imagination × gigawatts, you end up with a fully aligned blueprint for AI value capture. My takeaways • Org first, tech second. Fund power pods and remove the ticket queues that slow them down. • Upskill to ABT and promptcraft. The real moat is problem framing, not Python syntax. • Governance as an accelerator. Automate compliance so trust scales with velocity. • Rethink rewards. Refresh equity and tie bonuses to shipped business impact before talent walks. • Secure watts before GPUs. Lock in energy and cooling capacity now or risk empty racks later. Three questions for Monday morning 1. Where can a micro pod unlock significant value in ninety days? 2. Are we teaching our leaders to think in prompts and patterns? 3. Will our data centers and vendor contracts handle the compute load we plan to unleash next quarter, or will capacity gaps slow our AI roadmap? MIT drew the map, Jensen revved the engine. Time to pick the destination and hit the throttle. Links and sources in the comments. I welcome your field stories on turning vision into velocity.

Stop playing with coding agents. Get back to building. An ex-NASA developer who ran AI agents for four months without human input shared a hard truth: AI can generate code faster than ever. But fast code is not always maintainable code. Here are 15 lessons every AI builder should know: 1. Coding is faster. Engineering is not. Someone must still ensure reliability, security and maintainability. 2. Code generation is no longer the bottleneck. Reviewing, testing and trusting the code are. 3. Start with the user problem. Never ask an agent to build something you cannot clearly explain. 4. Define measurable success. Not “Improve onboarding,” but “Increase completion from 45% to 60%.” 5. Write the announcement first. If you cannot explain why customers should care, don’t build it. 6. Design the user experience before the technology. Decide what users will see and do before choosing databases and APIs. 7. Define the architecture. Map the services, endpoints, data flow, security and failure scenarios. 8. Complete the program design. Decide file locations, functions, interfaces, call flows and tests. 9. Ask what the agent is uncertain about. Resolve important decisions before it generates thousands of lines. 10. Build in vertical slices. Create one small end-to-end workflow, test it and then expand it. 11. Passing tests does not mean the code is good. Most benchmarks measure immediate correctness not long-term maintainability. 12. Never lose your understanding of the system. You may not read every line, but you must understand the logic and critical workflows. 13. Context engineering matters more than clever prompting. Give the agent accurate requirements, constraints, decisions and operational context. 14. More agents do not guarantee more output. If review is the bottleneck, more coding agents only create more unreviewed code. 15. Human judgment remains the highest-leverage resource. AI can implement, test and automate. Humans must decide: • What is worth building • What success means • Which risks are unacceptable • What must remain maintainable • What the real bottleneck is The future is not humans versus AI. It is humans providing judgment and AI providing execution. The goal is not maximum AI autonomy. The goal is maximum reliable value delivered to users. Will coding agents eventually eliminate code reviews or make human judgment even more important?

Dear software engineers, you'll definitely thank yourself later if you spend time learning this today: ⥽ Redis  > Your AI tools can help you write code, but when it comes to fast data access and handling spikes, Redis knowledge will save you in prod. ⥽ Docker & Kubernetes > Knowing how to build, ship, and scale with Docker and K8s will make you valuable even if you’re using Copilot for code. ⥽ Message Queues (Kafka, RabbitMQ, SQS, etc.)  > When you need to decouple services or handle unpredictable spikes, nothing beats message queues. AI will not explain why your system dropped messages at 3AM, but you’ll know if you actually practiced queues. ⥽ ElasticSearch  > Search is not just about matching keywords. If you ever need to build search, logs, or analytics at scale, Elastic will come up. ⥽ WebSockets > For anything real-time, chat, games, live dashboards, WebSockets are key. Your AI tools won’t design a fault-tolerant, low-latency messaging layer for you. Only learning, building, and breaking things will. ⥽ Distributed Tracing > In a microservices jungle, tracing lets you see exactly where requests slow down or fail. AI will tell you “add tracing,” but you have to know where and how to trace. ⥽ Logging & Monitoring > Nobody wants to get a “site down” call at midnight with zero logs. Learn what, when, and how to log. Monitoring will help you detect issues before users even see them. ⥽ Concurrency & Race Conditions > Even the best AI misses subtle concurrency bugs. Learn how to write and review multithreaded or async code. You’ll save days in debugging. ⥽ Load Balancers & Circuit Breakers > When your backend goes down or traffic spikes, these patterns keep your service up. Theory is easy, handling real-world outages is something you have to experience. ⥽ API Gateways & Rate Limiting > If you ever ship a public API, you’ll thank yourself for knowing this. Otherwise, get ready for abuse, cost overruns, or worse. ⥽ SQL vs NoSQL > Not every problem is a nail, not every DB is a hammer. Know when to pick each. ⥽ CAP Theorem & Consistency Models > Distributed systems fail in surprising ways. CAP and consistency are what separate coders from engineers. ⥽ CDN & Edge Computing > Speed isn’t just about code. If your users are global, your infra better be too. ⥽ Security Basics > Learn OAuth, JWTs, and encryption because you don’t want to be the engineer who lets a breach happen. ⥽ CI/CD & Git > Automated tests, clean deployments, and rollback plans. AI can’t fix a botched deploy for you. Write some scripts, push your local hardware, break things, and figure out why. This is how you build instincts that AI can’t teach. AI will make you faster, but these fundamentals make you effective and productive.