Sagar Arora

RAG vs Fine-Tuning for LLMs (2026): What Actually Works in Production RAG vs fine-tuning in 2026 explained with real tradeoffs, latest trends, and a practical decision framework for production LLM systems. What you'll find inside: - RAG - Fine-Tuning - LLM Engineering Curious how you'd approach this. Drop your take in the comments. Read the full post: https://lnkd.in/gKBrQs2M #RAG #FineTuning #LLMEngineering #AIArchitecture #GenAI2026

16

1 Comment

Breaking into AI is hard. Passing a large language model (LLM) interview is even harder. Over the past few years, I’ve seen incredibly talented engineers struggle — not because they lack skill, but because LLM interviews now test system-level understanding, not just theory or coding. That’s why I wrote Cracking the LLM Interview — a practical guide designed to help engineers, researchers, and AI practitioners prepare for interviews focused on modern AI systems. 👉 You can check it out here: https://lnkd.in/gF9me7Uz Instead of overwhelming theory, it focuses on the exact topics hiring teams expect you to understand — explained clearly with mental models and structured reasoning. Here’s what you’ll learn: ✔ How transformers and LLMs actually work ✔ Fine-tuning, RAG systems, and tool use ✔ Evaluation, inference optimization, and scaling ✔ AI safety, alignment, and real-world risks ✔ How to structure strong system design answers Whether you're transitioning into AI, preparing for senior machine learning roles, or strengthening your system design skills, this book is designed to help you move from memorized answers to confident reasoning. If you're preparing for LLM-focused interviews this year, I hope this helps you walk in more prepared and more confident. #AI #MachineLearning #LLM #InterviewPrep #GenerativeAI #TechCareers #BookLaunch

14

4 Comments

Excited to share that our paper “Binary Split Categorical Feature with Mean Absolute Error Criteria in CART” has been accepted for an Oral presentation at AAAI 2026 🎉 This work tackles a long-standing challenge in decision-tree learning — how to efficiently split categorical features under the Mean Absolute Error (MAE) criterion. We show why traditional numerical encodings fail for MAE and present a new, efficient splitting algorithm that advances categorical data handling in CART. Albert Bifet Jesse Read Chao Xu  Yike Chen 📄 Read the paper https://lnkd.in/euHyBJsZ

55

1 Comment

I recently wrote this blog with Mahimna Darji, where we explain how DSPy changes the way developers build and optimize language model systems. The post explores how DSPy moves beyond prompt engineering by turning model behavior into structured, programmable logic that improves with data. It also covers how DSPy works internally, where it fits in real projects, and why many organizations are adopting it for reliable and reproducible workflows. #AI #MachineLearning #DataScience #DataEngineering #LLM #LargeLanguageModels #DSPy #Data #PromptEngineering #AmazonWebServices #Google #StanfordUniversity #Medium #TechInnovation #GenerativeAI #AIApplications #IndianaUniversityBloomington #IUB #IU #Data #Tech #LangChain #LangGraph #WomenInTech

14

Q: What do you do at Amazon? A: This. ⬇️ Our paper "GCF: Estimating Unobserved Demand Using Graph Causal Forecasting" has been published in the AAAI 2025 Industry Track 🎉 Thank you Mohit kumar and Sivaramakrishnan Kaveri for your contributions in this work. ❤️ We built: 🚀 A graph-based causal forecasting model to answer tough questions like: “What would demand have looked like without a certain intervention?” → Provides product recommendations to sellers proactively to fill demand-supply gaps in Amazon. 🚀 A novel Boundary-Alpha Positive Penalization (B-APP) loss to handle time series imbalance 🚀 Use of co-viewed product graphs to create superior synthetic control for counterfactual estimation 🚀 A multi-relational spatio-temporal RGCN + dilated CNN model that forecasts unobserved demand 🔥 Impact: Probably this model is the reason you see fewer out-of-stock products on Amazon these days 😉 Read: https://lnkd.in/gZV5h5bD

119

27 Comments

Serving LLMs on TPUs with SGLang ! I've been exploring the JAX ecosystem for inference recently and came across SGL-JAX, a high-performance, JAX-based inference engine for LLMs, specifically optimized for Google TPUs. Some highlights: 🌳 The "Radix Tree" KV Cache: SGL-JAX implements a Radix Tree (similar to PagedAttention) to manage memory. This enables efficient prefix sharing for multi-turn chat or agentic workloads where the system prompt remains constant. ⚡ JAX-Native FlashAttention: It integrates a high-performance FlashAttention kernel directly into the JAX compute graph. This is critical for faster, more memory-efficient attention usage, particularly when dealing with long sequence lengths on TPUs. 🧩 Native Tensor Parallelism: It handles the sharding logic natively using JAX distributed primitives. It supports distributing large models (like the Qwen 3 MoE series) across multiple TPU devices without needing complex, manual mesh definitions. ⏳ Continuous Batching: It dynamically schedules incoming requests to maximize TPU core saturation. You aren't waiting for a fixed batch size to fill up while tail latency spikes. 🔧 Drop-in Compatibility: It exposes an OpenAI-compatible API standard. You can literally point your existing LangChain or LlamaIndex setup at the new endpoint and switch hardware backends seamlessly. For more info about the architecture and how it works, check out the repo below! #JAX #MachineLearning #TPU #LLMOps #GoogleCloud #AIEngineering

46

1 Comment

Most existing #lean4 datasets contain only correct proofs. Models learn error correction with RL, that's expensive. With UW Math AI lab we release a dataset of 260k erroneous Lean proofs with - compiler feedback - reasoning trace - corrected proof Improvements in Error Correction: - Goedel 8B: 2x - Kimina 8B: 3x Paper: https://lnkd.in/gaybt4bd

117

2 Comments

⚡ Exciting to see our Block-Max Pruning (BMP) technique in Infinity, an open-source AI-native database designed for LLM applications! In their latest VLDB paper, “Balancing the Blend: An Experimental Analysis of Trade-offs in Hybrid Search”, Hai Jin, Yingfeng Zhang, and co-authors present a rigorous evaluation of hybrid search architectures — combining full-text, sparse, dense, and tensor retrieval. To support efficient sparse vector search at scale, they’ve integrated BMP into Infinity’s SVS engine — a nice validation of our work on fast, top-k lexical retrieval. 🔗 BMP paper: https://lnkd.in/dsc33hGc 🔗 BMP code: https://lnkd.in/dxBxv225 🔗 Infinity: https://lnkd.in/ddRK5mbr 🔗 Hybrid Search paper: https://lnkd.in/dfBuDXmt Great to see ideas from traditional IR continuing to shape the next generation of retrieval infrastructure!

31

I'm looking into Google's recent FunctionGemma release and it's something ... FunctionGemma is an open-model, tool-calling variant of their Gemma 3 270M which, due to its pruned size, can be run at the edge, on constrained consumer hardware, giving you all the juicy benefits of doing so (ie. privacy and 100% uptime with no vendor cost per token). I'm playing around with its 4bit quantized variant, trying to use Karpathy's approach outlined in his excellent llama2.c repo to make a bare-metal runtime for it. This is so exciting! Read more of their release here: https://lnkd.in/de-pS4vE

6

1 Comment

The ML course market is crowded — but this one? An absolute masterpiece. 🔥 No need to even mention Krish Naik 😊 — we all know his teaching is on another level! I actually took this course almost half an year ago, motivated by a Graph Neural Network (GNN) project under my professor Dr Muhamed Safeer Pandikkadavath Sir at NIT Calicut. It turned out to be one of the best learning decisions I’ve made. If you’re unsure about which ML course to pick, this one packs ~100 hours of pure value, covering: 1️⃣ Complete Python 2️⃣ Essential Mathematics 3️⃣ Data Science 4️⃣ Machine Learning 5️⃣ Deep Learning 6️⃣ NLP 7️⃣ Plus Git, Streamlit, MLOps, end-to-end projects & more! Now, I’m revisiting it to refresh my deep learning concepts before diving into TensorFlow 2 with Keras — and it feels like I’m back in those exciting deep learning days! The math refresh alone was worth it. I still have NLP, Time Series, and CNNs to revise, but what I’ve covered so far is enough to get going. The first ~15 hours of the TensorFlow course I’m taking focus on: > TensorFlow fundamentals > Neural network regression > Neural network classification Next up: deeper into NLP, CNNs, and Time Series in TensorFlow — and yes, I’ll be hopping back to Krish’s course whenever I need a solid refresher. ✨ This is my first post on my TensorFlow journey. Stick around — I promise you’ll see not just concepts, but real, practical implementations along the way! 🚀 #MachineLearning #DeepLearning #DataScience #Python #TensorFlow #Keras #NLP #TimeSeries #ArtificialIntelligence #MLOps #KrishNaik #GNN #GraphNeuralNetworks #AI #NeuralNetworks #MLProjects #LearningJourney #DataScienceCommunity #LinkedInLearning #CodingLife

377

10 Comments

This week has been HUGE for the AI/ML community, with a flurry of open-source initiatives shaking up the landscape. Check out these key developments: 1. Anthropic challenges OpenAI: Offering Claude to all three branches of government for $1 demonstrates a strategic move to gain government adoption and influence. https://lnkd.in/e-Q77Vfn 2. NASA and IBM unlock solar secrets: Their new AI model reveals insights into the sun, showcasing AI's power in scientific discovery. https://lnkd.in/enQPbU3j 3. OpenCUA rivals proprietary models: This open-source framework offers a compelling alternative to closed systems, fostering innovation and competition. https://lnkd.in/eJudx7w4 4. Elon Musk open-sources Grok 2.5: xAI's move to release Grok 2.5 on Hugging Face further fuels the open-source AI revolution. https://lnkd.in/e6E_wggG What are your thoughts on this surge in open-source AI? Share your opinions below! 👇 #OpenSourceAI #AI #MachineLearning #ArtificialIntelligence #OpenAI #Anthropic #xAI #NASA #Innovation ============================================================ 📋 WEEKLY AI MODEL COMPILATION 📋 𝟭. 𝗚𝗲𝗻𝗲𝗿𝗮𝗹: The Day ChatGPT Went Cold New developments in AI model architecture and capabilities. 𝙇𝙞𝙣𝙠: https://lnkd.in/ecW6Urta 𝟮. 𝗚𝗲𝗻𝗲𝗿𝗮𝗹: How to watch OpenAI’s GPT-5 announcement livestream today (updated) OpenAI continues advancing with new model capabilities and improvements. 𝙇𝙞𝙣𝙠: https://lnkd.in/eQDdZG6N 𝟯. 𝗚𝗲𝗻𝗲𝗿𝗮𝗹: Sam Altman launches GPT-oss, OpenAI's first open-weight AI language model in over 5 years OpenAI continues advancing with new model capabilities and improvements. 𝙇𝙞𝙣𝙠: https://lnkd.in/eRdY-hMW 𝟰. 𝗚𝗲𝗻𝗲𝗿𝗮𝗹: ChatGPT-parent OpenAI To Open First India Office In New Delhi This Year OpenAI continues advancing with new model capabilities and improvements. 𝙇𝙞𝙣𝙠: https://lnkd.in/ekX8ubEU These models represent the latest advances in AI capabilities and accessibility. #artificialintelligence #machinelearning #llm #openai #anthropic #meta #google #opensource

2

🚀 Can we get AI to ship production-grade code? 🤖💻 In our brand-new fireside chat, my colleagues Sachin Sharma and Chirag N R peel back the curtain on how we’re weaving LLMs into a real production codebase—no flashy “hello-world” demos, just candid lessons from the trenches. 5 takeaways you’ll hear in the video 1️⃣ Coding ≠ Engineering: AI now writes ~40 % of our diffs, but humans still own spec design, reviews, staged releases, and on-call. 2️⃣ Treat the model like a junior dev. Build “memory banks” so it accumulates project context over time—output quality scales with that context. 3️⃣ Cost matters. We landed on a free stack (Cline + Gemini 2.5 Pro preview) that rivals proprietary tools, thanks to enterprise-grade rate limits. 4️⃣ 8-10× productivity bumps on unglamorous tasks (e.g., migrating an entire dashboard to dark/light themes in < 1 hour vs. a week). 5️⃣ Adoption playbook: Start with bite-size problems, showcase quick wins, then formalize into cookbooks & hackathons for wider buy-in. Why share this now? Satya Nadella recently said 30 % of Microsoft’s code ships via AI. We’re already hovering around 40 % and aiming for 70 % in the next 6 months. If you’re curious—or skeptical—about moving beyond “copilot” to co-worker, give the chat a listen. Watch the conversation (38 min): https://lnkd.in/ggg5cA6T #AI #DevTools #SoftwareEngineering #GenAI #Juspay #LLM #Productivity

24

I got tired of manually comparing LLMs… so I published this, rankthellmmodel.info (Rank the LLM model), which I built for my own use. based on my requirements. While working with LLMs, I kept hitting the same problem: • Checking multiple sites for benchmarks   • Looking up pricing separately   • Comparing everything manually   • Still guessing which model is actually “better.” It was slow and confusing. 📊 A simple leaderboard that ranks models based on: ✔ Quality   ✔ Cost   ✔ Speed  (Data sourced from artificialanalysis.ai , will keep adding more benchmarks) ⚙️ How it works Quality → GPQA, MMLU, coding scores   Cost → price per million tokens   Speed → tokens/sec, latency  Overall = 0.5 × Quality + 0.3 × Cost + 0.2 × Speed   👉 Good + Fast + Affordable 💡 What I realized • The smartest model is not always the most useful   • Faster models are often “good enough.”   • Some top models are too expensive for production  🚧 Next: • Custom weights   • Real API latency testing   • More benchmark data  Feel free to check it out and share your feedback #AI #LLM #GenAI #MachineLearning #OpenAI #Gemini #Claude #Engineering

54

1 Comment

🚀 Making Large Language Models Faster and Smarter Today I read a research paper by Harikrishna Narasimhan, Wittawat Jitkrittum, Ankit Singh Rawat, Seungyeon Kim, Neha Gupta, Aditya Krishna Menon, and Sanjiv Kumar, titled “Faster Cascades via Speculative Decoding.” The authors propose speculative cascading, which integrates cascade deferral rules into speculative execution — blending the efficiency of cascades (small model first, defer to large when needed) with the speed of speculative decoding (draft + parallel verification). 🔑 Key findings: Achieves better cost–quality trade-offs than either cascades or speculative decoding alone. Tested on T5 and Gemma models across tasks like summarization, translation, reasoning, and coding. The token-level variant delivers the best trade-off, improving quality while keeping inference efficient. 💡 Why it matters: As LLMs grow larger, speculative cascading offers a practical path to make high-quality inference both faster and more cost-effective. 👉 Question: If efficiency techniques like this become standard, do you think we’ll see more smaller models deployed smartly—or will industry still rely on ever-larger LLMs? 👉 Research paper link in comments. #AI #LLMs #Efficiency #Research

1

1 Comment

🧠 Fine-Tuning vs. RAG — When to Use What? Had an incredibly insightful session today led by Siddanth, where we explored decision-making frameworks for choosing between fine-tuning and Retrieval-Augmented Generation (RAG) when optimizing LLMs. 🔍 Key Takeaways: ✅ Fine-tuning is ideal when the model needs to learn deeply and behave consistently — such as adopting a domain-specific style, language, or fixed workflows. Great for repeated, predictable tasks where alignment and tone matter. ✅ RAG shines when the model needs to retrieve relevant, real-time or evolving knowledge — perfect for dynamic use cases where accuracy and freshness are critical. Another great perspective we explored was this: 🔧 Where exactly does the problem lie — context or behavior? If it’s a context limitation, use prompt engineering or RAG If it’s a behavioral gap, consider fine-tuning If it's a capability need, use tool/function calling This session really helped clear up doubts I had around when to use which method — and emphasized that LLM optimization isn’t a one-size-fits-all process. #day31 #0to100xEngineers

6

Just discovered Daigram - turns YouTube videos into interactive concept maps in ~60 seconds. What it does: Paste any YouTube link → get an interactive diagram showing how concepts connect. Why this is useful: • Visual learners can see the structure at a glance • Students can organize research material • Great for creating presentations from video content • Teachers can share visual summaries • Perfect for complex topics with many interconnected ideas How it works: AI analyzes the transcript, extracts key concepts, and generates a force-directed graph. Click nodes for summaries, drag to rearrange, save custom layouts. Try it free (5 diagrams): https://daigram.ai Works on tech tutorials, philosophy, science explainers, history - any educational content. Thought this might be useful for folks who learn better visually! #AI #EdTech #VisualLearning #Productivity

3

✨ Want your AI to speak real Hindi, not the robotic kind that sounds like Google Translate’s intern? Here’s the trick: 🧾 Instead of saying: “Translate this to Hindi” Try this: “Translate into spoken Hindi as used by a kind teacher in North India. Keep the tone warm and polite, use natural phrasing, and follow Hindi’s SOV grammar.” 💡 Bonus: Pair it with a good Text-to-Speech engine and you’ll get output that actually sounds like a person, not a confused robot from Delhi Metro announcements. This little prompt tweak respects: • Tone (casual vs. formal) • SOV vs. SVO • Region & context • TTS pronunciation realism #PromptEngineering #TTS #MultilingualUX #IndianLanguages #Hindi #VoiceTech #AIUX

7

2 Comments

Excited to share our work which was one of the first finetuning works in tool use using Amazon Nova models, launched in reInvent last year. Accurate tool call is the fundamental block of building successful agentic workflows. Finetuning helps LLMs to learn accurate tool calling with correct arguments. Read the blog, and our code is released as well .

19

We just published at the ES-Reasoning workshop at ICLR 2026! Me, Yahia Alqurnawi, Preetom Kumar Biswas and Tejas Anvekar worked on ViTaB-A to better understand the structured data citation capabilities of current mLLMs. We kind of realized early that QA and Citation are two very separate tasks and models aren’t very performant or optimized for the latter. Here are some other results we concluded in the paper: 1. Surprisingly, citations in images are easier than MD and JSON. This isn’t the case for QA though. 2. Citing rows is easier than citing columns. 3. Confidence scores (internal or verbal) are not a good verification mechanism for how the model might have performed. Also, a huge thank you to Dr. Chitta Baral and Dr. Vivek Gupta for all of their support and guidance on this. If you’re interested in reading more, here’s the preprint: https://lnkd.in/gmTDV__5 We’ll keep updating the paper with any new insights or numbers we find. School of Computing and Augmented Intelligence #research #interpretability #nlp

36

10 Comments