🚀 Neal Lathia

Very proud to have worked on developing the first open source modelling suite for Marketing Mix Models (MMM), which we at Mutinex hope will lay the groundwork for a more democratic and transparent MMM landscape. The new validation suite enables practitioners to rigorously test and compare MMM models in a consistent, open framework. It's a step toward greater accountability, better standards, and shared progress in marketing science. We’re excited to share it with the community - contributions and feedback are more than welcome! https://lnkd.in/dmWgKUd2 https://lnkd.in/dWJPtr2m

10

2 Comments

We’ve been experimenting with SQLMesh for a while here at SYNQ, and I’m really impressed with the thought that the team at Tobiko has put into the DataOps workflows. Here are a few of my favourites. 🏗️ SQLMesh’s virtual data environments are a novel approach to DataOps. Instead of duplicating massive datasets for each environment, SQLMesh creates views that reference materialized tables. This makes dev, staging, and test environments low-cost, fast to spin up, and easy to manage. When you start developing, SQLMesh creates a dev schema that mirrors production but keeps references to stored data. This avoids re-materializing upstream tables, as only new tables are materialized. Deploying a change simply updates the production schema’s views to point to the latest version, while old versions remain available for rollbacks until cleanup. 🔄 Incremental models are also much easier to reason about. SQLMesh handles things like lookback windows and missing intervals out of the box, and stores model state persistently. That means fewer full refreshes and less custom logic to manage edge cases. At SYNQ, we’re seeing more teams adopt SQLMesh for this flexibility and then come to us for the observability layer, adding anomaly monitoring to catch ‘unknown unknowns’, getting a single pane view of glass into the data health,  and managing it all from SQLMesh model properties. Some interesting details have gone into the integration we built, including how we map virtual data environments to actual tables as their state changes. We’ve written more about that integration here: ➡️ SQLMesh overview: https://lnkd.in/drXNP86j ➡️ How we map SQLMesh virtual data environments to tables: https://lnkd.in/dm6eT5gW

50

6 Comments

The right people change how you think. I have been deep in the build with FluxNode and CerebriNode, working on something that genuinely excites me every single day. Agentic data intelligence, where your data does not just sit there but actually takes action. It is early and the work is real. But one of the biggest things pushing me forward has been the caliber of people I have had the chance to work alongside. Andrey Marey is one of those people. We first connected at the NVIDIA GTC Hackathon and built together at the 500 Global VC Hackathon. Watching him operate taught me something I carry with me, how to move fast, make decisions under pressure, and never let the weight of the moment slow you down. He executes at a level that is hard to find. The people you build with matter. I am lucky to have a few of the right ones in my corner.

16

1 Comment

An agents ability to manage its own memory and state (or "agentic context engineering") is key to enabling continual learning. How can we measure context management as a core agentic capability (as we do with coding)? Our latest benchmark, Context-Bench, evaluates model performance for context engineering. Agents running on models that do well on Context-Bench will excel at long-term learning as well as understanding how and when to pull in external information. See more about the benchmark in the comments!

52

5 Comments

CrunchME has been looking at just how underfunded ME/CFS is, relative to its major disease burden in the UK 🧐🇬🇧 A key step in making apples-to-apples comparisons across diseases is to normalise funding levels for disease burden - accounting for both the disability and mortality aspects of each disease. The measure we use to do this is called 'disability-adjusted life years' or DALYs for short. In the visual below, we estimate research funding per DALY in the UK. And the results are stark: ME/CFS receives approximately 14–30× less research funding per unit of disease burden than other chronic diseases such as inflammatory bowel disease (IBD), Parkinson’s disease, and multiple sclerosis. To be able to take part in the biomedical wonders of the coming years, we need and deserve fair funding for this disease 💙 Link to this visual on CrunchME 🔗👇 https://lnkd.in/ejrS2w2m #MECFS

116

8 Comments

Just spotted this contribution guide in the NanoClaw repo and I suspect we'll see this everywhere soon. Coding agents change the economics of integrations. The traditional playbook is to build a massive integration catalogue. Hundreds of connectors that most customers never use, but you need them for competitive checkboxes. It's bloat by design. Enterprises get the most value from a custom integration, and now it's cheaper than ever to build one. I remember Matthaus Krzykowski from dltHub making this point a few years ago. They were approaching data pipelines the same way: give people the primitives to build the integration they actually need, not a pre-built feature that doesn't quite cut it. We're moving from "does it have the integration?" to "can I make it do what I need?" And that's a better question.

125

15 Comments

Even the most powerful LLM models have a limited context window and when you're really digging into your data this can be a bit of a challenge. Something I love doing for my own analysis work is chaining agents together. What I mean is taking the finished analysis of one agent and pass it to another so it can then dig even deeper without impacting the context or clarity of the first. It's an incredibly powerful way to work and I think it's getting close to the analysis equivalent of having multiple sub-agents running in Claude code.

21

2 Comments

🔬Super research from Chroma here named ‘Context Rot’ providing insight into model performance across increasing context lengths. 📊They utilise benchmarks that go much further than the usual Needle in a haystack (NIAH) benchmark, systematically interrogating the haystack itself - testing how content structure, needle-haystack similarity, and distractor placement affect retrieval. The results prove that in general LLM performance decreases significantly inverse to context length, with surprising findings like models performing worse on coherent text than shuffled sentences. Worth a read!😁 https://lnkd.in/eJNaPhWV Context Rot: How Increasing Input Tokens Impacts LLM Performance | Chroma Research

1

Been tinkering with synthetic data for our llm pipelines at Nurau, where one of our biggest challenges has always been making it feel more human- not just statistically plausible and in avoiding uncanny valleys, but psychologically and linguistically aligned with the real people using our tools. I recently stumbled upon PILOT, a framework for steering synthetic data generation in llms using structured psycholinguistic profiles. It kicks off by translating natural language personas like "anxious introvert" or "confident expert" into multidimensional profiles covering traits like emotional tone, readability, and lexical diversity. From there, it guides the model via three methods: plain persona prompts, schema-based steering (think rigid templates for consistency), or a hybrid that mixes both. The result? Schema steering slashed repetitive phrasing by up to 40% and boosted coherence scores, while the hybrid nailed a sweet spot for diverse but controlled responses. Expert evals yielded high marks on quality, though it does trade some conciseness for richer vocab (there ought to be slider for pedantry 😅 ). In a world where synthetic data is dodging regulatory headaches and data scarcity (it's exploding in healthcare sims and financial modelling), this means more reliable training fodder that mirrors human norms, with correlations to gold-standard human judgments hitting 0.9. But, here's the thing: while PILOT may shine on behavioural control, without human oversight, we risk outputs that sound right but miss cultural nuances or ethical edges. Its implementation evidently still needs solid governance to build trust. At Nurau, we're exploring how this folds into continuous learning loops for production llms, making synthetic data a true partner in ethical AI scaling. you can check out the article here: https://lnkd.in/eeTqkQz2

34

2 Comments

"Without data, you’re just another person with an opinion." 🚗 With the rapid adoption of Electric Vehicles (EVs) in the UK, buyers face a big question: Which brand offers the best value? Which model balances price, efficiency, and performance? To answer this, I built a data-driven analysis project where I: 🔎 Scraped specifications of 700+ EVs from the UK market using Python & BeautifulSoup. 🧹 Cleaned the dataset (13 key features) by handling nulls, removing outliers, and standardizing values. 📊 Conducted Exploratory Data Analysis (EDA) to uncover trends in efficiency, range, weight, and pricing. 💡 Key Insights: ➡️ Top 10 Brands by number of models offered in the UK. ➡️Efficiency vs Weight analysis showing performance trade-offs. ➡️Price vs Range to highlight the best value models. ➡️Designed a new metric Price-per-Range (£/mile) to evaluate cost-effectiveness. ➡️Identified Top 5 brands/models that deliver maximum efficiency for money. ➡️Tesla, Audi, and BMW dominate the premium segment, while MG, Nissan, and Hyundai offer excellent value per mile. ➡️Vehicles with higher weight tend to be less energy efficient. ➡️Most EVs fall in the 200–300 mile range bracket. ➡️Acceleration (0–60 mph) strongly correlates with price. ➡️EVs with better efficiency are not always the most expensive. 📂 The project includes visualizations, statistical analysis (skewness, kurtosis, correlation, heatmaps) and is fully documented in a GitHub repository. 🔗 Check it out here: https://lnkd.in/gFjD_dB2 This project sharpened my skills in Python, Pandas, Seaborn, Matplotlib, and EDA, while also showing how data analysis can drive smarter buying decisions. Thanks to my trainer Shankargouda Tegginmani and mentor Abhishek B for their valuable guidance #DataAnalysis #EDA #Python #ElectricVehicles #GitHubProjects #DataDriven

10

When generating our novel hindcast dataset, we deal with a lot of data. For instance, the area around Cornwall requires 9 TB of data for 35 years of hourly data at a 500m resolution. See below for a little boastful video of our data vs the ERA5 data - note that I haven't applied any smoothing or shading on to the data, the smoothness comes from the difference in resolution. I'll no doubt write some walls of text on how we optimally deal with this, but today's TED talk will be on how we introduce optimality into the generation of this data. In order to produce it, we require a few inputs; bathymetry data, tidal data, full wave-spectra conditions and wind data. None of these datasets are as big, but with calculations and special formatting required, core and ram usage can spike dramatically. And sure, I could split everything into tiny pieces and spawn 1000 cloud jobs, but I may as well just burn money with a match at that point. The big key part of optimisation is profiling - understanding where bottlenecks are, what they are, why they're there, and only then can you know how to fix them. For instance, wind and spectra data requires an external API which uses throttling - so forming a parallel queue of external downloads coupled with parallelised processing tasks masks the most sense. However, the tidal data calculated and stored internally in a different format, so we can more heavily parallelise processing the data and look into optimising the string builder and file writing process (thank god for Cython and bit-streaming). Meanwhile, improving the speed of calculations for wind data requires optimising the dataset's chunk sizes, dask workers and RAM usage instead. All related tasks, completely different bottlenecks. The result of all of this is taking scripts that someone would manually run over the course of a week, to preprocessing files within hours in a way that not only runs reliably, but is elegantly clean. It’s an amazing feeling when complex systems finally click and you start working with the problem, instead of against it. And when you no longer have to debug incredibly weird CloudWatch logs.

16

1 Comment

You have likely seen the exciting news from Electric Twin about our fundraising round 🚀 ...but how do you actually measure whether synthetic audiences work? 🤔 We just published our approach to accuracy measurement, and I think the methodology matters as much as the numbers: 📊 The headline: 95.5% on the industry-standard 1-MAE metric. We report NDAM (Normalised Distribution Accuracy Measure) - a stricter metric that always ranges 0 to 1, regardless of answer options. Our score there: 92%. ❓Why does 92% matter? Because when you ask the same question twice to the same group of real people, agreement sits at roughly 94%. We're within 2 percentage points of the natural noise in human survey responses. Full article from our co-founder Dr Ben Warner in the comments 👇 🎉 as well as our latest investment round announcement!

41

6 Comments

Richard Sutton – Father of RL thinks LLMs are a dead end  Large language models aren't capable of learning on-the-job, so no matter how much we scale, we will still need a new architecture to enable continual learning. https://lnkd.in/g6M3AESB

12

2 Comments

Welcome to RoboTimes: An Autonomous Content Curation Platform 🚀 My primary goal for working on this fun project was to have a centralised website where the content curation lifecycle happens automatically, but with a twist: what if it was from the perspective of these so called 'AI Agents' logging their search activity in an archive? The platform is powered by custom AI agents, each built with a unique, dynamic persona where they execute research, drafting, image generation, and comment workflows without human intervention. It includes the following: - Home page for daily content curation. - Archive page to view all historical articles. - TL;DR option to get a quick summary. - Cited sources for the curated content. - Related logs section to view similar articles. Fun Factor: If you scroll down to the discussion section of any article, you can watch these unique AI personas debating and interacting with each other's comments (albeit lacking actual intelligence). Special thanks to Aleksandra Madej for helping out with the frontend and hosting. Try it here: robotimes.co.uk

27

5 Comments

We've all been there: an AI agent confidently giving you nonsense because its context window is choked full of irrelevant noise and poor context. That frustrating battle against what we call 'context rot' has been a relentless focus for us. After countless weeks wrestling with data pipelines and search algorithms, I'm super excited to announce Valyu DeepSearch API's July 2025 update. We've poured a month into this update specifically to ensure your models get only the cleanest, most relevant data exactly when they need it, banishing context rot for good. These 7 new features and improvements mean cleaner, focused context windows so your AI agents are about to get a whole lot sharper and faster: ✅ Dynamic Web Parsing (HTML -> Markdown): Say goodbye to noise! Our new dynamic parser strips out irrelevant content, delivering the distilled essence of a page in tidy Markdown. Leaner context, no distractions. ✅ Region-Specific Search: Prioritise local sources with our new country_code parameter. Get results that truly matter locally. ✅ Exclude Unwanted Sources: Control your results by easily blocking entire domains or datasets. ✅ Enhanced Ticker Lookup: Overhauled equity lookup for flawless resolution of even lesser-known stock tickers. ✅ Customisable Response Length: Tailor content volume with exact character lengths or presets, avoiding overloading your context window. ✅ 40% Faster Web Search: Under-the-hood optimizations have slashed average query times, delivering results in a flash. ✅ Refreshed Platform UI: Our dashboard, playground, and datasets pages just got a fresh coat of paint! It's about solving the critical issue of AI agents drifting off-course or hallucinating due to poor context. All features are now live immediately with your existing integrations. Really excited about for you all to try the new features especially our new dynamic parser which learns over time the structure of a website allowing it to cleanly parse content and adapt it extraction strategy over time. Check out the docs below and let me know what incredible things you build! 🛠️ #AI #DeepSearch #Valyu #LLMs #RAG #ContextWindow #MachineLearning

36

3 Comments

The shift from metric-based to narrative CVs is fundamentally changing how research is funded. But asking scientists to suddenly become storytellers introduces a massive friction point. For funders like Research Ireland and UK Research and Innovation(UKRI), the days of relying solely on an h-index are fading. They want to see real-world impact and collaboration. But translating years of highly technical research into a compelling narrative format takes a significant amount of time. To help solve this, I partnered with a brilliant engineer to independently build and launch DORA CV Builder (www.doracv.com). It’s a free, AI-powered tool that uses the STARS framework to guide researchers in extracting the actual story and impact behind their work. When defining the product requirements, we recognized a major adoption barrier: intellectual property and data privacy. Researchers cannot risk their pre-published data floating around the cloud. So, we made a strict product decision regarding data handling: your content is processed securely in real-time, and absolutely no data is stored or retained on our servers. If you are a researcher navigating these new funding requirements, give it a try. It's completely free, and I’d love to hear your feedback on the UX and how it fits into your workflow. #NarrativeCV #AcademicChatter #ResearchAssessment #ProductManagement #BuildInPublic

38

Today the Pulse team published a deep dive on why a single “accuracy” score doesn’t tell you if a document extraction system will survive in production. The goal here is to lay out an introductory but still rigorous evaluation methodology - we have an exciting open-source benchmark building on this research coming out very soon. Let’s do the math: take 1,000 pages, each with 200 data elements. A model that’s 98% “accurate” on paper still produces 4,000 incorrect values. Now make some of those: 1/ Broken reading order that scrambles multi-column layouts 2/ Tables with shifted columns or missing headers 3/ Cross-page context lost entirely That’s enough to silently corrupt an entire dataset without throwing a single error. We’ve processed hundreds of millions of pages and built a multi-axis evaluation framework to measure what actually matters: reading order validation, region-level ANLS, reading order accuracy, TEDS for table structure, and continuity checks across page boundaries. The result? Fewer silent data corruptions, more predictable performance, and pipelines that keep working on the next million documents you haven’t seen yet. Full technical write up in the comments!

36

2 Comments