Jiarui Cao, PhD

𝗘𝘅𝗽𝗹𝗮𝗶𝗻𝗮𝗯𝗶𝗹𝗶𝘁𝘆 𝘄𝗼𝗿𝗸𝘀 — 𝘄𝗶𝘁𝗵𝗶𝗻 𝗹𝗶𝗺𝗶𝘁𝘀 Over the past decade, post-hoc explainability has become a widely adopted practice: Feature importance scores SHAP values Local surrogate models They provide insight into how predictions are generated. Yet they can: • Shift under small model perturbations • Appear stable even when models change materially • Be manipulated without altering predictive performance • Capture statistical associations rather than causal structure This does not invalidate interpretability tools. It clarifies what they are and what they are not. Most explanation tooling operates downstream of inference. Governance challenges emerge upstream, in how systems are deployed, updated, and assigned clear ownership If accountability depends primarily on dashboards, it becomes difficult to sustain over time. Part 3 will outline the architectural design choices that make AI systems defensible, not just explainable.

15

This week: https://lnkd.in/ejz9xxxp (Apple & UCSB) DEEPAMBIGQA: Ambiguous Multi-hop Questions for Benchmarking LLM Answer Completeness This paper looks at how LLMs can answer questions better by combining them with knowledge graphs. Firstly they generate question–answer pairs using multi-hop data from Wikidata, requiring more than just simple lookups or semantic similarity, for example the question “Which actors in the film Heat have won an Oscar?”. They then test three versions of LLMs answering questions: firstly a traditional LLM trying to answer directly, then one using query expansion, where the model reformulates the question using the entities within the graph to help disambiguate information. This means it can ignore songs or albums and focus on films, expanding the question to give more context → “Which actors in the 1995 film Heat directed by Michael Mann have won an Oscar?”, and finally an LLM that tries to write and execute SPARQL queries on the graph. The outcome? Disambiguating the entities using graph classes was the most helpful, increasing recall by about 13 percentage points, but the LLM-written SPARQL queries failed in more than 40% of cases. How can this be improved? Wikidata itself doesn’t explicitly define domains and ranges for most relations, so property use is ambiguous. The authors address this by deriving a simplified schema (an inferred ontology) from instance data, which helps constrain reasoning paths. Having more explicit domains and ranges, or using subproperties or OWL restrictions to describe where relationships can occur, would likely help LLMs generate more valid queries. Another benefit of a clear ontology is that it enables schema-based validation: you can check whether the generated SPARQL fits the expected pattern. For example, “You used a relationship that expects X as a subject but you used Z instead.” That feedback loop could help the LLM select the correct property next time. It’s great to see benchmarking that pushes beyond simple fact lookup and highlights the value of classifying entities and applying ontological modelling alongside LLMs. Great work Jiabao Ji, Min Li, Priyanshu Kumar, Shiyu Chang, Saloni Potdar #KnowledgeGraph #Apple #LLM #GenerativeAI #RDF #Wikidata

2

1 Comment

Another short blog post, and my first hand-drawn post for a while. https://lnkd.in/eYDjbw7K This post attempts to explain differences between the chatbot-style LLMs most people are familiar with, and Agentic AI. In brief: With chatbots Human and Bot interface through a chat window. With Agentic AI we need to think about Human, Bot and Machine as distinct elements, with Human and Bot both sharing access to (and so indirectly communicating via) the Machine, as well as the chat window. This one simple difference is key to why AAI can be so much more powerful than AI through chat window alone.

4

Get small 🤏 "Our key empirical finding: Determinism is not universal across model architectures. Evaluating five models across 480 runs (n=16 per condition), we demonstrate that well-engineered smaller mod- els (7-8B parameters) achieve perfect output consistency, while 120B parameter models fail at 12.5% consistency even with identi- cal configuration (𝑇=0.0, greedy decoding, fixed seeds). This inverse correlation between model scale and determinism fundamentally alters deployment strategies for regulated applications, where audit requirements mandate reproducibility over raw capability." https://lnkd.in/emzbF4RC

11

1 Comment

We just released our new arXiv paper: “Structure and Diversity Aware Context Bubble Construction for Enterprise Retrieval Augmented Systems”. arXiv: https://lnkd.in/gJY9USty Many RAG pipelines still default to Top K. In real enterprise docs (multi sheet Excel, scopes, contracts, technical packs) that causes fragmentation, duplicate evidence, and wasted tokens. Context Bubbles treat context assembly as an explicit constrained selection problem. -> Structure aware scoring -> Diversity and redundancy control -> Strict token and per section budgets -> A full retrieval trace for auditability and deterministic tuning In our case study we cut context from 780 tokens to 214 while improving multi section coverage and reducing overlap. Guess what? There is allot more we did behind the scenes and this paper is the first in a series on building more ‘aware’ enterprise retrieval systems, where we combine structure, diversity constraints, and auditable selection with a broader memory and agent based architecture we are rolling out. The coolest thing we have built is our AI Engine which has created it's own code base and language and is taking the route of quantum inspired structures. This is already running in our ERP workflows and our medical research pipelines to reduce context bloat and improve coverage and citation faithfulness under limited windows. Keen to hear thoughts from anyone building RAG on structured enterprise data. #RAG #LLM #EnterpriseAI #InformationRetrieval #AIEngineering

5

DeepSeek’s recent paper introduces "Hyper-Connections" (HC) as a method to enhance model capacity by expanding the residual stream into multiple parallel paths. This approach increases information capacity; however, the unrestricted mixing of these streams disrupts the identity mapping property, resulting in signal instability during training. I have written a blog post that analyzes their solution, mHC.

18

1 Comment

We published a survey kind of paper on "From Attention to Disaggregation: Tracing the Evolution of LLM Inference".  It will be invaluable for anyone interested in learning more about LLM Inference, its comparison with the CAP theorem (a concept familiar to those who have worked with distributed systems), and the state-of-the-art research in Generative AI inference https://lnkd.in/gjVe5P8J

77

🧠 𝗠𝗟 𝗳𝗼𝗿 𝗣𝗗𝗘𝘀 = 𝗘𝗻𝗱 𝗼𝗳 𝘁𝗵𝗲 𝗖𝘂𝗿𝘀𝗲 𝗼𝗳 𝗗𝗶𝗺𝗲𝗻𝘀𝗶𝗼𝗻𝗮𝗹𝗶𝘁𝘆 📊🔥 High-dimensional PDEs, long considered intractable, are now being tackled head-on by Deep Learning. Once seen as a numerical bottleneck, PDE resolution via ML enables mesh-free, scalable and generalizable solvers – from option pricing to physics. — 🧠 𝗗𝗲𝗲𝗽 𝗟𝗲𝗮𝗿𝗻𝗶𝗻𝗴 𝗳𝗼𝗿 𝗣𝗗𝗘𝘀 𝗶𝗻 𝗢𝗽𝘁𝗶𝗼𝗻 𝗣𝗿𝗶𝗰𝗶𝗻𝗴 – 𝗧𝗶𝗺𝗲 𝘁𝗼 𝗧𝗮𝗸𝗲 𝗘𝗿𝗿𝗼𝗿 𝗔𝗻𝗮𝗹𝘆𝘀𝗶𝘀 𝗦𝗲𝗿𝗶𝗼𝘂𝘀𝗹𝘆 📉📈🧮 𝗡𝗲𝘄 𝗽𝗮𝗽𝗲𝗿 𝗯𝘆 𝗝. 𝗥𝗼𝘂 (𝗠𝗮𝘆 𝟮𝟬𝟮𝟱) proposes a systematic empirical error analysis of two deep PDE solvers for option pricing: → 𝐃𝐞𝐞𝐩 𝐆𝐚𝐥𝐞𝐫𝐤𝐢𝐧 𝐌𝐞𝐭𝐡𝐨𝐝 (𝐃𝐆𝐌) → 𝐓𝐢𝐦𝐞 𝐃𝐞𝐞𝐩 𝐆𝐫𝐚𝐝𝐢𝐞𝐧𝐭 𝐅𝐥𝐨𝐰 (𝐓𝐃𝐆𝐅) ✔ Evaluated on both Black–Scholes and Heston models ✔ Convergence rates for 5 key training parameters (layers, nodes, samples, time steps…) ✔ Surprising insight: number of samples has almost no effect on DGM accuracy ✔ Clear edge of TDGF with second-order discretization — 🔍 𝗪𝗵𝗲𝗿𝗲 𝗱𝗼𝗲𝘀 𝗶𝘁 𝗳𝗶𝘁 𝗶𝗻 𝘁𝗵𝗲 𝗱𝗲𝗲𝗽 𝗣𝗗𝗘 𝗹𝗮𝗻𝗱𝘀𝗰𝗮𝗽𝗲? ➡ 𝐏𝐈𝐍𝐍𝐬 (Physics-Informed Neural Networks) Proposed by Raissi et al. (2019), PINNs solve PDEs by penalizing residuals directly in the loss function. Ideal for high-dimensional problems – but can be unstable or slow to converge. Use case: widely supported in 𝗡𝗩𝗜𝗗𝗜𝗔 𝗠𝗼𝗱𝘂𝗹𝘂𝘀, a powerful PINN framework for scientific ML. ➡ 𝐃𝐞𝐞𝐩 𝐁𝐒𝐃𝐄 𝐌𝐞𝐭𝐡𝐨𝐝 From Huré–Pham–Warin (2019), solves semilinear PDEs by reformulating them as backward stochastic differential equations. Trains one NN per time step (forward simulation) – robust for pricing American/early exercise options. 🧠 Widely used in quantitative finance for high-dimensional PDEs. ➡ 𝐃𝐆𝐌 (Sirignano & Spiliopoulos, 2018) Solves time-dependent PDEs by training a single NN on space–time, using Galerkin-inspired residual loss. No time stepping needed – clean and efficient on smooth solutions. ➡ 𝐓𝐃𝐆𝐅 (Papapantoleon & Rou, 2024) Reformulates PDE solving as energy minimization per time step. Very stable, compatible with higher-order schemes, great for option pricing. — ⏳ 𝗦𝘁𝗶𝗹𝗹 𝘄𝗮𝗶𝘁𝗶𝗻𝗴 𝗳𝗼𝗿 𝗮 𝗴𝗲𝗻𝗲𝗿𝗮𝗹-𝗽𝘂𝗿𝗽𝗼𝘀𝗲 𝗺𝗲𝘁𝗵𝗼𝗱? Every architecture has trade-offs: PINNs are versatile, BSDEs are robust, DGM is elegant, TDGF is precise. This paper provides practical guidelines to tune and compare solvers, beyond just theory. 🔗 https://lnkd.in/esEyvtjy

11

The following 2 papers were accepted at arXiv last week: - Benchmarking Cross-Lingual Semantic Alignment in Multilingual Embeddings ( https://lnkd.in/eNte3aQW ) - Geometric Patterns of Meaning: A PHATE Manifold Analysis of Multi-lingual Embeddings ( https://lnkd.in/ew2AFY-d )

22

𝗙𝗿𝗼𝗺 𝗠𝗼𝗻𝘁𝗵𝘀 𝗶𝗻𝘁𝗼 𝗢𝘃𝗲𝗿𝗻𝗶𝗴𝗵𝘁: 𝗔𝗰𝗰𝗲𝗹𝗲𝗿𝗮𝘁𝗶𝗻𝗴 𝗠𝗮𝗰𝗿𝗼 𝗙𝗼𝗿𝗲𝗰𝗮𝘀𝘁𝗶𝗻𝗴 𝘄𝗶𝘁𝗵 𝗠𝗔𝗧𝗟𝗔𝗕 How can economists generate reliable confidence bands for nonlinear DSGE models without waiting weeks for results? At the MathWorks Finance Conference, Kadir Tanyeri (International Monetary Fund) showed how a MATLAB‑based workflow accelerates high‑dimensional macroeconomic forecasting—cutting simulations from 100,000+ to about 3,600. Using a fully automated, nonlinear forecasting pipeline, the workflow integrates model solution, data preparation, filtering, and large‑scale simulation in one environment. 🔍 𝗛𝗲𝗿𝗲'𝘀 𝘄𝗵𝗮𝘁 𝗺𝗮𝗸𝗲𝘀 𝗶𝘁 𝘄𝗼𝗿𝗸 🔹 Nonlinear inflation dynamics with asymmetric output‑gap effects 🔹 Automated data transformation and baseline forecast generation 🔹 Latin hypercube sampling for faster convergence in a 1,680‑dimensional shock space 🔹 Distributed computing on a 128‑worker cluster to scale simulations efficiently 🔹 Single‑function execution of full nonlinear confidence‑band computation ℹ️ 𝗪𝗵𝘆 𝗶𝘁 𝗺𝗮𝘁𝘁𝗲𝗿𝘀 Macroeconomic projections—GDP, inflation, and unemployment—are only useful when paired with quantified uncertainty. Confidence bands support: 🔹 Risk‑aware policy decisions 🔹 Recession and deflation probability estimation 🔹 Clear communication of upside/downside risks As nonlinearity and dimensionality grow, traditional Monte Carlo approaches become computationally prohibitive. MATLAB provides a unified, reproducible, and scalable environment for modern macroeconomic forecasting. Have you used Latin hypercube sampling or distributed computing in your forecasting work? 🔗 𝗥𝗲𝗮𝗱 𝘁𝗵𝗲 𝗳𝘂𝗹𝗹 𝗯𝗿𝗲𝗮𝗸𝗱𝗼𝘄𝗻 𝗵𝗲𝗿𝗲: https://spr.ly/6048hCT3p #Macroeconomics #Forecasting #DSGE #ParallelComputing #Econometrics #RiskManagement

1

Stop treating your favourite LLM like a single source of truth. Andrej Karpathy just dropped “LLM Council” – a simple local app where multiple models answer your question independently, then review and rank each other’s answers, and finally a “chairman” model synthesises the final response via OpenRouter. This matters because it: - Assumes models are fallible and need cross checking - Builds critique and review into the workflow - Focuses your effort on orchestration, not hero-worshipping one provider If you are a student or early-career dev, do not build yet another single-model chat UI. Fork this pattern and ship your own “council” on a real problem in your domain. Repo: https://lnkd.in/ghj4hjJ2

28

5 Comments

Check the latest addition to our working paper series! Especially, if you are interested in mechanics of GenAI and LLMs. Very informative and insightful. https://lnkd.in/eZUUHQrG

3

1 Comment

What do Black-Scholes, NLP, and yield curve modeling have in common? More than you'd think. In his last July blog, Charlie Browne traces the surprising parallels between quantitative finance and AI, from Ito’s calculus to generative models. Why it matters: Whether you're pricing options or optimizing neural nets, both disciplines rely on probabilistic thinking, curve fitting, and risk minimization. And both are driving innovation in how we model the future. Read now: https://lnkd.in/ehGTrSZp

9

Fantastic work by the team here. So many interesting questions are being raised by the adoption of LLMs and it is only through performing novel research such as this, coupled with real-world applications and experimented of using them that we will be able to develop robust controls and guardrails that allow us to use AI safely. #responsibleAI

19

Where are Funds putting their AI-Money? Based on the current roles we're focused on, there's large investment in LLM integration for Quantitative Workflows, Time-Series Forecasting/Simulation and Reinforcement Learning/Agentic Systems for Research Specific focus on NLP within unstructured data and RL environment generation around time-series modelling, backtest, simulation and rebalancing, aswell large-scale research and analytics capability. Roles are a healthy mix of Quantitative, Software, Data and Product, all at Senior IC and all with a clear orientation toward AI/ML commercial experience. All firms are looking for hands-on individuals who can blend an exec presence with a sleeves-up approach. Vast majority of hiring sentiment is focused on ICs, not leadership. Most of the leadership roles are filled. Larger firms still favour financial backgrounds and will wait for lengthy NCs to secure the right talent. Smaller firms are open to buy-side to buy-side shift, but are more aggressively looking at MedTech, Pharma, FAANG or Tech StartUps as their source of exceptional talent. Compensation is through the roof, the battle for talent is intensifying. All very much to play for.

14

7 Comments

🚀 New Story on Medium: Hands-On with Hugging Face LLM Course – GLUE Text Classification I’ve just published a new article where I walk through fine-tuning BERT for the GLUE benchmark as part of my Hugging Face LLM learning journey. 🔹 What’s inside: Why GLUE is a gold-standard benchmark for NLP models How to fine-tune BERT for paraphrase detection (MRPC task) Lessons learned from combining Hugging Face hands-on work with Stanford CS224n theory 💡 Whether you’re starting in NLP or looking to refine your LLM skills, this project will give you a practical, end-to-end path — from dataset to evaluation. #NLP #MachineLearning #HuggingFace #BERT #DeepLearning #LLMs #AIResearch #CS224n #AI #ArtificialIntelligence 📅 Coming Next Week: Building a Retrieval-Augmented Generation (RAG) Pipeline — where we combine LLMs with external knowledge sources for more accurate, context-aware responses. Using LangChain with Hugging Face models Implementing document retrieval + embedding search Deploying a working RAG app in Hugging Face Spaces Stay tuned! 📖 Read here: [https://lnkd.in/e4nuPUTw]

10

Large language models dominate AI headlines, but in real engineering environments, something else is quietly happening: the rise of small, specialized models. In this Medium article, we take Gemma 3 1B-IT, a compact instruction‑tuned SLM, and transform it into a domain‑aware financial sentiment analyst. Instead of teaching the model only to classify text as positive, neutral, or negative, we train it to explain its reasoning, an essential skill in financial applications where transparency matters. #Gemma3 #Gemmaverse #AI #LLM #FineTuning #SentimentAnalysis #NLP

26

Interesting read. As Claude is giving answers Anthropic has been tinkering with the activations of its Neural Net in 50% of the cases and the model is to report for all test and control cases if it is noticing any concept injection into its thought processes. The latest Claude Opus model in particular reported a manipulation of its internal states for considerably more test cases with actual external intervention than reporting false positives for control cases without external intervention. The difference between the correct identification of concept injections and false positives was 20% and statistically significant: „Taken together, our experiments suggest that models possess some genuine capacity to monitor and control their own internal states. This doesn’t mean they’re able to do so all the time, or reliably. In fact, most of the time models fail to demonstrate introspection—they’re either unaware of their internal states or unable to report on them coherently. But the pattern of results indicates that, when conditions are right, models can recognize the contents of their own representations. In addition, there are some signs that this capability may increase in future, more powerful models (given that the most capable models we tested, Opus 4 and 4.1, performed the best in our experiments).“

2