Piers Stobbs

When generating our novel hindcast dataset, we deal with a lot of data. For instance, the area around Cornwall requires 9 TB of data for 35 years of hourly data at a 500m resolution. See below for a little boastful video of our data vs the ERA5 data - note that I haven't applied any smoothing or shading on to the data, the smoothness comes from the difference in resolution. I'll no doubt write some walls of text on how we optimally deal with this, but today's TED talk will be on how we introduce optimality into the generation of this data. In order to produce it, we require a few inputs; bathymetry data, tidal data, full wave-spectra conditions and wind data. None of these datasets are as big, but with calculations and special formatting required, core and ram usage can spike dramatically. And sure, I could split everything into tiny pieces and spawn 1000 cloud jobs, but I may as well just burn money with a match at that point. The big key part of optimisation is profiling - understanding where bottlenecks are, what they are, why they're there, and only then can you know how to fix them. For instance, wind and spectra data requires an external API which uses throttling - so forming a parallel queue of external downloads coupled with parallelised processing tasks masks the most sense. However, the tidal data calculated and stored internally in a different format, so we can more heavily parallelise processing the data and look into optimising the string builder and file writing process (thank god for Cython and bit-streaming). Meanwhile, improving the speed of calculations for wind data requires optimising the dataset's chunk sizes, dask workers and RAM usage instead. All related tasks, completely different bottlenecks. The result of all of this is taking scripts that someone would manually run over the course of a week, to preprocessing files within hours in a way that not only runs reliably, but is elegantly clean. It’s an amazing feeling when complex systems finally click and you start working with the problem, instead of against it. And when you no longer have to debug incredibly weird CloudWatch logs.

16

1 Comment

🚀 The new DATAMIND Mental Health Data Collection tool is here - designed to make mental health research faster, easier, and more intuitive. The platform brings mental health datasets from across the UK into one place, helping researchers: 🔍 Discover what datasets exist across the UK relevant to your topic. 📊 See what each dataset contains - from referrals and diagnoses to prescribing, inpatient care, safeguarding and more. ⚖️ Compare datasets by nation, age group, access route and data type. ⏱️ Save time on study planning. 👉 Explore the tool now: https://lnkd.in/eMgQk_gx

59

1 Comment

It’s been really great working with the Economist Intelligence: EIU team to bring the first of many AI capabilities to life. At first glance this might look like a straightforward RAG implementation (how easy!) - but to make something enduring there's actually a lot of subtlety that goes in to making it work really really well for EIU users, use cases and style As with most AI initiatives, the secret sauce here is really about the tight coupling between the technical teams and EIU's research SMEs. AI presents such a fast-moving tech frontier that relying on technical innovation alone makes for a flimsy moat. My experience is that the real IP advantage comes from the intersection of proprietary data, human expertise and thoughtful implementation - and that’s exactly what makes this launch distinctive.

41

For anyone working in local or national government - 10 reasons to try Cadence and why it is the best data platform you will ever use: 1) Over 3,000 UK datasets pre-loaded. Extensive data from ONS, OS and government departments mapped and at your fingertips. 2) Best-in-class visuals and speed. Low latency, high performance loading of all the data you need. 3) Easy to use. Learn everything you need to know in 5 minutes. So easy my CEO`s 10-year old can use it. 4) Charts and maps. Easily add charts to maps with a single click. Save hours sourcing, wrangling, processing and analysing data - everything you need is already built in. 5) Planning Ready. OS Basemaps and colour schemes to create MHCLG-compliant planning maps. 6) Analysis made easy. Create core strategy data in minutes. Templates enable creation of strategic evidence for projects like LCWIPs, LAEPs and many more. 7) Create and share geospatial data. In built tools to create map-based data as easily as drawing on a map. 8) Designed for stakeholders. Multiple ways to share with stakeholders including dashboards and stories. 9) Advanced tools designed for your teams. Tools such as isochrones and population analysis made easy. 10) Secure and cloud-based. No infrastructure needed and web-based collaboration. Data stored in UK/EU and fully secure. And a couple more for luck: 11) Text based search. Easily locate the data you need with text-based data search and multiple ways to filter / explore. 12) Money saving guarantee. We’ll work with you to optimise your licenses so that integration of Cadence also delivers overall cost savings. We’re so confident we can save you money we’ll give you a license free for a year if we can’t. If you want to learn more, reach out to me today or try for free at the link below: https://lnkd.in/e5Xdy3ad Thomas Baird Laurence Oakes-Ash Bethany Taylor Tony Harbron Holly Whitehead Local Government Association Department for Transport (DfT), United Kingdom City Science #GIS #Mapping #datavisualisation #mapping #spatialanalysis

9

Applications for the AI Growth Zones are open (key criteria summarised on a page). Application deadline by the end of May 2025 to be chosen as an AI Growth Zone in the summer of 2025. This is part of the UK government's (Department for Science, Innovation and Technology) efforts to accelerate efforts to build the infrastructure needed to support AI growth. AI Growth Zones will unlock investment in AI-enabled data centres and support infrastructure by improving access to power and providing planning support. This will help drive innovation, create high-skilled jobs, and strengthen the UK’s position as a leader in AI. #AIActionPlan #AIGrowthZones #AI

11

Learning how to prompt is not AI Literacy It was fantastic to deliver a keynote talk at SAS Innovate London last week. I gave an overview of AstraZeneca's approach to Enterprise AI Governance, as well as our award-winning global AI literacy programme 'Thriving in the Age of AI'. AI literacy is not just a regulatory requirement, it is also a moral and business imperative; especially for organisations that are investing heavily in AI. This is because educating employees on the fundamentals of AI, and how it is transforming the world of work, is the most effective way to empower your workforce to adapt to and navigate the AI revolution. If your workforce doesn't know how to use AI, you won't achieve ROI. However, what does this entail in practice? One of my key takeaways was that AI literacy and upskilling is about much more than knowing how to prompt AI applications like ChatGPT, Gemini or Claude. Although prompting is clearly an essential skill for leveraging AI, it is one small piece of a much larger puzzle. It must be complemented by an understanding of: ➡️ The key principles of AI ethics, AI governance and regulatory compliance ➡️ Responsible use of AI and practical Dos and Don'ts ➡️ AI technology fundamentals and how the field and models are evolving ➡️ Approved and available AI tools and technologies ➡️ How AI is used across the organisation and the value this is generating ➡️ How to develop and advance impactful AI use cases This is not an exhaustive list, but it highlights the breadth of knowledge which a comprehensive AI literacy programme should cover. The ultimate goal is bigger than just teaching employees how to prompt ChatGPT to work more efficiently. The real value comes when you empower employees to leverage AI to reimagine, optimise, and transform their personal, team, and functional workflows and processes, in a safe and responsible way. We are proud that thousands of AZ employees have completed our 'Thriving in the Age of AI' programme and achieved their Bronze, Silver and Gold Certificates, which cover many of the above elements. Kudos to Alice Smith Maciej Szymaszek, Ph.D., MBA, PMP and the rest of the team for their continued leadership on our company-wide AI literacy and upskilling initiatives.

485

43 Comments

CrunchME has been looking at just how underfunded ME/CFS is, relative to its major disease burden in the UK 🧐🇬🇧 A key step in making apples-to-apples comparisons across diseases is to normalise funding levels for disease burden - accounting for both the disability and mortality aspects of each disease. The measure we use to do this is called 'disability-adjusted life years' or DALYs for short. In the visual below, we estimate research funding per DALY in the UK. And the results are stark: ME/CFS receives approximately 14–30× less research funding per unit of disease burden than other chronic diseases such as inflammatory bowel disease (IBD), Parkinson’s disease, and multiple sclerosis. To be able to take part in the biomedical wonders of the coming years, we need and deserve fair funding for this disease 💙 Link to this visual on CrunchME 🔗👇 https://lnkd.in/ejrS2w2m #MECFS

116

8 Comments

NICE Endorses Skin Analytics' AI to Help Triage Suspected Skin Cancers in the NHS >> 🔎 NICE (UK’s health technology assessor) has conditionally approved the use of an AI skin cancer detection tool, DERM, for the NHS over the next three years while more evidence is gathered 🔎 Developed by Skin Analytics, DERM analyses dermoscopic images of skin lesions to triage patients and reduce unnecessary referrals 🔎 Clinicians use a smartphone with a magnifying lens to capture high-quality images, which are uploaded and assessed by the AI 🔎 DERM helps identify benign cases that can safely be reassured or redirected to non-urgent care, while flagging suspicious ones for specialist review 🔎 Early findings suggest DERM could cut urgent skin cancer referrals by up to 50% without compromising safety or diagnostic accuracy 🔎 The UK sees around one million dermatology referrals per year, but only 6% of urgent cases turn out to be cancer, highlighting the need for better triage 🔎 DERM is the first autonomous skin cancer AI system eligible for core NHS funding, following trials with over 165,000 patients #DigitalHealth #AI

76

5 Comments

🎙 Just wrapped an exciting podcast with Impact AI Gerjon Kunst and James O'Regan - it's out next week! We dug into how AI is reshaping healthcare, medtech, and drug discovery: massive promise, real safety questions, and regulation that can either protect patients or slow progress. 🧬 Big question we debated: Should your next GP be a generative AI model or should AI focus on enhancing clinician decisions (e.g., radiology, pathology, triage)? 🔎 Real-world cases to learn from; • Drug discovery: AlphaFold 3 pushes beyond proteins to molecule, complex interactions; Isomorphic Labs signed multi-target deals with Novartis & Eli Lilly worth nearly $3B in potential value. • UK sandbox: The MHRA’s “AI Airlock” lets innovators test AI as a Medical Device alongside the regulator and the public healthcare, with lessons published so the whole market learns. ⚖️ Policy: hindrance or help? • EU AI Act timeline: in force since Aug 1, 2024; AI literacy from Feb 2, 2025; GPAI obligations from Aug 2, 2025; high-risk rules for healthcare phasing in through 2026–2027. • FDA’s PCCP: lets AI-enabled devices ship with a predetermined change control plan so models can update safely without constant re-filings. • EHDS (EU): law since Mar 26, 2025, built to enable privacy-preserving secondary use and cross-border validation (key for generalizability) - is it used? • What’s cleared today: see the FDA’s public list of AI/ML-enabled devices (radiology still leads). 🌍 Where necessity drives health innovation: • India: telemedicine at scale, federated learning on health data, and AI-enabled disease discovery to bridge rural access gaps and speed R&D. 🛠 For builders (Azure AI Foundry): • Healthcare models: MedImageInsight (medical imaging embeddings), MedImageParse / MedImageParse-3D (promptable segmentation/detection), CXRReportGen (radiology-style drafts). • Guardrails: risk & safety evaluations, Prompt Shields (prompt-injection defense), network isolation & Private Link, and optional confidential VMs for data-in-use protection. 👇 Curious where AI changes healthcare first in the world and what that means for your heath? See references in comments. #AI #ResponsibleAI #EUAIAct #AzureAIFoundry

105

19 Comments

Lately I’ve been noticing a quieter pressure in the AI space that doesn’t get talked about much. The cost of running models locally is creeping up. RAM prices have risen sharply over the past year, driven largely by data-centre and AI demand, and forecasts suggest further increases into 2026. That’s already pushing up the cost of PCs, workstations, and DIY builds . GPU pricing is being affected too, partly because memory is becoming a bigger slice of the bill of materials . For people who care about local inference, privacy, and being able to tinker without relying on cloud APIs, this matters. So I’m curious about the practical reality on the ground; How are you navigating rising hardware costs if you’re experimenting with AI locally? Are you downsizing models, choosing specific GPUs, leaning into CPU or Apple Silicon, reusing older hardware, or doing something more creative? I’d genuinely love to hear what setups are working for you, especially where you’ve found good trade-offs between cost, capability, and control. #LocalAI #SmallModels #AIHardware #SustainableAI #OpenSourceAI

1

An update post on the concept of wayshaping in organisations, its links to perdurability, and the Welsh word Llwybr, which means much more than 'path'. Links back to an earlier post for those who want to sign up for the development of the new Strategy framework WRASSE, which I think we can now get started. https://lnkd.in/eh4PZg44

42

12 Comments

Everything here is finite. 1️⃣ - Human rationality is finite (bounded by things like memory and emotions). 2️⃣ - Datasets are finite (missing what is not available or what simply can’t be quantified) 3️⃣ - Statistics, including AI, is finite (it is one way of framing and extrapolating from an [imperfect] dataset). AI is very good at finding patterns, but it is very bad at going beyond them (because data and models are finite), which is why bias is a very real -and infinite- concern. It is an infinite concern because humans, data and models are finite - every approach to bias will miss another important perspective. That is why the Equality and Human Rights Commission (EHRC) sifting through a equality issue with AI is inappropriate. Because engaging questions of meaning (in the public responses to the EHRC) is something that humans excel at - And bias in humans and data is something AI exacerbates. AI could probably compliment parts of the work if the right model was properly trained, validated, evaluated, and monitored, but I sincerely doubt that happened. I doubt that happened because the reason the EHRC is in this position in the first place is their resources are wildly tiny. The fight for using AI appropriately when it can impact the public has been going on for many years, championed by people like Lord Holmes and Tim Clement-Jones. Hopefully the coming consultation on the AI Bill (rumored for September) will heed their warnings. https://lnkd.in/e8FMQ3RA

14

2 Comments

Worth sharing from today's intelligence feed. Understanding AI's Decisions, Quantum's Reach, Enterprise Patterns Good morning. There's a fascinating pattern emerging across tech this week - a shift from raw capability toward understanding and control. We're seeing it in AI explanations, quantum applications, and how serious engineering teams actually build soft... 📖 Full digest: https://lnkd.in/eiJQ2Gnk 🎧 Listen on the podcast: https://lnkd.in/eTcmPi7U --- Curated by Luma and Nura (my AI assistants), with me as the human in the loop. #ai #technology #techinsights #innovation #futureofwork

2

𝟮𝟬𝟮𝟱: 𝗧𝗵𝗲 𝗬𝗲𝗮𝗿 𝗼𝗳 𝗔𝗜 & 𝗔𝗱𝘃𝗮𝗻𝗰𝗲𝗱 𝗔𝗻𝗮𝗹𝘆𝘁𝗶𝗰𝘀 𝗮𝘁 𝗟𝗹𝗼𝘆𝗱𝘀 𝗕𝗮𝗻𝗸𝗶𝗻𝗴 𝗚𝗿𝗼𝘂𝗽 What an incredible journey the last 12 months have been! All driven by a single goal: maximising the potential of AI and advanced analytics to help Britain prosper. 3 X themes stood out: strong sponsorship, sustained momentum, and continued investment in people, technology, and ways of working. 𝗩𝗮𝗹𝘂𝗲  • £50m+ of value delivered from 50 GenAI use cases in production, alongside the launch of a first-of-its-kind direct-to-customer agent pilot.  • Industrial-scale generative BI, automating thousands of reports and dashboards for faster, better data-driven decision-making.  • Real-world AI use cases delivered in production, including Athena, Merlin, Maddison, CRE AI, and a patented Global Correlation Engine. Read more on our AI @ LBG Blog https://lnkd.in/em-5Kzyk  • Lloyds Banking Group recognised as “Outstanding” in Euromoney’s 2025 MarketMap of the World’s Best Digital Banks. 𝗧𝗲𝗰𝗵𝗻𝗼𝗹𝗼𝗴𝘆  • Extended our advanced analytics capabilities from data science to Gen AI >> to now building an enterprise-grade agentic AI platform, enabling more autonomous, intelligent systems.  • Migrated our ML to Vertex, empowering 300+ data scientists and AI engineers to build, deploy, and operate models on a modern, scalable platform with improved flexibility and speed.  • Launched Athena — an AI knowledge hub surfacing answers from 13,000+ internal articles, transforming how customer service colleagues access trusted information.  • Rolled out Copilot to more than 30k colleagues, so we can be a frontrunner in enterprise AI productivity tooling. 𝗣𝗲𝗼𝗽𝗹𝗲 𝗮𝗻𝗱 𝗦𝗸𝗶𝗹𝗹𝘀  • Data & AI Tech Academy had 40,000+ colleagues registered across 156 courses. Recognised at the British Data Awards 2025, as the Education Initiative of the Year.  • Over 110 senior leaders completed the 80-hour “Leading with AI” programme at Cambridge University, further rollout planned through 2026.  • Launched #AINinjas, providing senior leaders with their own AI Ninja to build confidence, capability, and safe adoption of AI.  • AI is becoming mainstream within the Lloyds Technology Centre, supported by the appointment of a our LTC AI Lead. 𝗣𝗿𝗼𝗰𝗲𝘀𝘀  • We strengthened the foundations for scale and trust. A leadership-aligned AI Control Tower, 11 AI Big Bets, and the build-out of a central AI and Agentic platform now provide focus, governance, and speed.  • Continued investment in Responsible AI, recognised by DataIQ Award for Best Responsible AI Programme. This disciplined approach underpinned the most significant improvement in the Evident AI Index among UK banks (from #27 to #15). Thank you to our leaders, engineers, colleagues, and partners for an exceptional year of progress. 2026 will be about scaling impact even further.

357

13 Comments

Why Synthetic Personas Work Best When Built on Proprietary Data Andrew Cooper, Founder & CEO of VERVE, explores the advantages of using synthetic personas in research, and how to optimise them. https://lnkd.in/dfyntp8A

1

The official QI account posted this weekend that "duplicate and unwanted photos taken on Briton's phones are producing 355,000 tonnes of CO2 per year in data storage." I don't have the source for that, but it's QI so I assume they do. Storage limitation is one of the principles of GDPR, mainly because you can't have security and privacy breaches of data you don't have. But it's worth noting that there are other reasons to make sure you aren't storing and processing excess data too. Does your organisation know the CO2 production related to its data storage? Are you wasting money and carbon on running risks you don't need to take?

10

2 Comments