Onehouse

Also doing a talk next week on "Anatomy of Our Data Agent: How AI Supports Analytics at Preset" at OpenXData by Onehouse, Wednesday, April 29 at 2:35 PT. Free, virtual. Hope to see you there! https://hubs.li/Q04dd_ql0

🔥 Don’t miss the OpenXData opening keynote. Few topics feel more important right now for data and AI teams.

⚡ What does it take to make an open lakehouse work across formats? Join us next Wednesday at OpenXData for a closer look at that question.

🔥 A must-watch on what AI-native lakehouse architecture really looks like.

What does compaction, cleaning, and clustering look like when you operate at Uber scale? At OpenXData, Uber engineers Vamshi Pasunuru and Xinli shang will share how their team built scalable table services to balance ingestion latency with query performance, and how they decouple background maintenance to keep data fresh and analytics fast. In a recent blog post, Uber noted that its Apache Hudi deployment supports 19,500 datasets, 10 PB of daily ingestion, and 70,000 table service operations per day. This talk should be especially relevant for teams running large lakehouse deployments where table maintenance directly impacts reliability and performance. Catch it at OpenXData on April 29 👉 https://www.openxdata.ai/ #OpenXData #ApacheHudi #DataEngineering #Lakehouse #DataPlatform #OpenSource

A great example of the potential impact of our Spark Jobs with Quanton acceleration.

Meta. Lyft. Amazon. Tosh Rayadhurgam has worked on AI and ML platforms at serious scale. What happens when non-deterministic agents enter such systems? That is what he will unpack at #OpenXData. Most data architectures still assume queries are deterministic. Same query in, same answer out. Agents break that assumption. 🤖 Tosh's session gets into what changes for governance, trust, and the architectural decisions teams should make before an agent layer reaches production. If agents are moving from demo to production in your stack, register here: 👉 https://www.openxdata.ai/

Optimizing a vectorized query engine is 90% analysis, 10% code changes. You run a benchmark. Parse DAG nodes, check cache hit rates, compare cold vs warm runs, capture flame graphs, monitor native memory. Then do it all over again. Dozens of times. The analysis isn't hard. It's just repetitive. So we automated it. At Onehouse, we built agents that handle the entire query execution analysis loop — DAG operator metrics, CPU flame graph analysis, native memory diagnostics down to allocator fragmentation. What used to take hours of manual analysis now takes seconds. It's not about saving time. It's about how many ideas you can test per day. When the analysis loop is fast, the bottleneck shifts from "can I analyze this fast enough" to "what should I optimize next" — which is where real engineering happens. AI doesn't optimize the engine for you. It removes the friction between having an idea and testing it. If you're looking for 4x faster Apache Spark and an engine that keeps getting faster every week try the Quanton Operator in your cluster with a one-line change. https://lnkd.in/gA37rQ_m

I am speaking at OpenXData (by Onehouse) - the Open Data Architecture event of the year! OpenXData is honestly the event to be at if your focus is technical deep dives. The speaker lineup is pretty amazing and covers an array of things such as: - AI-native data platforms - Data Engineering for AI - Cost/Performance Optimization at scale - Interoperability in Lakehouses, etc. I am doing a talk on "What is Really Open in an Open Lakehouse (ft. Apache Iceberg, Apache Hudi & Delta Lake)" Jargons like 'Open Lakehouse', 'Interoperable Lakehouse' are used everywhere nowadays, but what does it really mean for your data architecture? Join me as I do a technical breakdown of all these. Also, looking forward to these amazing talks by other folks in the list. ✅ What’s new in Spark 4.2 / 4.3 and how to optimize your UDFS in Spark 4+ by Holden Karau ✅ Column Storage for the AI era by Julien Le Dem ✅ Driving Iceberg Adoption with Open Catalog and Open Datasets by Kevin Liu ✅ Polaris Meets Hudi, Unifying Lakehouse Metadata Across Table Formats by Yufei Gu ✅ Apache Hudi™ for the next generation of AI: Unstructured Data and Vector Search on open lakehouse storage by Rahil Chertara & Timothy Brown Link in comments! #dataengineering #softwareengineering

Spark on Kubernetes has always come with a frustrating tradeoff: You want the control, security, and flexibility of running on your own infrastructure. But you do not want to give up the kind of performance you get from a more managed stack. What if you could get both? 🚨 Join us for: 4x Faster Spark on Your Own Infrastructure: Bringing Quanton to Self-Managed Kubernetes. 📅 Apr 14, 2026 ⏰ 9am PT We’ll show how teams can run Spark workloads 3 to 4x faster on self-managed Kubernetes, with zero code changes. In the session, we’ll walk through: 1. what Quanton K8s Operator actually does 2. how to install and configure it 3. how to use Spark Cost Analyzer to estimate savings from your existing Spark history logs So if you are running Spark on K8s and tired of choosing between performance and infrastructure control, this one is for you. Performance you need. Infrastructure you control. Zero code changes. Register here 👇 #ApacheSpark #Kubernetes #DataEngineering #Spark #DataInfrastructure