Datacenter Management Practices

Every developer should know that tenant isolation is not a database problem. It’s a blast-radius problem. I learned this the hard way. One missing tenant filter. That’s all it takes to turn a normal deploy into a security incident. Every multi-tenant system eventually picks one of three isolation levels. Each one trades safety, cost, and operational pain in different ways. 1. Database per tenant This is the strongest isolation you can get. Each tenant lives in its own database. No shared tables. No shared state. The upside is obvious. A bug in one tenant cannot leak data from another. Audits are simpler. Compliance conversations are shorter. When something breaks, the blast radius stays small. The downside shows up later. Operational overhead grows fast. You manage hundreds or thousands of databases. Migrations become orchestration problems. Costs scale with tenant count, not usage. This model works when tenants are large, regulated, or high-risk. It breaks down when you try to apply it blindly to long-tail customers. 2. Schema per tenant This is the middle ground most teams underestimate. All tenants share a database, but each one gets a separate schema. Tables stay isolated, but infrastructure stays manageable. You get clearer boundaries than row-level isolation. You avoid the explosion of databases. Audits remain reasonable. Most accidental data leaks disappear. But complexity still creeps in. Migrations must run across many schemas. Cross-tenant reporting becomes awkward. Automation is not optional anymore. Without it, this model collapses under its own weight. This approach works well when tenants vary in size and you want isolation without full separation. 3. Row-level isolation This is the cheapest and most dangerous option. All tenants share the same tables. Isolation lives in a tenant_id column and your queries. Infrastructure stays simple. Costs stay low. Scaling is easy. The risk is brutal. One missing filter equals a data leak. One refactor can break isolation. One rushed hotfix can expose everything. Security depends on every layer doing the right thing every time. This model only works when you add heavy guardrails: strict query scoping, database policies, service-level enforcement, and tests that actively try to cross tenant boundaries. Without those, you’re betting the company on discipline. Tenant isolation is not a storage choice. It’s a trust decision. Learn this, it's a classic Interview question.

AI adoption is accelerating faster than the energy systems built to support it. Data centers are already among the most power-intensive assets on the grid and are seeing demand rise at rates that legacy infrastructure, static operating models, and fragmented regional grids were simply not designed to handle. The consequence is predictable: higher costs, growing emissions, and mounting pressure on utilities and operators trying to maintain reliability while integrating renewables. I’ve spent much of my career working at the intersection of technology, energy policy, and industrial systems, and this challenge is proving to be one of the defining infrastructure questions of the decade. It’s increasingly clear that the sector needs new ways to manage load, forecast demand, and coordinate resources across highly variable conditions. This week, I had the opportunity to hear from senior leaders at Hanwha Qcells about a model they are developing that aims to address these pressures. What stood out to me was the architectural shift behind the technology: using AI, interoperable language, and digital twins to unify diverse equipment, link operations to real-time grid signals, and automate many of the repetitive, checklist-style decisions that currently consume operator time. This broader concept of treating data centers as intelligent, grid-aware assets aligns with conversations happening across industry and government. The framework they described integrates clean generation, storage, and control software into a single adaptive system. The goal is straightforward but ambitious: reduce wasted energy, cut emissions, and improve resilience as AI demand grows. Their lofty projections (20–30% cost reductions, up to 35% emissions cuts, faster response times through agentic operations) reflect why approaches like this are gaining momentum. What interests me most is how these ideas fit into the larger trend: the shift toward an “Intelligent Age” where digital growth and energy management are inseparable... remember when VPPs were unheard of? Solutions that improve transparency, interoperability, and operational flexibility will be essential, and not just for data centers, but for manufacturing, transportation, and other power-intensive sectors facing similar constraints. As we look ahead, the real opportunity is in building systems that scale, adapt, and operate with far greater situational awareness. The conversation with Qcells underscored how quickly this space is evolving and why collaboration across utilities, technology developers, operators, and policymakers will be critical in the years ahead. Article link: https://bit.ly/4qggMLd #Hanwha | #HanwhaQcells | #Microsoft | #AI | #DataCenters | #EnergyManagement | #GridModernization | #CleanEnergy | #Innovation

Operating our data centers more sustainably means being thoughtful about every step - including the materials we use for things like circuit boards and hardware devices. Copper is one of those essential materials, and now there’s a way to source it that supports our goal of The Climate Pledge. Amazon Web Services (AWS) is the first buyer of copper produced from Rio Tinto's innovative Nuton technology. It's a breakthrough process that uses microorganisms - or "bioleaching" - to extract copper from sulfide ores (which are traditionally hard to process and often become waste). Why does that make a difference?   It removes the need for traditional concentrators, smelters, and refineries. The process uses up to 80% less water usage than traditional mining methods. It also has a carbon footprint well below the global average. It significantly shortens the mine-to-market supply chain.   This innovation is another example that solutions exist, and forward momentum continues. Amazon is working across our entire value chain - from steel and concrete to copper - to source materials differently, and I'm thrilled to see AWS leading the industry in the right direction! Learn more about our work on copper in this The Wall Street Journal article by Ryan Dezember: https://lnkd.in/g9AgshDn  

Fortune 500 CEOs are finally learning about the 7 Deadly Sins of Data Centres. But do you know them? I've built infrastructure on five continents. Spent billions on cutting-edge tech. But the very same demons that haunted my first server room in 1996 are alive and thriving in today's AI data centres. We just dress them up better now. 𝗧𝗵𝗲 𝗦𝗲𝘃𝗲𝗻 𝗦𝗶𝗻𝘀 𝗦𝘁𝗶𝗹𝗹 𝗥𝘂𝗻𝗻𝗶𝗻𝗴 𝗬𝗼𝘂𝗿 𝗗𝗮𝘁𝗮 𝗖𝗲𝗻𝘁𝗿𝗲 1. 𝗣𝗿𝗶𝗱𝗲: "Our infrastructure is world-class" → Translation: We spent $2B and still can't cool a single AI rack → Reality check: Your 5MW facility is now a 50MW monster (overnight) → Tier 4 certified but running like Tier 0 when GPUs arrive 2. 𝗚𝗿𝗲𝗲𝗱: More racks, more power, more everything! → 100kW per rack demands (was only 5kW in 2010) → Hoarding grid capacity (while towns go dark) → Fighting over the last transformer (like it's Black Friday) 3. 𝗪𝗿𝗮𝘁𝗵: The blame game when systems fail → "It's the vendor's fault" (100 cable breaks annually) → "The grid can't handle us" (you knew this 3 years ago) → Finger-pointing while customers lose millions per minute 4. 𝗘𝗻𝘃𝘆: Watching competitors' announcements → Google's nuclear deals making you sweat → Microsoft's $80B war chest keeping you awake → Amazon uses its own chips while you sit in Nvidia's waiting room 5. 𝗚𝗹𝘂𝘁𝘁𝗼𝗻𝘆: Consuming electricity like Pac-Man on steroids → One ChatGPT query = 10x a Google search → Data centres will eat 9% of US power by 2030 → Water consumption that'd make farmers weep 6. 𝗦𝗹𝗼𝘁𝗵: Moving like dial-up in a 5G world → 5-year power agreements for 5-month AI cycles → Still air-cooling when liquid is mandatory → Committees debating while Rome burns 7. 𝗟𝘂𝘀𝘁: An insatiable hunger for the next shiny thing → Quantum-ready (but can't handle current load) → Edge everywhere (while the core melts down) → Chasing trends while foundations crumble 𝗧𝗵𝗲 𝗨𝗻𝗰𝗼𝗺𝗳𝗼𝗿𝘁𝗮𝗯𝗹𝗲 𝗧𝗿𝘂𝘁𝗵 We've built "Cathedrals of Compute" with stone-age sins. → $1 trillion in infrastructure investment coming → Still making the same mistakes, just faster → The sins scale with the servers 𝗕𝗼𝘁𝘁𝗼𝗺 𝗟𝗶𝗻𝗲 Infrastructure evolution didn't kill human nature. We're running 2025 hardware with 1995 habits. And the data centre demons? They've now learned to speak AI. 𝗬𝗼𝘂𝗿 𝗧𝘂𝗿𝗻: Do you still think infrastructure alone will save us? Which of these sins is harming your data centre strategy? ♻️ Repost if you recognise these demons in your facility ✅ Follow me, Guy Massey, for infrastructure truths nobody else will tell you

Great to be back in London on Bloomberg Daybreak Europe yesterday, where I discussed the rapidly expanding data center industry with Tom Mackenzie and how – with the growing energy demands of AI workloads – we keep the focus on energy management and efficiency. Global demand for data center capacity could more than triple by 2030. And we need data centers in order to drive digital transformation. Our focus is to make them more sustainable and efficient with connected infrastructure. AI and other technologies housed in data centers help to optimize processes and reduce emissions in industries, presenting opportunities for us to improve efficiency and make progress towards sustainability targets. While data center energy consumption is expected to grow, their carbon footprint is predicted to remain stable or decrease due to efficiency and clean energy adoption. Schneider Electric’s position is that we need three key strategic pillars: 1️⃣ developing an energy strategy for the AI era 2️⃣ deploying advanced infrastructure 3️⃣ providing comprehensive sustainability consulting Currently, the sector accounts for around 2-4% of total electricity consumption in the EU and this is only set to grow. Schneider Electric is involved in many projects and innovations supporting data centers to manage energy consumption and costs. For example, with our EcoStruxure technology, EcoDataCenter built the world’s first climate-positive data center in Sweden, and the Wellcome Sanger Institute in the UK cut its data center energy use by 33%. You can watch the interview here: https://lnkd.in/eWJJaqjz It was also brilliant to spend time with our UK team in our newly upgraded Central London offices. Following the announcement of our £42 million investment in a new manufacturing facility in the north of England at the end of last year, I’m looking forward to more exciting projects and collaborations across the UK in 2025. Kelly Becker #Europe #Competitiveness #Innovation #Sustainability

Efficiency isn't sustainable if it incentivises trade-offs that harm the planet. The 🇪🇺 proposed data center sustainability rating scheme overlooks a massive energy-saving gap. We risk penalising tech that could save 10,000 households’ worth of energy. Getting the "𝐖𝐚𝐭𝐞𝐫-𝐄𝐧𝐞𝐫𝐠𝐲 𝐍𝐞𝐱𝐮𝐬" right is essential for Europe’s twin transition. As we scale digital infrastructure at Google, our focus remains on energy efficiency and "climate-conscious" cooling. However, the current EU Sustainability Rating Scheme proposal contains a significant oversight regarding water usage. 𝐓𝐡𝐞 𝐂𝐨𝐫𝐞 𝐈𝐬𝐬𝐮𝐞: The scheme risks penalising highly energy-efficient evaporative cooling, even in regions where water is abundant and sustainably sourced. 𝐓𝐡𝐞 𝐔𝐧𝐢𝐧𝐭𝐞𝐧𝐝𝐞𝐝 𝐂𝐨𝐧𝐬𝐞𝐪𝐮𝐞𝐧𝐜𝐞𝐬: 🧊 𝑬𝒏𝒆𝒓𝒈𝒚 𝑰𝒏𝒆𝒇𝒇𝒊𝒄𝒊𝒆𝒏𝒄𝒚: Evaporative cooling can save the annual energy equivalent of ~10,000 households compared to dry-coolers for an average data center. ⚡ 𝑮𝒓𝒊𝒅 𝑷𝒓𝒆𝒔𝒔𝒖𝒓𝒆: Forcing a shift to waterless, energy-intensive cooling adds unnecessary strain to Europe's electrical grids and increases carbon emissions. We are calling for a more holistic, two-pronged approach: 1️⃣ 𝑨 𝑾𝒂𝒕𝒆𝒓𝒔𝒉𝒆𝒅-𝑨𝒘𝒂𝒓𝒆 𝑺𝒄𝒂𝒍𝒆: Metrics should recognise local ecological realities. We must disincentivise water-based cooling in water-stressed regions while allowing it where it significantly reduces a facility's energy use and carbon footprint. 2️⃣ 𝑮𝒓𝒂𝒏𝒖𝒍𝒂𝒓 𝑫𝒆𝒄𝒂𝒓𝒃𝒐𝒏𝒊𝒔𝒂𝒕𝒊𝒐𝒏:To truly reach net-zero, renewable energy consumption should be matched in real-time, every hour of the day, on the same grid as the data center By refining these metrics, the EU can ensure the rating scheme rewards truly resource-efficient infrastructure and keeps digital leadership in lockstep with climate objectives. Check out the link to our consultation response in the first comment ⬇️ And let me know your views on the topic. #Sustainability #DataCentres #CleanEnergy #DigitalInfrastructure #WaterEnergyNexus

New Data Center Projects Embrace Recycled Water A growing number of new datacenter projects plan to use recycled water instead of drinking water to reduce their impact on local water resources. Amazon Web Services plans to expand its use of recycled water to over 120 data center locations by 2030, which it says will preserve 530 million gallons of drinking water each year. AWS is currently using recycled water in about 20 data centers, but plans to include it in future campuses in Georgia and Mississippi. In Arizona, the proposed Project Blue data center in Tucson plans to build an 18-mile high-capacity pipeline to bring reclaimed water from a Tucson Water plant. The system, which the development team is funding, is designed to include extra capacity to help reduce the city’s overall potable water demand. Recycled water is hardly new in the data center industry. Some prior examples: - In Ashburn, Loudoun Water has been supplying reclaimed water to local data centers since 2013. - Google built its own water recycling plant in a facility near Atlanta. - Microsoft paid $31 million to help build a water recycling near its cloud campus in Quincy, Washington. - In 2021 Switch announced plans to build a 16-mile pipeline to bring recycled water to its Reno campus. But with growing focus on water scarcity and AI’s impact on local resources, its good to see data center operators include water recycling - along with closed-loop cooling systems that reduce overall usage - as part of a broader approach to water stewardship.

Amazing! “The story began with Emerald’s Phoenix load flexibility pilot, involving Oracle, Nvidia, Emerald AI, and the utility Salt River Project, and also a DC Flex flagship demonstration. The leap to the Aurora announcement, a live innovation hub, signals that the tech ecosystem is serious about getting this done. AI factories can align with grid needs to relieve peak stress and improve utilization of the power network. It will work like this: Several software and hardware features will work together to enable a tight coordination between the grid and the data center’s controls, with Emerald AI’s platform serving as the grid-facing control layer. Grid and operator conditions feed into Emerald, which translates them for the data center building’s management systems and ultimately, the compute stack. In tech speak, Emerald’s GridLink and Conductor integrate with Nvidia’s AI Enterprise stack and Mission Control to coordinate workload scheduling and power management so the facility can dial demand when the grid needs it — while maintaining acceptable Quality of Service for training and inference. To validate this, EPRI’s DCFlex Initiative will run demonstration testing, measuring precise, real-time responses to simulated grid-stress events like summer heatwaves or sudden drops”

When I was at Airbnb, I reduced the pricing and availability data sets to 3% their original size! This removed a few petabytes from the cloud and made Jeff Bezos cry. How did I do this? 1. I recognized that listing and listing night information should be in one table not two. This meant adding an ARRAY<STRUCT> to a listing-level dimensional table. The struct modeled the night-level information. 2. I consolidated the definition of availability to one agreed upon thing where before there were many different definitions of availability. 3. I had all downstream dependencies migrate from the listing night table to the listing level table and the new definition which deprecated tons of tables. These three things allowed for many benefits. One of the biggest being downstream JOINs wouldn’t cause shuffle that would break the parquet file format compression. I break this down further in this 45 minute YouTube video: https://lnkd.in/gsUmQNub

Your application’s uptime depends on equipment your code will never see. A switch. A firewall. A storage system. A power supply. This rack is a useful reminder of how much has to work before a single API request succeeds. Whether you build backend services, data pipelines, or AI systems, these fundamentals matter: • Patch panel: Organizes cable connections so teams can trace, maintain, and change them. • Switch: Connects devices within the network and moves traffic between them. • Firewall: Controls which network traffic is allowed through, based on security rules. • Servers: Run applications, databases, virtual machines, and processing workloads. • NAS: Provides shared storage that devices access over the network. • Storage: Holds the data your applications depend on. Performance and availability matter alongside capacity. • Load balancer: Distributes requests across servers, often using health checks to avoid unhealthy instances. • UPS: Provides temporary backup power during interruptions, allowing equipment to keep running briefly or shut down safely. • PDU: Distributes electrical power to equipment inside the rack. But the most useful engineering lesson here is about shared dependencies. Five application servers can still go offline together if they depend on the same failed switch. Multiple services can become unavailable when their shared storage stops responding. An entire rack can lose power if its power path has no working backup. Adding more servers only improves resilience when you also examine what they share. That thinking carries directly into cloud architecture: availability zones, storage dependencies, network paths, and failover design. When reviewing a system, ask: “If this component fails, what else becomes unavailable?” The answer often reveals more about reliability than the number of servers you have.