When you first learn to program, memory seems pretty straightforward. You have a bunch of bytes, each with an address. If you want to read or write something, you specify the address, and the computer goes to that place and does your bidding. Sure, some addresses might be reserved for hardware or the operating system, but the general idea is simple: memory is a big array of bytes.
Just like we pretend that a wire is a perfect conductor, the big array of memory model used to be a slight twisting of the truth. On a modern computer, however, it is almost entirely a lie. The address you use probably isn’t a physical address. The data you read might not come from RAM at all. Even if it does, it might be sitting in one of several caches. And if you follow the rabbit hole far enough, you discover that your SSD is arguably just another level in the memory hierarchy.
The Need For Speed
The fundamental problem is that processors have gotten much faster than memory. A modern processor can execute several instructions per clock cycle, but accessing ordinary DRAM can take hundreds of cycles. If the processor had to wait for RAM every time it needed data, much of that impressive computing power would go to waste.
The solution is caching. Keep copies of frequently used data in smaller, faster memories closer to the processor. Most modern desktop processors have at least three levels of cache. The L1 cache is tiny but extremely fast, and often split between instructions and data. L2 is larger and somewhat slower, while L3 is even larger and usually slower as well.
Things get interesting with multiple cores, as is often the case these days. Typically, each core has its own L1 and L2 caches, while several cores share an L3 cache. But that’s not universal. Some processors share L2 caches between cores. Others have multiple L3 caches, each serving a group of cores. The exact arrangement depends on the processor architecture.
One important detail: caches don’t generally work byte by byte. They transfer and track blocks called cache lines, commonly 64 bytes on modern desktop processors. If you read one byte that isn’t cached, the processor generally fetches the entire line containing that byte.
This is why sequential memory access is often much faster than random access. If you read the next byte, there’s a good chance it’s already in the cache. Hardware prefetchers may even anticipate what you’ll need next.
But there can be problems. Imagine two cores modifying completely different variables. If those variables happen to occupy the same cache line, the cores may repeatedly invalidate each other’s copies, wasting time even though they aren’t logically sharing data.
Protocols such as MESI keep the caches coherent, coordinating which core owns a line and which copies are valid. MESI is an acronym for cache line states: modified, exclusive, shared, or invalid. Memory systems must figure out how to keep caches in sync. But coherence doesn’t eliminate the need for synchronization or memory-ordering rules in multithreaded programs. The point is that even our simple byte-addressed memory is secretly operating in larger chunks.
The Address Isn’t Real, Either
So far, we’ve assumed that the address you supply identifies a location in RAM. Usually, it doesn’t. Modern operating systems use virtual memory. Your program sees a virtual address space that the operating system and processor cooperate to map onto physical memory.
The memory management unit, or MMU, translates virtual addresses into physical addresses, typically using page tables. This lets different programs use identical virtual addresses without interfering with one another. It also provides memory protection and allows physical pages to be shared.
Of course, translating addresses takes time. So we add another cache.
The translation lookaside buffer, or TLB, remembers recently used address translations. A TLB hit avoids a potentially expensive page-table walk. Some processors even have multiple levels of TLBs. Unlike the ordinary CPU cache, which generally operates on cache lines, virtual memory usually works with pages, often 4 KB, although larger pages are also supported.
Notice what’s happening. We have caches for data, caches for instructions, and caches for the information needed to locate the data and instructions.
The simple act of reading a variable has become surprisingly complicated.
Not All RAM Is Created Equal
Even if we get through the caches and finally reach RAM, things aren’t necessarily simple. On systems with multiple processors or certain multicore arrangements, memory access can be non-uniform. This is called NUMA, or Non-Uniform Memory Access.
A processor may have relatively fast access to memory attached to its own NUMA node, while accessing memory associated with another node requires crossing an interconnect. Both locations are ordinary RAM, and both may appear in the same address space, but the access times and available bandwidth can differ.
The operating system tries to place memory near the processors using it. Moving a thread to another CPU can therefore change its memory performance, even though the program’s addresses haven’t changed.
And NUMA isn’t limited to traditional multiprocessor servers. Modern systems with chiplets, heterogeneous memory, and technologies such as CXL are making memory topology increasingly interesting.
What If It Isn’t In Memory?
Virtual memory has another trick. A virtual page doesn’t necessarily have to be backed by physical RAM at the moment you access it.
When a program accesses a page that isn’t currently mapped in a usable way, the processor generates a page fault. The operating system decides what to do. Sometimes it simply allocates a fresh page. Sometimes it retrieves data from an executable or memory-mapped file. Other times, the page was moved out of RAM to swap space and must be retrieved. Note that things like executable data are generally immutable, so most operating systems won’t swap them out. They simply trash them and know they can reload the same page later when needed.
This means your program can address data that isn’t presently in physical memory. Naturally, accessing an SSD is much slower than accessing RAM, so excessive swapping can bring a computer to its knees.
Virtual memory doesn’t actually require swapping, though. Address translation and protection are useful even on systems that never move pages to disk.
Your SSD Is Memory, Too
Suppose your program reads a file. You might imagine that the operating system sends a request to the SSD and waits for the data. But if the file was recently accessed, the data may already be in RAM.
Operating systems maintain a filesystem page cache, keeping frequently or recently accessed file data in memory. Writes may also be buffered in RAM before eventually reaching persistent storage.
This makes RAM act like a cache for secondary storage, much as L3 acts like a cache for RAM.
The analogy isn’t perfect. RAM is also the primary working storage for running programs, and anonymous memory isn’t necessarily a copy of anything on disk. But the general hierarchy is clear: registers, CPU caches, RAM, and persistent storage provide progressively different tradeoffs between speed, capacity, and cost.
Even SSDs have their own caches and address translations. Their flash translation layers map logical block addresses onto physical flash locations, hiding wear leveling and garbage collection from the host. Apparently, nobody wants to tell the truth about addresses.
Compressing The Middle
But what if we could make RAM effectively larger without adding more? Compression offers another way to complicate our memory hierarchy. Instead of immediately moving an infrequently used page to disk, why not compress it and keep it in RAM?
Linux has two well-known mechanisms for doing this: zram and zswap. Zram creates a compressed block device in RAM. It’s commonly used as a swap device, allowing pages to be stored in compressed form rather than written to physical storage.
Zswap, on the other hand, is a compressed cache in front of a swap device. When Linux decides to swap out a page, zswap attempts to compress and retain it in RAM. If necessary, pages can eventually be evicted to the backing swap device.
Both trade processor time for memory capacity and reduced storage I/O. Decompressing a page costs CPU cycles, but that can be considerably cheaper than retrieving it from an SSD.
There’s a catch, though. These mechanisms operate through the swap subsystem. When a compressed page is needed again, it must be decompressed into an ordinary page before the program can access it. That means page faults, memory management overhead, and moving data around. But what if compressed memory could behave more like ordinary RAM?
Meta Wants To Change The Rules
That’s the idea behind CRAM, or Compressed RAM, a project presented by Meta’s Gregory Price at the October 2026 Linux Plumbers Conference.
Rather than treating compressed memory as a conventional swap device, CRAM proposes exposing it through a special NUMA node. Linux can then use existing memory-management mechanisms to move pages between ordinary DRAM and compressed memory. The most interesting part is the hardware-assisted compression and decompression, which allows access at much finer granularity than conventional page-oriented compressed swap.
The developers report impressive benchmark results, including read performance that approaches ordinary DRAM in some tests and dramatic improvements over zram for selected workloads.
That doesn’t mean compression has suddenly become free. Writes, compression ratios, hardware requirements, and managing incompressible data all matter. CRAM is still developmental work, not ready for everyday use.
But it illustrates where the memory hierarchy is heading. Instead of simply deciding whether a page is in RAM or on disk, the operating system may choose among ordinary RAM, compressed memory, slower memory tiers, and secondary storage.
The Illusion Is The Feature
Most programmers don’t need to think about any of this most of the time. That’s the point of an abstraction. But when performance matters, the illusion starts to leak. Sequential access can beat random access by enormous margins. Two threads modifying unrelated variables can fight over cache lines. Moving a process between CPUs can change its memory latency. A page that looks like ordinary memory might actually require decompression or a trip to an SSD.
On smaller embedded systems, of course, the picture may be very different. Without an MMU or elaborate caches, memory can look much more like the simple physical address space we learned about in the first place.
We started with an array of bytes. We ended up with a hierarchy of caches, translation tables, NUMA nodes, compressed memory pools, and persistent storage devices, all cooperating to maintain that simple illusion. Amazing how complicated something so simple can be.
This reminds me of a thought I had.
With the success of AMD’s 3D Vcache, I feel like the next step is for them to introduce a RAM chiplet onto their CPUs. It would as another lower latency option before going to system memory. They if it’s large enough, it could also allow people to run RAMless mini machines.
You’ve reinvented L4 cache, also sometimes called eDRAM (embedded DRAM)
Cool, now get them to do it
Done, about a decade ago
https://www.intel.com/content/www/us/en/products/sku/88040/intel-core-i75775c-processor-6m-cache-up-to-3-70-ghz/specifications.html
Not AMD and not with modern chiplet design.
BTW, part of the point of my thought on the design is that the RAM wouldn’t be on the chip like L3 cache (or L4 eDRAM), it would be a separate chiplet on the board and connected to the Infinity Fabric like another die.
Since it’d be an entire other chip it could be huge with special memory management hardware and much better latency than standard RAM.
Actually embedded RAM isn’t a cache. It is just memory.
That is the idea behind the use of HBM (High Bandwidth Memory) on GPU and APU products from AMD. It is in the supercomputer parts, for example. https://en.wikipedia.org/wiki/High_Bandwidth_Memory
Lol those are things you learn during the first semester of CS course.
If people without education want to write code they should not be allowed becuase they cause more harm than they develop.
Good thing people can do things without being “allowed” to, then!
Exactly, they vibe-code so everyone can enjoy their incompetence :)
Like dentists, doctors and surgeons?
Well, yes and no, code does not allow you to switch letters of commands, so the compiler will disallow Gardoni to produce code :)
(just teasing)
Right now there is $3 trillion riding on the premise that nobody should ever understand code they write again.
That’s a rather gross mischaracterization. The premise is not that no one should, it’s that no one needs to. You’re “allowed” to do whatever you want. You want to learn the old slow and sloppy way to code? Go right ahead. You want to skip all that and let a tool do it better and faster? Go right ahead. The hope is democratization and choice, not removal of permission to learn.
SHEESH!
You need to ignore the “better”. But you’ll have fast (very fast) and cheap (we’ll see)… I think eventually the clanker will just spit out binaries, ready to flash, bypass the whole github, os and compile mess.
We have noted your superiority, unfortunately it is negated by reading the article after the clickbait headline.
I don’t need to read. I just click and comment ( ͡° ͜ʖ ͡°)
This statement is ridiculous. All of these features are there to abstract programming problems from the software developer through hardware. This was done to let programmers focus on the fundamental task of writing code while ignoring the details of memory location and coding for performance. Hardware designerd could save themselves a lot of heartache leaving this out. Processors could be faster, cost less, and use less power if they did.
The problem with CS majors is they know programming and not necessarily the specific topic/application the program is for. This is why we end up with people programming without CS degrees. Some times if you want something done right you have to do it yourself.
That last paragraph is a huge bingo. If you have to wait for the IT guys to configure things you never get exactly what you want, because of the blindingly obvious condition that they don’t understand your job.
Me personally, I think they should shut down all computing journalism, discussion forums and educational materials since if you got a proper education at an expensive prestigious university (like I did) you wouldn’t need those things; those resources do more harm than good because they let people other than me make an attempt at coding. Me personally I just read man pages and language specs all day.
[… other than] “Me” and not even “people like me” ? Well, there’s no doubt about the health of your ego, heh, heh. You might be making a joke. Not everyone can, by design, go to a prestigious university, so should we deprive them of any education at all? I am not sure why dabbling in code is worse… at least it teaches respect for coding (which can inform the prestige of programmers). In fact, I rather think programming should be a more wide-spread hobby. The People cannot govern themselves over matters of ignorance. Who is the “they” who would “shut down” said materials? The basic conundrums of computing are things like UNIX Philosophy vs. AI. The basic conundrum of good government is a balance of interests. Like it or not, these conundrums are near merging. The current degeneration of (Aristotelian) polity towards democracy can mostly (IMO) only be stemmed by education – not in facts, but in wisdom (but facts are necessary). If you want to be programming king, you sir or madam, will have to qualify with me as not a tyrant. If you would like to admit other prestigious programmers, you will need to qualify your club as an (Aristotelian) aristocracy and not an oligarchy.
Pfft, allowed to? This is a hacking site. The whole premise is doing things we’re not “allowed” to do. If we only wanted to read manuals and follow instructions, we wouldn’t be here.
:p
This 100%
This has been the case for most high-level languages and libraries where programmers are not supposed to know how the computer works to get results. That also matches your statement where high-level languages are as fast to write as they are slow to execute.
Clickbait title. Why HA goes so low?
Bc HA dig always deeper to keep headroom in the -Z direction
Ya, also a different pacing to the writing than I’d expect. Almost feels a tad AI with the paragraph blocks and vauge slightly clickbaity headers.
Maybe it’s that AI writing is changing how humans write…. Which might be even scarier than AI itself being used to write this?
Oh no you were correct the first time; it’s definitely LLM-“assisted”.
…
‘Most programmers don’t need to think about any of this most of the time. That’s the point of an abstraction. But when performance matters, the illusion starts to leak. Sequential access can beat random access by enormous margins. Two threads modifying unrelated variables can fight over cache lines. Moving a process between CPUs can change its memory latency. A page that looks like ordinary memory might actually require decompression or a trip to an SSD.’
…is a classic example of LLM-generated output. The style is unmistakable.
I saw that “let me explain what I’m saying as I say it with a list of things to say”-style of writing long before LLMs were a thing. Must less common, sure, but it existed.
Well, I didn’t really believe you, so I cut’n’paste that paragraph into several different AI text detectors. Three said 100% AI, one said 92% chance it is AI… I also tried large chunks (since accuracy improves with more data) of the article, still 100% AI (with 92% one going to 96%).
For reference I used grammarly.com, humalingo.com, app.gptzero.me and pangram.com. Humalingo also called out a 91% plagiarism risk.
I’m not saying they did or didn’t use AI to write this. But your remarks appear to be backed by the data.
Well, Al IS the editor in chief. Ignoring the fact that he’s the author of this article, they use him to write every article. You don’t need grammarly.com to figure that out. You just have to check the staff page.
Maybe we read different slop, but the slop I read: two mosts in the same sentence is usually written by a human. That’s classic human slop. People LOVE to repeat words, especially modals, in the same sentence. Never heard anyone say, “I’m already ready already?” That’s not AI. Starting a sentence with a conjunction? AI knows better than to do that! It’s more accepted in the UK, but everyone who went to school in the US should know better than that, but most people don’t remember what they learned in school, especially classes they didn’t particularly like. I don’t know. I don’t think this is AI generated. Maybe assisted or edited, but to me, it sounds pretty human.
Just out of curiosity, how many of the humans here claiming to know slop when they see it are English/language teachers or linguists? I’m thinking I’m one of the few, but I seriously hope not.
Yeah, but you’re totally wrong. Al wrote this by himself 100%. I know him, and he’s human.
This discussion is speculation, and incorrect speculation to boot. Hackaday is 100% made by our writers.
Everybody has a unique style of writing. I grew up as a lock key child and had a “unabridged English dictionary” which is a massive book, cool maps, and index’s. It some of the words had nice illustrations. I would read that dictionary and the Merck Manual (a physicians desk reference divided by practice area).
I also am a wordy kind of person being a lawyer and having a philosophy degree. Saying things clearly and my science degree compete with one another for being very clear to a fault and writing succinctly. At the same time, I find the new modern writing style limiting the number of words a person can use to express themselves could be incorrectly identified as synthetic intelligence “slop” but what I read often online is brain rot “chop” 🪓. As will become clear below, I don’t like the use of Artificial Intelligence or phrases like “deep machine learning”. There seem too anthropomorphic! Perhaps, a Convalusional Silicon Network is better as would “complex computer algorithm development” (deep machine learning).
I’m a big fan of Bertrand Russell who was a language analysis philosopher and co-wrote an anti-nuclear weapon article with Einstein. That article began a legal fight by a group which sounds like a historical and extremist version of MAGA. I don’t like the use of infinity in math and mostly cosmology.
I also don’t like the entire set of shorthand for concepts in cosmology and astrophysics which look very intelligent all written accross a big black board with a group of people sitting there looking up at it in contemplation.
We really don’t know much of what is underlying cosmological “hypothesis” and things which we call theories when in reality we don’t know for sure. We can’t certain on the issue of locality. We have no way st this time to test or design certain experiments to prove any of it. I heard a idea form the brother of Atrium’s creator. The speed of light is subject to an observer. From our perspective it is 300,000 km / s roughly. But, from the point of view of the light itself it is infinity fast or instantaneous. They also seemly measured recently the time to cause the breakdown of a spooky action at distance (entanglement) and an subsequent conjecture that the “metadata on the state of the other entangled particles” flows through microworm hole.
We know very little beyond conjecture beyond what we can test at the local area. People will scream to the contrary but their tenure, positions, or bid for more or funding. They have to pick a side or team and back it their whole career. Or, they have been told or taught it rather than having really throught it through.
So, “they” tried to have Einstein’s US citizenship revoked or removed for his right to express himself and his right to political free speech. He was to my knowledge the first legal dual national as the code has been “interpreted” that nobody has the “intent” to expatriate or renounce their US citizenship. They want to keep as many people on the hook for taxes which has served the US tax base well. The case was thrown out. He was given special permission to hold two citizenships as he wouldn’t take the US passport unless they let him keep his Swiss citizenship.
I’ve read part of the article however I intend to look it up and read the remainder. It’s on display at his museum here in Bern, Switzerland. So are his grades which I saw photographed on Threads recently. They were the results of his “Swiss Maturity” or our version of a International Bachelorette/ Pre-University High School Diploma. Most US Universities will give university credits for grades 5 and 6. It’s a super hard exam. Einstein wouldn’t have been credits for Geography or French :-) Maths, Physics, and the subjects you would assume he would do well in he would have entered a US university almost as Sophomore beginning his second semester. It’s possible to start as a Junior or very close.
See – I’m wordy and have ADHD 😁!
Notice I didn’t even do a good job of proof reading before I hit – the 🔘
If someone is interested in reading about memory hierarchies and memory consistency, imho the canonical book to read is “Computer Architecture: A Quantitative Approach” by Patterson and Hennessy.
If anyone can suggest some other quality papers, please share the titles.
It’s come quiet a way from using a RAM disk in DOS to avoid waiting for the floppy drive.
it’s a fine survey article as it goes, but i really did not like the headline, this ‘lie’ framing. no one is being deceived. the people who understand memory enough well enough to imagine the counterfactual (one bus transaction for each load/store instruction) also understand memory well enough to know all this stuff already. there are people who still have yet to learn about cache and coherence and all the details, but for the most part they don’t have false ideas, they just don’t know yet.
an address is just a handle. no need to make it spooky.
Spooky addressing at a distance.
Bravo for the Einstein reference!
@Greg A this article is almost entirely LLM-generated, that’s why it has this nauseating clickbaity style. As soon as you start noticing it, you can tell whether the YouTube video you’re watching is LLM-generated slop or mis/disinformation…
Ah so my spidey senses served me well again by telling me to shitpost in comments without reading the “article” 😂
That’s a grade A meatship there ♪
Unfortunately for you, I did actually read through the whole article before “shitposting” in the comments – that’s how I knew the third last paragraph was one of the best examples for illustrating my point (if you view my other comment above). Interestingly, other commenters seem to have tested/peer-reviewed my “shitposting” and come to the conclusion that my “spidey senses” serve me rather well here.
I give your “efforts” Grade “EF” for Epic Fail.
Upgrayedd don’t hurt yourself patting your own back that hard. Your “spidey sense” isn’t much better than blind luck:
https://www.pnas.org/doi/10.1073/pnas.2208839120#sec-1
Wut? I don’t know whether this is generational, or you have an unusual mind, or we “just” think of different meanings for several of these words.
The idea that an address would represent a single physical location is something that many people understood long before “computer” referred to an electric device. You pick something up, then you put it back down. Maybe you are careful enough to put it back where it belongs.
On the other hand, you seem to think of “handle” as a basic and obvious concept. I can assure you that I learned the word before I had seen a computer, and the computer usage still (decades later) doesn’t quite make sense. It helps to have learned about interfaces and APIs and to have seen examples where it made sense to swap the implementation, but what distinguishes a handle from a name or an address or a token or a key … is not always consistent. Handle vs address vs context vs userdata doesn’t even have predictable implications for performance or security, unless you are staying within a single ecosystem.
Sorry, but the article is written in a style “They’re lying to us! Everything is a hoax!”.
We’re completely aware that our computers have virtual memory. The personal ones (aka PC) have it from ~1985, aka 41y ago. The reason it was adopted was to allow the OS and multiple applications running at the same time without breaking/interfering with each other. If you don’t want virtual memory and protected mode (which is more than simply memory protection), you can turn it off and run whatever software is available for such configuration (DOS maybe?).
Regarding the caches, everyone is completely aware of their existence, and basically no one cares to the extent that their applications are running fine (and engineers design their memory hierarchies in a way that makes customers’ applications run a little bit faster with each CPU model release).
These topics are extensively studied in university CS programs, and there’s a ton of technical literature that covers them. All these technologies are just the natural evolution of solutions and compromises that we’re wiling to make in order to advance the performance of our computers.
Can we build architectures where the fundamental optimization target is information density rather than instruction throughput? Basically more bang for the buck.
Well, you don’t need a whole architecture for this, just a density-optimized storage.
The current NAND FLASH chips seems to fit well – they have excellent areal, volumetric and bit-level density. Also we already have filesystems capable of different levels of compression, including open-source ones.
Do you have some exotic application which is not satisfied by the current technologies?
Who cares whether an SSD has sectors and cylinders or not. As long as it serves the right bytes I’m OK with it.
IBM had MXT back in the 2000s. I haven’t read the CRAM paper, but real-time chipset based memory compression has been around a while.
Virtual memory is a look up table wired to some of the upper bits of your address. Logically everything except the bottom 12 bits (4K), but really it’s tiers of lookup to make a sparse data structure so you can have lots of unused (and unmapped) areas, or collapse some levels together to make larger pages (huge pages).
Virtual memory isn’t conceptually very difficult to understand. And even in the physical world we use one table to look up something in another. (or at least we did back when people knew how to use a card catalog or a office filing system)
Oh, yes, everything is so simple when someone else will implement it :D.
If it is there, and you can see it, it’s real
If it’s there, and you can’t see it, it’s virtual
If it’s not there, and you can see, it’s imaginary
If it’s not there, and you can’t see it, it’s been stolen.
In that case, you stole my million dollars!
I actually liked this article.
I did too.
It’s not all new, but it leads into the recent push by Meta at the compression thing. Most articles are ignoring the whole part where it requires new hardware that our computers don’t have now, and make it sound like we can just download more RAM. Al avoided that.
You say lie, I say technique. “Lie” is such a strong word..
I liked this article.
But no longer nerds physical disk as backing.
Some guy did some good research and turns out zswap is superior to zram + swap nowadays.
Can you bypass some of this, by reading & writing to I/O Ports?
Port 3F8h for example, and associated IRQ 4. Doesn’t that bypass the MMU?
I vaguely recall the serial port having a multi character buffer (16?).
Reading the same port gives you different bytes. Do these end up in a cache line?
Probably not, processors from P5 on use Local APIC for all external interrupts, so even if you’re using a legacy interface it’s still going to be virtual and you could remap it in another part of the OS.
Otherwise all the cores would get interrupted.
If you want real interrupts though you can just use a single processor embedded system.
An abstraction isn’t a lie. Premise is flaky
One should also ask why are there these L1, L2 and L3 caches?
These are all optimizations that where developed when there was an existing code base. Intel and AMD optimizing their product to make existing software faster. There is also division of labor, like the north and south bridge. Exposing Intels internal development structure (CPU vs Motherboard). I know a few people that find that architecture “ugly”.
Other CPU developers wanted to leave these type of optimizations to the software developer and compilers.
Wondering if all of this complexity and conventions, like OS, compilers, memory boundaries, are still needed in the future. I can see a future where AI just makes machine code, but perhaps that is not where the money is. More source code, more services, more github, more better…