After this lecture you will be able to:

Over the past five lectures, you have been working inside the CPU. You learned to load values into registers and address memory using segment:offset pairs (L03), manipulate data with arithmetic and logical instructions (L04), organize programs with the stack and procedures (L05), examine the system architecture surrounding the CPU (L06), and interact with the operating system through software interrupts (L07). Along the way, you built real programs that read keyboard input, printed text to the screen, performed recursive calculations, and handled multiple function calls with proper stack discipline — all in 16-bit Real Mode assembly.

But there is a vast layer of complexity sitting between your assembly instructions and the physical world that we have not yet explored.

Consider what actually happens when you press a key in DOSBox. Your program calls INT 16h and gets back a scan code in AH and an ASCII character in AL. That is the software abstraction — clean and simple. On the hardware side, something far more interesting occurred. A physical key closed a circuit on a matrix grid inside the keyboard. A dedicated microcontroller (not the main CPU — a separate processor inside the keyboard itself) scanned that matrix, detected the change, waited for the electrical signal to stop bouncing, translated the row-column position into a numeric scan code, and transmitted that code over a serial link to another controller chip on the motherboard. That motherboard controller stored the code in a one-byte buffer and raised an electrical interrupt signal on the IRQ1 line. The CPU paused your program, jumped to a BIOS handler at the address stored in IVT entry 09h, which read the scan code from I/O port 60h using the IN instruction, translated it into an ASCII character using lookup tables and the current shift/lock state, stuffed the result into a 16-element circular buffer in the BIOS Data Area, sent an end-of-interrupt signal to the interrupt controller, and returned via IRET — all transparently, before your INT 16h call returns with the result. Your program never touched any of that hardware directly. The BIOS did it for you, communicating with the keyboard controller through something called I/O ports.

Or consider memory. When you write a value with MOV [BX], AX, you think of it as a single step — the value goes "to memory." But the reality is far more layered. On a modern processor, that write might first land in the L1 cache (a tiny, extremely fast SRAM buffer holding perhaps 32 KB of recently accessed data), and the cache controller must eventually propagate the data to the L2 cache, then to the L3 cache, and finally to main DRAM. Each level is larger but slower than the one above it. The DRAM itself is not a simple array of flip-flops — it is organized in rows and columns of capacitors that must be periodically refreshed (re-charged) every 64 milliseconds to avoid losing their contents. And if the data were stored on a hard drive rather than RAM, access would be roughly a million times slower.

These are not esoteric details. The memory hierarchy directly affects program performance. A program that accesses data in patterns friendly to the cache can run 10 to 100 times faster than one that constantly causes cache misses. Understanding why requires knowing how caches work, what SRAM and DRAM actually are, and how the hierarchy from registers to disk is structured. This lecture gives you that understanding.

And then there are data structures. In a high-level language like C, you write struct Student { char name[32]; int age; } and the compiler handles memory layout, alignment, and pointer arithmetic. In assembly, you are the compiler. You decide where each field goes, how many bytes it occupies, how to navigate from one struct to the next in a linked list, and how to index into an array of such structs. Understanding how data structures map to raw bytes is one of the most practical skills you will take away from this course — it deepens your understanding of every data structure you will encounter in future courses on algorithms, operating systems, and systems programming.

Think of it this way: until now, you have been learning the language of the CPU — its registers, instructions, and calling conventions. This lecture teaches you the environment the CPU lives in: the devices it talks to, the memory hierarchy it relies on, and the data layouts it navigates. A programmer who knows only instructions is like a speaker who knows vocabulary but has never left their house. This lecture takes you outside.

To give you a concrete sense of where we are headed, here is a single example that touches all three themes of this lecture.

Suppose you want to write a program that reads a key from the keyboard and stores the character in a struct field. At the highest level of abstraction (C code), this looks trivial:

At the BIOS level (which you know from L07), the keyboard read becomes:

But at the hardware level (which you will learn in this lecture), the same operation involves reading the keyboard controller directly:

And the struct access, which the C compiler hides behind r.key, becomes an explicit offset calculation:

Each level strips away an abstraction and reveals the mechanism underneath. By the end of this lecture, you will be comfortable working at all three levels — and you will understand exactly how each one maps to the ones below it.

A network switch — one of many I/O devices that communicates with the CPU through the system bus.

This lecture — "Memory Systems, I/O Devices, and Low-Level Data Structures" — bridges the gap between the CPU-centric view of earlier lectures and the full system perspective. We will open the hood on how the CPU communicates with external devices, map out the complete memory hierarchy from registers to tape storage, and then bring everything back to assembly programming by implementing high-level data structures (structs, linked lists, string arrays) directly in memory.

The three themes of this lecture — I/O, memory, and data structures — are more interconnected than they first appear. I/O devices sit on the same bus as memory, and the CPU communicates with both using similar mechanisms (addresses and data transfers). The memory hierarchy determines how fast your program can access data structures. And the data structures themselves must be laid out in memory in a way that aligns with the hardware's access patterns. By the end of this lecture, you will see these connections clearly.

The lecture is organized in three parts:

Part 1: I/O Devices (Sections 2-6). We begin by revisiting the system bus from L06 and then focus on how I/O devices connect to it. Section 2 reviews the three bus types and introduces the distinction between memory-mapped and port-mapped I/O. Section 3 uses the keyboard controller as a detailed, concrete case study — tracing the complete path from a physical keypress to a scan code in the BIOS buffer. Sections 4-5 introduce the IN and OUT assembly instructions, keyboard commands, and scan code tables. Section 6 presents the standard I/O port address map of the IBM PC. By the end of Part 1, you will understand how to communicate with hardware directly — bypassing BIOS and DOS entirely.

Part 2: Memory Hierarchy and Types (Sections 7-11). We expand the memory hierarchy introduced briefly in L06 into a thorough treatment. Section 7 motivates the hierarchy with the fundamental speed-capacity trade-off. Section 8 examines cache architecture in detail: L1, L2, and L3 levels, split vs. unified caches, and how multi-core processors share cache. Section 9 quantifies the performance gaps between hierarchy levels with concrete bandwidth and latency numbers. Section 10 compares memory technologies (SRAM, DRAM, Flash, ROM, magnetic, optical) by speed, cost, volatility, and access type. Section 11 introduces virtual memory and paging — the mechanism that lets modern operating systems give every process the illusion of having the entire address space to itself.

Part 3: Low-Level Data Structures (Sections 12-15). We return to assembly programming with a practical focus: how do the data structures you know from high-level languages — arrays, 2D arrays, structs, and linked lists — actually exist in memory? Section 12 revisits 1D and 2D arrays (building on L05) with emphasis on row-major layout and address calculation. Section 13 introduces the TASM STRUC/ENDS directives for defining structs. Section 14 implements linked lists using pointers (segment:offset pairs) in assembly. Section 15 covers string arrays and tables of pointers. We implement each structure in x86 assembly, showing the exact memory layout and the instructions needed to create and traverse them.

In Lab 8, you will work directly with I/O ports for the first time, reading keyboard scan codes using the IN instruction instead of relying on INT 16h. You will also work with text-mode video memory at B800:0000, writing characters and attribute bytes directly into the video buffer. These are not artificial exercises — they are exactly how real operating systems and game engines interact with hardware. The BIOS keyboard handler that INT 16h invokes does precisely what you will learn to do by hand in this lecture: read a byte from port 60h, check the status register at port 64h, and translate the result.

Later labs will require you to define complex data structures in assembly: arrays of strings, structs with mixed field types, and linked lists built from dynamically allocated memory blocks. The TASM STRUC/ENDS directives and pointer-chasing patterns introduced in Part 3 of this lecture will be directly applicable to that work.

Beyond the labs, the concepts in this lecture have lasting value. When you study operating systems in a future course, topics like virtual memory paging and device driver architecture will build directly on what you learn here. When you encounter a "segmentation fault" in a C program, the mental model of raw pointers, memory layout, and address spaces that you build here will help you diagnose it faster than any debugger output. And when you profile a slow program and discover that cache misses are the bottleneck, the memory hierarchy section of this lecture will explain exactly why.

This lecture sits at a crossroads in the course. It draws on nearly every topic from the first seven lectures, while introducing several entirely new concepts. Here is a map of what connects to what:

If any of these prior topics feel rusty, now is a good time to review the relevant sections. The I/O port material (Part 1) leans most heavily on L07 (interrupts) and L06 (buses). The memory hierarchy material (Part 2) extends L06 Section 9. The data structures material (Part 3) builds on L05 (arrays, stack frames) and L03 (memory addressing).

A CPU without I/O devices is a calculator in a sealed box — it can compute, but it cannot receive input or produce output. The entire purpose of a computer system is to interact with the world: reading keystrokes, displaying images, storing files, sending network packets. All of these interactions pass through the system bus, the electrical highway that connects the CPU to everything else.

In L06 (Section 7), we established this architecture and covered the three types of buses — data, address, and control. That section focused on how memory connects to the bus. Now we shift focus to I/O devices: how they attach to the same bus, how the CPU addresses them, and what distinguishes an I/O access from a memory access.

Recall from L06 that the system bus consists of three logical sub-buses, each carrying a different type of signal:

The three system buses — data, address, and control — connecting CPU, memory, and I/O devices.

The three buses work in concert for every data transfer. To read a byte from memory address 1234h, the CPU places 1234h on the address bus, asserts the RD (read) signal on the control bus, and then reads the byte that appears on the data bus. To write, it places the address, puts the data on the data bus, and asserts the WR (write) signal. Every device on the system bus — RAM, ROM, video cards, keyboard controllers, disk controllers — shares the same physical wires. When the CPU wants to communicate with a specific device, it places an address on the address bus and asserts the appropriate control signals. Only the device that recognizes that address responds.

This is analogous to a postal system: everyone shares the same postal network, but each house has a unique address. A letter addressed to "123 Main Street" reaches only that house, even though it travels through the same sorting facilities as letters to every other address. Similarly, the byte placed on the data bus reaches only the device whose address decoder matches the value on the address bus.

One control line deserves special attention: IO/M (I/O or Memory). On the 8086, this signal indicates whether the current bus cycle is accessing the I/O address space or the memory address space. When the CPU executes a MOV instruction, it drives IO/M low (Memory). When it executes an IN or OUT instruction, it drives IO/M high (I/O). This single signal line is what allows the system to have two separate address spaces using the same physical address bus wires. The address 0060h can refer to a memory location or an I/O port — the IO/M signal determines which.

With the bus architecture in mind, we can now ask a more specific question: how does the CPU distinguish between accessing memory and accessing a device? There are two fundamentally different models, and the x86 architecture supports both.

Memory-mapped I/O. The device's registers are assigned addresses within the normal memory address space. The CPU uses ordinary MOV instructions to read from and write to these addresses, and the bus hardware routes the access to the device rather than to RAM. Video memory at B800:0000 is a prime example you are already familiar with: when you execute MOV ES:[DI], AX with ES pointing to segment B800h, the data does not go to a RAM chip — it goes to the video adapter, which immediately interprets the bytes as a character and attribute and displays them on screen. From the CPU's perspective, it looks exactly like a memory write. You could even use REP STOSW to fill the entire screen — the string instruction writes to consecutive memory addresses, and every write goes to the video adapter.

In the Real Mode memory map (from L06 Section 10), the regions A0000h-BFFFFh (128 KB) are reserved for video memory. The hardware does not place RAM at these addresses; instead, the video adapter's memory is mapped there. Reading from these addresses reads the current contents of the video buffer; writing to them updates the display.

The advantage of memory-mapped I/O is flexibility: any instruction that can reference memory — MOV, ADD, CMP, TEST, even string instructions like STOSW — can interact with the device. You do not need special instructions. The disadvantage is that the device's registers consume addresses in the memory map, reducing the space available for actual RAM. On the 8086 with its 1 MB address space, this trade-off matters: the top 384 KB is reserved for ROM, video memory, and device buffers, leaving only 640 KB for conventional RAM (the famous "640K barrier").

Port-mapped I/O (isolated I/O). The device is assigned a separate address in a dedicated I/O address space, completely distinct from the memory address space. The x86 I/O address space is 16 bits wide, providing 65,536 possible port addresses (0000h-FFFFh). The CPU uses special instructions — IN and OUT — to communicate with these ports. Critically, the control bus carries a signal (IO/M on the 8086, or M/IO on later processors) that tells the bus hardware whether the current access targets memory or I/O space. This means address 60h in I/O space and address 0060h in memory space are completely different locations accessed through completely different mechanisms.

The same bus architecture with the I/O device interface highlighted — input and output ports connect external devices to the system bus.

The keyboard controller at ports 60h and 64h uses the port-mapped model. So do the Programmable Interval Timer (ports 40h-43h), the Programmable Interrupt Controller (ports 20h-21h), the DMA controller (ports 00h-0Fh), and the serial ports (ports 3F8h-3FFh for COM1). In contrast, video memory (B800:0000 for text mode, A000:0000 for graphics mode) uses memory-mapped I/O, as does the BIOS Data Area and the ROM BIOS itself.

Most real x86 systems use a combination of both models. The general pattern is: large data buffers (like video memory) are memory-mapped because transferring thousands of bytes through IN/OUT one byte at a time would be painfully slow; control and status registers (like keyboard ports) are port-mapped because only a few bytes need to be exchanged and the explicit IN/OUT instructions make the code's intent clear.

It is worth noting that many other processor architectures — ARM, MIPS, RISC-V — use only memory-mapped I/O. They do not have a separate I/O address space at all. Every device is accessed through normal load/store instructions at specific memory addresses. The x86 family's dual-model approach (both memory-mapped and port-mapped) is somewhat unusual, a legacy of the Intel 8080/8085 ancestry. In this lecture, we focus primarily on port-mapped I/O because it introduces the IN and OUT instructions that you will use in lab work. We will also briefly work with memory-mapped video memory in the context of text-mode display.

Every I/O device — keyboard, mouse, disk drive, network card, serial port, sound card — has a device controller: a dedicated chip or circuit that translates between the device's physical interface and the system bus protocol. The CPU never talks to the raw hardware directly. It talks to the controller, which in turn manages the device.

Consider a hard disk drive. The physical interface involves spinning magnetic platters at 7200 RPM, positioning read/write heads over specific tracks with micrometer precision, and interpreting magnetic flux patterns to recover stored data. The CPU knows nothing about any of this. Instead, the disk controller exposes a set of I/O ports (for an IDE/ATA drive: ports 1F0h-1F7h) where the CPU can write commands ("read sector 1234 from track 56, head 2") and read back data and status information. The controller handles all the mechanical and electrical details internally. The CPU just sends commands and receives results through a handful of ports — the same IN/OUT instructions used for the keyboard controller.

This controller-based architecture provides hardware abstraction: the CPU communicates through standardized interfaces, insulated from the physical details of each device. A keyboard controller handles scanning a key matrix thousands of times per second, debouncing signals, and translating physical key positions into scan codes. The CPU only needs to read the final scan code from a single I/O port. If a manufacturer redesigns the keyboard's internal circuitry, the controller still presents the same ports to the CPU, and no software changes are needed.

The standardization of controller interfaces is what made the IBM PC clone industry possible. Any keyboard that speaks the same protocol as the Intel 8042 controller works with any PC motherboard, regardless of who manufactured either component. The same principle applies to IDE/ATA disk controllers, VGA video adapters, and 16550 UART serial controllers. Standards at the controller level enabled an entire ecosystem of interchangeable hardware.

Here is a summary of the most common I/O devices on an IBM PC and how they connect:

Notice the pattern: control and status registers are port-mapped (accessed via IN/OUT), while large data regions like video memory are memory-mapped (accessed via MOV and other memory instructions). You will encounter many of these devices in later sections and labs. We will return to the full I/O port address map in Section 6.

With this framework in mind, let us examine a specific controller in detail: the keyboard controller.

You have been pressing keys and reading their values since Lab 4. In L07, you learned that INT 16h invokes a BIOS handler that returns a scan code and ASCII character. But what is really happening between the moment your finger presses a key and the moment that scan code appears in the AH register? The answer involves a dedicated piece of hardware — the keyboard controller — that performs a surprising amount of work entirely on its own, independent of the CPU.

We use the keyboard as our case study for I/O device communication because it is concrete and familiar. Every student has interacted with a keyboard thousands of times. The keyboard is also architecturally representative: it uses port-mapped I/O, has both data and status registers, communicates through interrupts, and requires a specific command protocol. By the end of this section, you will understand the complete hardware path from keypress to software, and you will have a mental model that applies to virtually every other I/O device in a computer.

A mechanical keyboard with a key switch removed — each key is a physical switch in a matrix of rows and columns.

The original IBM PC (1981) used an Intel 8048 microcontroller embedded inside the keyboard itself. This small 8-bit processor scanned the key matrix and sent scan codes to the motherboard over a simple unidirectional serial protocol. Communication was one-way: the keyboard sent data to the PC, but the PC could not send commands back to the keyboard. There were no keyboard LEDs to control, no way to adjust the key repeat rate, and no way for the system to reset the keyboard programmatically. If the keyboard encountered an error, the system had no way to recover without a physical power cycle.

The IBM PC/AT (1984) introduced a fundamentally improved design with the Intel 8042 — a dedicated keyboard controller chip on the motherboard. This made communication bidirectional: the CPU could now send commands to the keyboard (set LED states, change scan code sets, adjust the typematic repeat rate and delay), and the keyboard could send responses back (acknowledgment bytes, error codes, self-test results, device identification). The 8042 became the standard keyboard controller interface for all IBM PC-compatible systems. Its register layout — data at port 60h, status/command at port 64h — has remained the standard for over four decades. Even modern systems that have integrated the 8042's functionality into the chipset (or the CPU's own south bridge) still present the same ports at the same addresses for backward compatibility.

The keyboard itself also contains a microcontroller — a descendant of the 8048, or a modern equivalent like an 8051. So the complete keyboard subsystem actually involves two processors communicating with each other over a serial link: the keyboard-internal controller that scans the matrix and generates scan codes, and the motherboard controller (8042) that receives those codes and makes them available to the CPU through I/O ports. Neither of these processors is the main CPU. The keyboard has its own firmware, its own scan loop, and its own internal state machine — it is a small embedded computer in its own right.

The 8042 keyboard controller sits as an intermediary between the PC system bus and the keyboard cable. It exposes four logical registers through just two I/O port addresses, using the direction of access (read vs. write) to distinguish between them:

The keyboard controller sits between the PC system bus and the keyboard, using ports 60h and 64h for data, commands, and status.

Let us examine each in detail.

Output Buffer (port 60h, read). When the keyboard sends a scan code, the 8042 controller places it in the output buffer. The CPU reads this buffer by executing IN AL, 60h. Despite the name "output buffer," it is an output from the controller's perspective — data flowing out of the controller toward the CPU. From the CPU's perspective, it is input data. This naming convention follows the device's viewpoint and is a common source of confusion. Remember: the controller outputs data to the CPU by placing it in this buffer.

Input Buffer (port 60h, write). When the CPU wants to send a command or data to the keyboard device itself (not to the controller — that distinction matters), it writes to port 60h using OUT 60h, AL. The controller picks up the byte from this buffer and forwards it to the keyboard over the serial link. The keyboard processes the command and may send back an acknowledgment byte (FAh for most commands). Again, "input" follows the controller's viewpoint: the buffer accepts input from the CPU destined for the keyboard.

Status Register (port 64h, read). This 8-bit register reports the controller's current state. Reading port 64h with IN AL, 64h returns the status byte. The bit layout is:

The two critical bits for everyday programming are bits 0 and 1. Before reading port 60h, a well-written program checks bit 0 of the status register to confirm that a byte is actually waiting. Before writing to port 60h or 64h, it checks bit 1 to ensure the controller has consumed the previous byte. Ignoring these checks can lead to lost data or garbled commands. Here is the typical polling pattern:

These busy-wait loops are simple but effective for the keyboard controller, which typically responds within a few microseconds. In a real operating system, you would use interrupts instead of polling, but for direct hardware access in a DOS program, polling works well.

Command Register (port 64h, write). Writing to port 64h sends a command to the controller itself (not to the keyboard). These are controller-level commands that manage the keyboard interface:

The distinction between writing to port 60h (data goes through the controller to the keyboard) and writing to port 64h (command goes to the controller itself) is critical. They are different operations that happen to share adjacent port numbers. Confusing the two will produce unpredictable behavior.

The keyboard itself is built around a key matrix — a grid of rows and columns with a switch at each intersection. A typical full-size keyboard has about 104 keys, arranged in a matrix that might be 16 columns by 8 rows (though the actual layout varies by manufacturer). Each key sits at a unique row-column intersection. No two keys share the same intersection, which is what allows the controller to identify exactly which key was pressed.

Inside the keyboard: a decoder drives rows, an encoder reads columns, and a counter cycles through the matrix to detect key presses.

Here is the complete sequence of events when you press a key:

Step 1: Physical contact. The key switch closes, connecting a specific row to a specific column in the matrix. In a mechanical keyboard, this happens when a spring-loaded switch mechanism (Cherry MX, for example) reaches its actuation point — typically about 2 mm of travel. In a membrane keyboard, a rubber dome collapses and presses two conductive membrane layers together. In either case, the result is the same: an electrical connection between one row and one column.

Step 2: Matrix scanning. The keyboard's internal microcontroller continuously scans the matrix by driving each row with a voltage in sequence and reading the column lines. When it detects that a particular column is active while a particular row is being driven, it identifies the row-column intersection where the key was pressed. This scanning happens thousands of times per second — far faster than any human can type — ensuring that no keypress is missed, even during fast typing.

Step 3: Debouncing. Mechanical switches do not make clean contact. When metal contacts come together, they "bounce" — alternating rapidly between open and closed states for several milliseconds as the contacts settle. Without compensation, a single keypress would register as two, three, or even more presses. The keyboard controller solves this by waiting for the signal to stabilize (typically 5-25 ms) before reporting the keypress. This delay is imperceptible to the user but essential for reliable input.

Step 4: Scan code generation. The controller translates the row-column position into a scan code — a unique numeric identifier for that physical key. This is not an ASCII code. The scan code identifies the physical key position on the keyboard, not the character it represents. The same physical key produces the same scan code regardless of the active keyboard layout or language setting. For Scan Code Set 1 (the default set used in Real Mode on IBM PC compatibles):

When a key is pressed, the keyboard sends the make code. When it is released, it sends the break code, which is the make code with bit 7 set (make code OR 80h). This make/break distinction is what allows the system to detect key combinations: when you press Shift+A, the system first receives the make code for Shift (2Ah), then the make code for A (1Eh). Because no break code for Shift has arrived, the system knows Shift is still held when A is pressed, and it generates an uppercase 'A'. When you release both keys, the corresponding break codes arrive.

If you hold a key down, the keyboard begins repeating the make code after a configurable delay (the typematic delay, typically 250-1000 ms), then continues at a configurable rate (the typematic rate, typically 2-30 characters per second). These parameters can be adjusted by sending the F3h command to the keyboard through port 60h.

Step 5: Serial transmission. The scan code is transmitted from the keyboard to the motherboard controller over a synchronous serial protocol using two lines: a clock line and a data line. Each byte is sent as 11 bits: 1 start bit (always 0), 8 data bits (LSB first), 1 odd parity bit, and 1 stop bit (always 1). The keyboard generates the clock signal at approximately 10-16 kHz.

Step 6: Buffering. The 8042 controller on the motherboard receives the serial data, reassembles it into a byte, checks the parity, and places the byte in its output buffer (port 60h). It sets the OBF (Output Buffer Full) flag — bit 0 of the status register at port 64h.

Step 7: Interrupt. The controller asserts the IRQ1 line, which triggers a hardware interrupt. As we learned in L07, IRQ1 maps to interrupt vector 09h through the Programmable Interrupt Controller (PIC). The CPU suspends whatever it was executing, pushes FLAGS, CS, and IP onto the stack, and jumps to the keyboard interrupt handler whose address is stored at IVT entry 09h (physical address 0000:0024h).

The BIOS keyboard handler (INT 09h) performs several tasks:

Step 8: Application reads the key. When your program calls INT 16h with AH=00h, the BIOS keyboard service checks the circular buffer. If a keystroke is available, it removes the oldest entry and returns the scan code in AH and the ASCII code in AL. If the buffer is empty, INT 16h enters a tight loop (HLT + check), waiting until IRQ1 delivers the next keystroke.

This entire chain — from switch closure to your program receiving the scan code — typically completes in under a millisecond for the hardware portion (steps 1 through 7). The software response time depends on how quickly your program calls INT 16h. The keyboard controller handles steps 1-6 entirely autonomously; the CPU only becomes involved at step 7, when the interrupt fires. This is a fundamental pattern in computer architecture: devices do their own work and only bother the CPU when they have something to report.

Understanding the keyboard controller is valuable far beyond keyboards themselves. The pattern you see here — a device controller with data and status ports, an interrupt-driven notification mechanism, and a software buffer managed by the OS — repeats across virtually every I/O device in a computer:

Once you understand the keyboard controller's architecture — data port, status port, command port, interrupt-driven notification, software buffer — you have a template for understanding all hardware I/O. The specific details change (different port numbers, different status bit layouts, different command sets), but the fundamental pattern remains identical.

In the next two sections, we will write assembly code that communicates directly with the keyboard controller using IN and OUT instructions, bypassing the BIOS entirely. You will see that direct hardware access is not dramatically more complex than using BIOS services — it just requires knowing the port numbers and respecting the protocol.

So far in this course, you have been communicating with hardware through layers of abstraction. When you called INT 21h to print a string, DOS called a BIOS routine, which eventually talked to the video hardware. When you called INT 16h to read a keypress, BIOS handled the keyboard controller on your behalf. But what does "talking to hardware" actually mean at the lowest level? The answer involves two instructions you have not yet seen: IN and OUT.

The CPU needs a way to exchange data with peripheral devices — the keyboard controller, the timer, the serial port, the disk controller, and so on. Computer architects have devised two fundamentally different approaches to this problem.

Memory-mapped I/O treats device registers as if they were memory locations. You already used this approach when writing directly to video memory at B800:0000 in text mode — every MOV to that address range didn't write to RAM at all; it sent data to the video adapter. The advantage is simplicity: any instruction that reads or writes memory can also interact with hardware. The disadvantage is that those memory addresses are "used up" and cannot hold normal data.

Port-mapped I/O (also called isolated I/O) gives devices their own separate address space, completely independent of the memory address space. The CPU uses dedicated instructions — IN and OUT — to access this space. The x86 architecture provides a 64K I/O port address space (ports 0x0000 through 0xFFFF), where each port is like a tiny mailbox connected to a specific device register.

Here is a side-by-side comparison:

The x86 platform uses both approaches simultaneously. Video memory is memory-mapped. The keyboard controller, the timer chip, the interrupt controller, and most legacy devices are port-mapped. This dual approach is a historical artifact of the IBM PC design, and understanding it is essential for systems programming.

Other architectures make different choices. ARM processors use exclusively memory-mapped I/O — they have no IN/OUT instructions at all. Every device register is accessed through regular load and store instructions to specific memory addresses. The x86 approach of maintaining two separate address spaces is somewhat unusual in the broader landscape of computer architecture, but it has the benefit of keeping the I/O space clearly separated from application memory.

The IN instruction reads a byte (or word) from an I/O port into the accumulator register:

There are two forms. The immediate form (IN AL, port) encodes the port number directly in the instruction as a single byte, so it only works for ports 0 through 255. The register-indirect form (IN AL, DX) takes the port number from the 16-bit DX register, which allows access to the full range of 65,536 ports. In practice, most common devices live in the low port range (below 0x100), so you will see both forms used frequently.

Why must DX be the register for the indirect form? This is not an arbitrary choice. The x86 instruction encoding reserves a specific opcode for IN AL, DX (opcode ECh) and another for IN AX, DX (opcode EDh). Only DX can be used — not BX, not SI, not any other register. Similarly, the accumulator (AL for bytes, AX for words) is the only valid destination. These constraints are wired into the CPU's instruction decoder.

The OUT instruction writes a byte (or word) from the accumulator to an I/O port:

The same two forms apply: immediate for ports 0-255, and register-indirect via DX for the full range. Note the operand order is reversed compared to IN — the port comes first, the register second (for the immediate form). This is consistent with the "destination, source" convention used by most x86 instructions, where the port is the destination you are writing to.

One important detail: IN and OUT do not affect the FLAGS register. They perform no comparison, set no zero flag, and trigger no overflow. They simply transfer data between the CPU and the I/O bus. If you need to check the value you just read (which you almost always do), you must use a separate instruction like TEST or CMP afterward.

When the CPU executes an IN or OUT instruction, what happens at the hardware level is fundamentally different from a memory access. For a regular MOV instruction that reads memory, the CPU puts an address on the address bus, asserts the memory-read control line, and reads the data bus. For an IN instruction, the CPU puts the port number on the address bus but asserts the I/O read control line instead. This separate control signal is what distinguishes I/O accesses from memory accesses, even though they share the same address and data buses.

The external hardware (the chipset, in modern systems) sees the I/O read signal and routes the request to the appropriate device based on the port address. Historically, this was done by discrete logic chips that decoded the address lines. On modern systems, the Platform Controller Hub (PCH) handles this routing internally. But the principle is the same: the port address acts like a device selector, and the I/O control signal tells the hardware "this is not a memory access."

This is also why port-mapped I/O does not interact with the CPU cache. Memory reads may be served from L1/L2/L3 cache, but I/O port reads always go all the way to the device. Every IN instruction generates an actual bus transaction to the hardware. This makes I/O inherently slower than cached memory access, but it guarantees that you always read the device's current state.

Let us see IN in action. In Lecture 7, you learned that the keyboard generates an IRQ1 hardware interrupt when a key is pressed, and the BIOS interrupt handler reads the scan code and places it in the keyboard buffer. But what if you want to bypass the BIOS entirely and read the keyboard hardware directly? Here is how:

Assembly code that polls the keyboard status register (port 64h) and reads scan codes directly from port 60h.

This pattern — check status, then read data — is fundamental to all port-mapped I/O. The keyboard controller has two ports that matter: port 64h is the status register (read-only from the CPU's perspective), and port 60h is the data register. Here is what the status register bits mean:

Bit 0 is the one we check before reading: it tells you whether the controller has placed a byte in the output buffer for the CPU to read. If you read port 60h when bit 0 is clear, you will get stale data from the previous read. Bit 1 is the one we check before writing (as you will see in Section 5): it tells you whether the controller has finished processing the previous command byte.

You might wonder: if we can poll the keyboard with a tight loop like this, why do we need interrupts at all? The answer is efficiency. Consider: a typical typist presses about 5-10 keys per second. A CPU running at even 4.77 MHz (the original IBM PC speed) executes millions of instructions per second. A polling loop would check the status port hundreds of thousands of times between each keypress, wasting all those CPU cycles on a bit that almost never changes. Hardware interrupts (IRQ1 for the keyboard) let the CPU do useful work and only respond when the keyboard actually has data.

The BIOS keyboard handler, triggered by IRQ1, reads port 60h, translates the scan code, and stores it in the BIOS keyboard buffer — all without your program having to poll. Your program can focus on computation, drawing, or anything else, and only check the keyboard buffer (via INT 16h) when it is ready to process input.

Direct port access via IN/OUT is still valuable in specific scenarios: writing device drivers, operating system kernels, bootloaders, or embedded firmware where you cannot rely on BIOS services. It is also indispensable for hardware that the BIOS does not know about, or when you need more control than the BIOS abstraction provides — as we will see in the next section with keyboard LED control.

In Protected Mode operating systems like Linux and Windows, user programs are not allowed to execute IN and OUT instructions. Attempting to do so triggers a General Protection Fault (exception 13). Only the kernel (running at privilege level 0, or "Ring 0") can access I/O ports directly. This is an important security boundary: imagine the damage a malicious program could do if it could write arbitrary bytes to the disk controller or reprogram the interrupt controller. The OS mediates all hardware access through device drivers, which run in kernel mode.

Now that you know how to read the keyboard controller's data port, let us look more closely at what that data actually means — and how to send commands back to the keyboard.

When you press a key on the keyboard, the keyboard's internal microcontroller does not send an ASCII character. It sends a scan code — a number that identifies the physical key on the keyboard, regardless of what letter or symbol is printed on the keycap. This is a crucial distinction: the keyboard hardware knows nothing about characters, languages, or shift states. It only knows which physical key went down or came up.

Why not just send ASCII characters? Consider: the keyboard has keys like Shift, Ctrl, Alt, F1-F12, arrow keys, and Num Lock — none of which correspond to printable characters. Moreover, the same physical key produces different characters depending on whether Shift is held (e.g., 'a' vs 'A', '1' vs '!'). The keyboard cannot know the correct character because it does not know the keyboard layout (QWERTY, AZERTY, Dvorak) or the current Shift state of the software. Scan codes solve this cleanly: the keyboard reports physical events (key down, key up), and the software decides what they mean.

Every key has two scan codes:

This make/break system is what allows the software to detect key combinations. When you press Ctrl+C, the system sees: make code for Ctrl (1Dh), then make code for C (2Eh). When you release them, it sees break codes 9Dh and AEh. The keyboard sends a continuous stream of these events, and it is up to the software (BIOS handler, OS driver, or your own code) to interpret them.

If you hold a key down, the keyboard begins sending the make code repeatedly after a short delay — this is the typematic (auto-repeat) behavior. The stream looks like this: 1Eh... (pause ~500ms) ...1Eh, 1Eh, 1Eh, 1Eh... (repeated at ~10/sec) ...9Eh (release). You can actually configure the repeat rate and delay by sending a command to the keyboard, as we will see shortly.

Extended keys on the 101/102-key keyboard (the arrow keys, Insert, Delete, Home, End, Page Up/Down, and the right Ctrl and Alt keys) send two-byte scan codes: an E0h prefix byte followed by the actual code. For example:

The E0h prefix tells the keyboard handler that an extended key is involved, not the regular key that shares the same second byte. For instance, the numpad '4' key and the Left Arrow key would both send 4Bh without the prefix — the E0h distinguishes them. The Pause/Break key is even more unusual — it sends a multi-byte sequence (E1h 1Dh 45h) and has no break code at all.

The following table shows the standard scan code set 1 used by IBM PC-compatible keyboards. Each physical key position has a unique number:

Table 90 from Art of Assembly: PC keyboard scan codes showing make (key-down) and break (key-up) values in hexadecimal.

Some important scan codes to know:

Notice that scan codes are assigned by physical key position on the keyboard, not by alphabetical order. The number row is 02h-0Bh, the top letter row (QWERTY) is 10h-19h, the home row (ASDF) is 1Eh-28h, and the bottom row (ZXCV) is 2Ch-35h. This reflects the order in which the keyboard scans its key matrix — hence the name "scan code."

Remember from Lecture 7: when you called INT 16h, AH=00h, the BIOS returned the scan code in AH and the ASCII character in AL. Now you understand what the BIOS was doing behind the scenes — it read the raw scan code from port 60h, looked up whether Shift or Ctrl was held (by tracking make/break codes for modifier keys), and translated the result into an ASCII character for your convenience.

This layering explains a subtle detail you may have noticed: INT 16h function 00h only recognizes 83 keys (the original PC/AT keyboard), while function 10h recognizes the full 101/102-key extended keyboard. The difference is that function 10h properly handles the E0h-prefixed extended scan codes. If you use function 00h with an extended keyboard, pressing keys like the right Ctrl or the arrow keys may produce unexpected results. This is why modern programs should use function 10h instead.

Communication with the keyboard is bidirectional. Not only does the keyboard send scan codes to the CPU, but the CPU can send commands to the keyboard's microcontroller by writing to port 60h. This allows you to control keyboard behavior: set the LED indicators, change the repeat rate, enable or disable the keyboard, or even reset it.

This bidirectional nature is why the status register has two separate buffer-status bits. Bit 0 (Output Buffer Full) tells the CPU "I have data for you to read." Bit 1 (Input Buffer Full) tells the CPU "I am still processing your last write — do not send anything yet." The terms "output" and "input" are from the controller's perspective, not the CPU's: the controller's "output buffer" is what the CPU reads, and the controller's "input buffer" is what the CPU writes to.

Before sending a command, you must check that the keyboard controller is ready to receive data. Bit 1 of the status register (port 64h) indicates whether the input buffer is full. You must wait until this bit is clear before writing:

Assembly code that sends the Set LEDs command (0xED) to the keyboard via port 60h, turning on all three indicator LEDs.

This is a two-step protocol: first send the command byte (0xED), then send the data byte (07h to light all three LEDs, or 00h to turn them all off). The LED data byte format is:

So 00000101b (05h) turns on Scroll Lock and Caps Lock but leaves Num Lock off. 00000010b (02h) turns on only Num Lock — which is the default state on most systems at boot.

The keyboard controller acknowledges each byte by sending back 0xFAh (ACK) through port 60h. A robust program would read this acknowledgment and check for errors (the keyboard sends 0xFEh to request a resend if it did not understand the command). In simple programs, the acknowledgment is often ignored, but production-quality keyboard drivers always check it.

The keyboard microcontroller recognizes a number of commands beyond LED control:

Keyboard microcontroller commands (port 60h) — the LED command (0xED) and other frequently used commands.

The complete keyboard command reference (Table 95) covering commands 0xED through 0xFF.

The Set Typematic Rate command (0xF3) is particularly interesting — it lets you control how fast held keys repeat. The data byte encodes both the repeat delay and the repeat rate in a single byte:

For example, sending 0x00 after 0xF3 gives the fastest possible repeat: 250ms delay then 30 characters per second. Sending 0x7F gives the slowest: 1000ms delay then 2 characters per second. You may have noticed that different computers seem to have different key repeat speeds — this is the command that controls it. Many BIOS setup utilities include a "Typematic Rate" setting that sends this exact command during boot.

These commands reveal something important about the keyboard: it is not a passive device. It contains its own microcontroller (historically the Intel 8042 or compatible) that runs its own firmware, maintains its own state, and responds to a protocol of commands and acknowledgments. The CPU and keyboard are two independent processors communicating over a shared bus.

This architecture — a main processor communicating with intelligent peripheral controllers through a command/response protocol — is a recurring pattern in computer systems. Your hard disk has its own controller. Your network adapter has its own processor. Even your USB hub has a microcontroller. The IN/OUT instructions you learned in this lecture are the simplest form of this inter-processor communication. More modern interfaces like PCI and USB use more sophisticated protocols, but the fundamental idea — send a command, wait for a response, read the data — remains the same.

You have now seen how the CPU uses IN and OUT to communicate with the keyboard controller through ports 60h and 64h. But the keyboard is just one of many devices that live in the I/O port address space. Let us zoom out and look at the full picture.

The x86 I/O address space spans 65,536 ports (0x0000 through 0xFFFF), completely separate from the memory address space. Over the decades since the IBM PC, standard port assignments have been established so that every device gets its own range:

Standard I/O port address assignments on the IBM PC — from the DMA controller at 0x0000 through the keyboard controller at 0x0060–0x0064.

Several of these devices should be familiar from earlier lectures. The PIC (Programmable Interrupt Controller) at ports 20h-21h is the chip that manages hardware interrupts — you learned in Lecture 7 that IRQ1 (keyboard) maps to INT 09h through the PIC. The PIT (Programmable Interval Timer) at ports 40h-47h is the system timer that generates IRQ0 at approximately 18.2 times per second — this is the heartbeat of the system, used by DOS to update the time-of-day counter at BIOS Data Area address 0040:006Ch. The DMA Controller at ports 00h-1Fh allows devices like the floppy drive to transfer data directly to memory without involving the CPU, which we will discuss in more detail in a later lecture.

Notice the pattern: most of the critical system devices cluster in the low port range (below 0x0400). This is the legacy I/O space inherited from the original IBM PC. The first 1,024 ports (0x0000-0x03FF) were defined by the original IBM PC/AT specification, and every PC-compatible computer since 1984 has maintained these assignments. This is why you can write IN AL, 60h on any x86 PC and be confident you are talking to the keyboard controller — it has been at that address for over 40 years.

Higher port ranges (0x0400 and above) are used by add-on cards such as sound cards, secondary disk controllers, and network adapters. The classic Sound Blaster card, for example, used ports 220h-22Fh. Network cards (NE2000 compatibles) typically used 300h-31Fh. These addresses were usually configurable via jumpers or DIP switches on the card — and getting them right was a notorious source of hardware conflicts in the DOS era (the dreaded "IRQ and I/O address conflicts").

On modern systems, many of these devices have migrated away from port-mapped I/O entirely, using memory-mapped I/O through the PCI/PCIe configuration space instead. But the core legacy devices — the PIC, PIT, keyboard controller, and CMOS — remain at their original port addresses, even in the newest x86 processors. Backward compatibility is the defining characteristic of the x86 platform.

Each device at its port range speaks its own protocol. The keyboard controller uses a status-then-data pattern (check port 64h, read/write port 60h). The PIT uses a command port (43h) and three counter ports (40h-42h). The PIC has an initialization sequence involving multiple writes to ports 20h and 21h. The CMOS/RTC uses an index-data pattern: write a register number to port 70h, then read or write the value at port 71h.

You do not need to memorize all of these protocols. What matters is the mental model: the I/O port space is a bus with many devices connected to it, each listening on its own address range. The CPU uses IN and OUT to send and receive individual bytes. Everything else — the meaning of those bytes, the sequence of reads and writes, the timing requirements — is defined by the specific device's datasheet.

Here is a quick example of reading the CMOS Real-Time Clock to get the current hour, just to show that the same IN/OUT pattern works for any device:

Two instructions. That is all it takes to read the time from the hardware clock. The CMOS uses an index-data protocol: write the register number you want to port 70h, then read (or write) the value at port 71h. Simple, but effective.

The CMOS chip contains 128 bytes of battery-backed memory. Registers 0x00 through 0x09 hold the real-time clock values (seconds, minutes, hours, day, month, year). Registers 0x10 through 0x3F hold BIOS configuration data — the floppy drive type, hard disk parameters, amount of installed memory, and boot device order. This is the data you configure in the BIOS Setup utility (the screen you reach by pressing F2 or Del during boot). Every time you change a BIOS setting, the firmware writes to CMOS ports 70h and 71h.

You might wonder why we spend time on port-mapped I/O when modern operating systems abstract all of this behind device drivers. There are several reasons. First, understanding IN/OUT is essential if you ever work on operating system kernels, embedded systems, or bootloaders — all environments where there are no drivers yet, and you must talk to hardware directly. Second, it completes your mental model of the system. You now understand the full chain from application code down to the physical bus transactions. Third, the concept of device registers accessed through a protocol of reads and writes applies universally, even on systems that use memory-mapped I/O instead of port-mapped I/O. The registers have different addresses, but the patterns — status polling, command/data pairs, index/data pairs — are the same everywhere.

Let us tie together everything from this section of the lecture by tracing the complete journey of a single keypress:

Alternatively, if you bypass the BIOS entirely (steps 6-9), you can poll port 64h and read port 60h directly — which is exactly what we did in the polling example earlier.

Notice how many concepts from earlier lectures converge in this single pathway: the stack (step 5 — FLAGS/CS/IP are pushed), the IVT (step 5 — looking up the handler address), hardware interrupts (step 4 — IRQ1 and the PIC), I/O ports (step 6 — IN from port 60h), and the segment:offset memory model (step 8 — the buffer at 0040:001E). A single keypress exercises nearly every layer of the system we have studied.

This end-to-end journey illustrates a central theme of this course: computer systems are built in layers. Physical signals become electrical pulses, which become scan codes, which become interrupt requests, which become buffer entries, which become the characters your program processes. Each layer provides an abstraction that simplifies the work of the layer above it — and each layer can be peeled back when you need direct control.

To consolidate what you have learned across Lectures 7 and 8, here are the three levels at which you can interact with the keyboard, from highest abstraction to lowest:

Each level builds on the one below it. DOS calls BIOS. BIOS reads hardware ports. The hardware ports connect to the keyboard controller's registers. Now you understand all three levels.

In a modern operating system like Linux or Windows, there is a fourth level — the OS keyboard driver — sitting between the hardware and the application. The driver reads scan codes (like the BIOS handler), translates them using a configurable keymap (supporting any language or layout), and delivers the result to applications through an event queue. The BIOS keyboard buffer is no longer used. But the fundamental hardware interface — IN from port 60h, status at port 64h — remains exactly the same as it was on the original IBM PC in 1981.

In Lecture 6, we introduced the memory hierarchy as a key architectural concept — a layered system of storage that bridges the enormous speed gap between the CPU and bulk storage. We noted the basic levels (registers, cache, RAM, disk) and their rough access times. Now it is time to go much deeper. Understanding the memory hierarchy is not just an academic exercise — it is one of the most practically important topics in computer architecture, because the way your program accesses memory can make a 100x or even 1000x difference in performance.

A motherboard with CPU socket and surrounding components — the physical realization of the memory hierarchy, with cache on the CPU die and RAM slots nearby.

Here is the fundamental problem. Modern CPUs can execute billions of operations per second. A single clock cycle on a 4 GHz processor takes 0.25 nanoseconds. But accessing main memory (DRAM) takes roughly 50–100 nanoseconds — that is 200 to 400 clock cycles of waiting. If the CPU had to go to main memory for every instruction and every piece of data, it would spend the vast majority of its time doing absolutely nothing, just waiting for data to arrive.

This mismatch between processor speed and memory speed is known as the memory wall. It has been a central challenge in computer design since the 1980s, when CPU clock speeds began to outpace DRAM improvements. Processors got faster at a rate of roughly 60% per year, while DRAM latency improved at only about 7% per year. The gap kept widening, and architects had to find a solution.

To put this in perspective: if you were a CPU core running at 4 GHz, and each clock cycle took one second of your subjective time, then waiting for main memory would feel like waiting three to five minutes. Waiting for an SSD would feel like waiting several days. Waiting for a hard drive would feel like waiting several months. The CPU is absurdly fast compared to the storage it depends on.

The memory wall is not just a historical curiosity — it continues to shape processor design today. Modern CPUs dedicate a staggering amount of their transistor budget to caches and memory management. On many contemporary processors, more than half the die area is occupied by cache memory. Look at a die photo of any modern CPU, and you will see that the actual computation logic (ALUs, execution units) occupies a surprisingly small fraction of the chip — the rest is predominantly cache.

The memory wall is also the primary reason CPUs have become so complex: techniques like out-of-order execution, speculative execution, and hardware prefetching all exist largely to keep the CPU busy while waiting for memory. Out-of-order execution allows the CPU to work on independent instructions while a memory request is pending. Speculative execution lets the CPU guess which branch a program will take and start working on it before the answer is known. Hardware prefetching detects sequential access patterns and loads data into cache before the program requests it. All of these techniques add enormous complexity to the processor, but they are necessary to mitigate the memory wall.

The solution exploits a fortunate property of real programs: they do not access memory randomly. Instead, programs exhibit locality of reference — they tend to access the same data and nearby data repeatedly over short periods of time. This locality comes in two forms:

These patterns are not accidental. They arise naturally from how programs are structured: loops repeat over the same code and data, arrays and structs store related data contiguously, and the stack grows and shrinks in a localized region.

Consider a simple example: a loop that sums an array of 1000 integers. The loop counter variable is read and written on every iteration — that is temporal locality. The array elements are accessed sequentially, one after another — that is spatial locality. Both the code (the loop instructions themselves) and the data (the array and the counter) exhibit strong locality. The memory hierarchy is specifically designed to exploit exactly these patterns.

There is also a related concept called sequential locality — a special case of spatial locality where access follows a strictly sequential pattern, such as reading bytes from a file or executing instructions in order. Sequential locality is the easiest pattern for hardware to detect and optimize through a technique called prefetching, where the cache proactively loads the next block of data before it is even requested.

How strong is locality in practice? Studies of real workloads have shown that typical programs access only about 10% of their total address space during any given interval of execution. Furthermore, a large fraction of memory accesses go to a very small set of "hot" addresses — the stack, the current function's local variables, and the data structures being actively processed. This concentration of accesses is what makes the memory hierarchy work: even though the cache can only hold a tiny fraction of total memory, it holds the fraction that matters most right now.

The memory hierarchy exploits locality by placing small amounts of very fast (and very expensive) memory close to the CPU, backed by progressively larger, slower, and cheaper storage. When the CPU needs data, it checks the fastest level first. If the data is there (a hit), access is nearly instantaneous. If not (a miss), the system fetches the data from a lower level and brings it up into the faster level, betting that locality means it will be needed again soon.

This is analogous to how you might organize information in your daily life. You keep the phone number you are about to call in your short-term memory (fast, tiny capacity). You keep frequently used contacts in your phone's favorites list (fast to access, limited space). You keep all your contacts in your phone's address book (larger, takes a bit longer to search). You keep old records in a filing cabinet (much larger, much slower). And you keep archived documents in a storage unit across town (enormous capacity, but it takes a trip to retrieve anything). Each level has more capacity but requires more effort to access. You naturally keep the most frequently needed information at the most accessible level — and the memory hierarchy does the same.

The memory hierarchy pyramid: smaller and faster at the top (registers, cache), larger and slower at the bottom (hard drives, tape backup).

Here is the full pyramid, from fastest to slowest:

Notice the enormous range: from sub-nanosecond register access to multi-second tape access — a factor of over 10 billion. Each level in the hierarchy effectively acts as a cache for the level below it. Registers cache values from L1, L1 caches data from L2, L2 from L3, L3 from RAM, RAM from SSD, and so on. The operating system and hardware collaborate to manage this transparently, so that from the programmer's perspective, you simply access memory addresses and the system does its best to serve them from the fastest available level.

The upper levels of the hierarchy (registers through RAM) are volatile — they lose their contents when power is removed. The lower levels (SSD, HDD, tape) are non-volatile — data persists without power. This is why your computer needs both RAM and a hard drive: RAM provides fast working storage while the system is running, while the drive provides permanent storage that survives reboots and power failures.

It is worth noting that the management of these levels is split between hardware and software. The cache levels (L1, L2, L3) are managed entirely by hardware — the CPU decides what to keep in cache and what to evict, with no software involvement. The boundary between RAM and disk is managed by the operating system through a mechanism called virtual memory (which we will discuss in a later lecture). The boundary between local storage and archival tape is managed by specialized backup software. Each layer of management adds complexity but is invisible to the layers above it.

This transparency is a key design principle. A programmer writing mov eax, [address] does not need to know whether that address is currently in L1 cache, L3 cache, or main memory. The hardware handles the lookup automatically. Similarly, a program accessing a memory-mapped file does not need to know whether the data is currently in RAM or still on disk — the OS handles the page fault and loads it transparently. This layered abstraction is what makes the hierarchy usable despite its complexity.

However, transparency does not mean that programmers can be ignorant of the hierarchy. While the hardware makes it invisible in terms of correctness (your program produces the correct result regardless of cache behavior), it is very much visible in terms of performance. A cache-friendly program and a cache-hostile program are both correct, but one may run 10x faster than the other. We will explore these performance implications further in Section 9.

It is helpful to understand the physical location of each hierarchy level. Registers are built directly into the CPU core, right next to the ALU and control logic — they are literally part of the execution engine. L1 cache is on the same silicon die as the core, physically adjacent to it — typically within a fraction of a millimeter. L2 cache is also on-die but slightly farther from the core logic. L3 cache occupies a large portion of the CPU die and is shared among all cores.

Main memory (DRAM) lives in separate chips, typically mounted on DIMMs (Dual In-line Memory Modules) that plug into slots on the motherboard. The physical distance from the CPU to the DRAM chips is several centimeters, and signals must travel across the motherboard's copper traces — this physical distance is one reason DRAM access takes so much longer than cache access.

SSDs and HDDs are connected via cables or PCIe slots, placing them even farther from the CPU. Tape storage is typically in a separate robotic library, possibly in a different room or building. The physical distance at each level mirrors the increasing latency — data literally has farther to travel.

When evaluating any level of the hierarchy, architects care about several properties:

These metrics are interrelated. Physically, placing memory closer to the processor core reduces latency (shorter wire delay) and energy (shorter signal path), but the space close to the core is limited, constraining capacity. Moving to a denser technology increases capacity but adds latency. Every level in the hierarchy represents a different balance point in this multi-dimensional tradeoff space.

The hierarchy exists because no single technology optimizes all of these properties simultaneously. SRAM is fast but expensive and low-density. DRAM is cheaper and denser but slower. Flash is cheaper still but much slower. Magnetic storage is the cheapest per bit but the slowest. By layering these technologies, architects create the illusion of a large, fast, cheap memory system — one that appears to the programmer as if it were both enormous and instant, even though no single technology can deliver both.

The concept of a CPU cache was first proposed by Maurice Wilkes in 1965, who called it "slave memory" — a small, fast buffer sitting between the processor and main memory. The idea was simple: keep the most recently used data in a fast store so the CPU does not have to wait for slow main memory on every access. The first commercial computer to include a cache was the IBM System/360 Model 85, released in 1968, which had a 16–32 KB cache. Today, caches are not just a single buffer but a multi-level hierarchy of caches, each with a distinct role. Every modern processor — from your laptop to a data center server to the chip in your smartphone — includes multiple levels of cache. It is one of the most universal features of contemporary CPU design.

You might wonder: why not just build one big, fast cache? The answer comes down to physics and economics. A very small cache (say, 32 KB) can be built extremely fast — it can be accessed in roughly 1 nanosecond because the data has a short physical distance to travel and the lookup logic is simple. But 32 KB is tiny; a program's working set often exceeds that, leading to frequent cache misses.

On the other hand, a large cache (say, 32 MB) can hold much more data, reducing misses. But the larger the cache, the longer the lookup takes — the electrical signals must travel farther, and the selection logic is more complex. A 32 MB cache might take 10 nanoseconds to access — still much faster than DRAM, but 10 times slower than a small 32 KB cache.

There is also a power consideration. Larger caches consume more energy per lookup because they drive longer wires and activate more circuitry. For a battery-powered device like a laptop or smartphone, the energy cost of cache access is a significant fraction of total power consumption. This is another reason architects do not simply make caches as large as possible — there is a power budget to respect.

The solution is to use multiple levels, each representing a different point on the size-speed tradeoff curve. A miss at a fast, small level falls through to a larger, slightly slower level before finally reaching main memory. Think of it as a series of safety nets: if the tightest, closest net misses the data, a wider net below catches most of what slipped through, and a still wider net below that catches nearly all the rest. Only a tiny fraction of accesses ever falls all the way down to main memory.

Cache hierarchy for a single CPU core: registers → L1 cache (on-chip, banked) → L2 unified → L3 unified → main memory.

The L1 cache is the smallest and fastest cache, sitting closest to the CPU core. Its defining characteristics:

Because L1 is so small, it relies heavily on temporal and spatial locality. The data you accessed one cycle ago is almost certainly still in L1. The next element in an array you are iterating through is very likely in L1 as well, because caches load data in blocks called cache lines (typically 64 bytes on x86 processors), not individual bytes.

This cache line concept is crucial. When you read a single byte from memory, the hardware does not fetch just that one byte — it fetches the entire 64-byte block (the cache line) that contains it. This is the hardware's way of exploiting spatial locality: if you read byte N, you will probably read bytes N+1, N+2, and so on, and they are already in the cache. For a 32 KB L1D cache with 64-byte lines, the cache holds 512 lines at any given time.

Why 64 bytes? This is a carefully chosen tradeoff. Larger cache lines exploit more spatial locality (more neighboring bytes are prefetched), but they also waste more bandwidth when the CPU only needs a few bytes from the line. Smaller lines reduce wasted bandwidth but miss out on spatial locality. The 64-byte line size has been the standard on x86 processors for over two decades because it balances these concerns well for typical workloads.

The L2 cache sits behind L1, serving as a backstop for L1 misses:

When the CPU requests data that is not in L1 (an L1 miss), the request goes to L2. If L2 has it (an L2 hit), the data is returned and also placed into L1 for future fast access. The 5 ns L2 latency is five times slower than L1, but it is still 10–20 times faster than main memory.

The relationship between L1 and L2 depends on the cache inclusion policy:

Modern AMD processors use an exclusive L3 design, while Intel has shifted from inclusive to non-inclusive policies in recent generations. These are internal architectural decisions that affect both effective capacity and the complexity of cache coherence protocols.

The L3 cache is the last level of cache before main memory:

Multi-core cache organization: each core has private L1 caches (split into data and instruction), with shared L2 and L3 caches and common main memory.

In a modern multi-core processor, the cache hierarchy typically looks like this: each core has its own private L1D, L1I, and L2. All cores share a single large L3. Main memory sits beyond L3. Some high-end server CPUs add an L4 cache (sometimes implemented in eDRAM) for even more capacity, but L3 is the last level in most consumer processors.

The L3 cache also plays a critical role in inter-core communication. When two threads running on different cores need to share data, the shared L3 provides a much faster path than going through main memory. If Core 0 writes a value and Core 2 reads it, the data can be served from L3 in ~10 ns rather than from RAM in ~100 ns. This makes the size and speed of L3 particularly important for multi-threaded applications.

On modern server processors, L3 caches can be enormous. AMD's EPYC processors offer up to 256 MB or more of L3 cache. AMD's consumer Ryzen 7 5800X3D, a gaming-oriented chip, famously added an extra 64 MB of stacked L3 cache (called 3D V-Cache) on top of the existing 32 MB, for a total of 96 MB. This extra cache improved gaming performance by 15–20% because many game workloads have data sets that fit in 96 MB but not in 32 MB — the cache was just large enough to eliminate a critical class of capacity misses.

The performance benefit of caches is quantified by the Average Access Time formula:

Here, hit_time is the time for a successful lookup at this cache level, miss_rate is the fraction of accesses that miss, and miss_penalty is the time to fetch data from the next level down (which may itself be the AAT of that next level). For multi-level caches, the formula is applied recursively — the miss penalty for L1 is the AAT of L2, and the miss penalty for L2 is the AAT of L3, and so on. This recursive structure is what makes multi-level caches so effective: each level reduces the miss penalty for the level above it.

Let us work through a concrete example. Suppose main memory access takes 50 ns, and we have these cache parameters:

Without any cache: every access goes to main memory. AAT = 50 ns.

With L1 only: AAT = 1 + (0.10 × 50) = 1 + 5 = 6 ns. That is an 8.3x improvement over no cache.

With L1 and L2: The miss penalty for L1 is now the AAT of L2, which is 5 + (0.01 × 50) = 5.5 ns. So: AAT = 1 + (0.10 × 5.5) = 1 + 0.55 = 1.55 ns. Another 3.9x improvement.

With L1, L2, and L3: The miss penalty for L2 is now the AAT of L3 = 10 + (0.01 × 50) = 10.5 ns. The miss penalty for L1 is the AAT of L2 = 5 + (0.01 × 10.5) = 5.105 ns. So: AAT = 1 + (0.10 × 5.105) = 1 + 0.5105 ≈ 1.51 ns. A modest further improvement, but every fraction of a nanosecond matters at billions of operations per second.

The progression is striking: we went from 50 ns (no cache) to 1.51 ns (three-level cache) — a 33x improvement in average access time. And note that the biggest win came from adding L1 (50 ns to 6 ns). Each subsequent level provides diminishing but still meaningful returns. This is why every modern CPU has at least two cache levels, and most have three.

The AAT formula also reveals a critical insight: even a small miss rate at L1 matters enormously if the miss penalty is large. A 5% L1 miss rate with a 100 ns miss penalty (no L2 or L3) would give AAT = 1 + (0.05 × 100) = 6 ns — meaning 83% of the effective access time comes from the rare misses, not the frequent hits. This is sometimes called Amdahl's Law for memory: the overall performance is dominated by the slowest path, weighted by how often it is taken. Reducing the miss rate by even a small percentage can have a dramatic impact on overall throughput.

A cache hit occurs when the requested data is found in the cache — the CPU gets its data quickly and continues execution. A cache miss occurs when the data is not in the cache, forcing a fetch from a slower level. There are three types of misses:

Programs that exhibit strong locality have high hit rates (95–99% at L1 is common for well-optimized code). Programs with poor locality — such as those that jump randomly through large data structures — suffer frequent misses and run much slower.

Understanding these miss types helps you optimize. Cold misses are unavoidable — you must access data for the first time (though prefetching can hide their latency by loading data before it is requested). Capacity misses can be reduced by restructuring your algorithm to work on smaller chunks of data at a time. Conflict misses can be reduced by changing data alignment or by using cache-friendly data layouts. In practice, capacity misses are the most common concern for application programmers.

How does the cache decide which line to evict when it is full and a new line must be loaded? This is the cache replacement policy. The most common policy is LRU (Least Recently Used): evict the line that has not been accessed for the longest time. The rationale is temporal locality — if a line has not been used recently, it is probably not going to be needed soon.

True LRU is expensive to implement in hardware for highly associative caches (tracking the exact access order of 12 ways requires substantial state), so most real caches use an approximation called pseudo-LRU that tracks access patterns with fewer bits. Other policies include random replacement (surprisingly effective — it avoids pathological worst cases that can trip up LRU) and FIFO (first in, first out — evict the oldest line regardless of recent usage).

When an evicted line has been modified (written to), it must be written back to the next level of the hierarchy before being discarded — this is called a write-back. If the line has only been read and not modified, it can simply be discarded (since a valid copy exists at the next level). Some caches use a write-through policy instead, where every write immediately updates both the cache and the next level. Write-through is simpler and guarantees consistency, but it generates more memory traffic. Write-back is more common in modern L1 and L2 caches because it reduces the number of slow writes to lower levels.

In a multi-core processor, each core has its own private L1 and usually L2, while L3 is shared. This design reflects a tradeoff:

However, sharing introduces a challenge: cache coherence. If Core 0 has a copy of some address in its private L1, and Core 1 modifies that same address, Core 0's copy is now stale. Hardware protocols (such as MESI) track the state of each cache line across all cores and ensure that all cores always see a consistent view of memory. The MESI protocol assigns each cache line one of four states:

This happens automatically in hardware, but it has performance implications. When one core modifies a cache line that other cores also hold, the hardware must invalidate those other copies and potentially write back the modified data. This takes time and consumes bus bandwidth. Excessive sharing of data between cores (called false sharing when it happens unintentionally — for example, when two unrelated variables happen to share the same 64-byte cache line) can degrade performance significantly, as the cache line bounces back and forth between cores.

To make these numbers concrete, here is the cache configuration of a typical Intel Core i7 (12th generation, Alder Lake) processor:

The "associativity" column describes how flexible the cache is in placing data, and this concept deserves explanation. Imagine the cache is divided into sets, and each memory address maps to exactly one set. Within that set, there are multiple ways — slots where the data can be stored. A 12-way set-associative cache means each memory address maps to one set, and within that set, there are 12 possible slots.

At the extremes:

The L1 caches use high associativity (8–12 ways) to minimize misses despite their small size. The tradeoff is slightly higher latency and power consumption per lookup, but for a cache this small and fast, that cost is acceptable.

By now you have seen the hierarchy and the cache levels. Let us step back and look at the broader picture: why does this hierarchy exist, and what does it mean for you as a programmer?

There is an iron law in memory design: faster memory is smaller and more expensive. This is not an arbitrary engineering choice — it is a consequence of physics and circuit design.

Fast memory (SRAM) uses six transistors per bit in a cross-coupled configuration that holds its state as long as power is applied. No refresh is needed, and access is nearly instantaneous. But six transistors per bit means very low density — you cannot pack much SRAM onto a chip, and the manufacturing cost per bit is high.

DRAM uses one transistor and one capacitor per bit. The capacitor stores charge to represent a 0 or 1, and the transistor controls access. This is far denser than SRAM — roughly 6 to 10 times more bits per unit area. But capacitors leak charge, so DRAM must be periodically refreshed (re-read and re-written) thousands of times per second, which adds latency and complexity. Reading a DRAM cell is also destructive — the act of reading drains the capacitor, so the data must be rewritten after every read.

Flash memory (used in SSDs) stores charge in floating-gate transistors that trap electrons behind an insulating layer. This allows data to persist without power — flash is non-volatile. The density is even higher than DRAM, and the cost per bit is much lower. But reading takes microseconds (not nanoseconds), and writing is slower still because it requires erasing an entire block before writing new data. Flash cells also wear out after a limited number of write cycles (typically 1,000–100,000 depending on the type), which is why SSDs include complex wear-leveling logic.

Magnetic storage (HDDs, tape) achieves the highest density and lowest cost by encoding data as magnetic orientations on a spinning platter or a tape ribbon. Access is glacially slow — the read/write head must physically move to the right location (seek time), and then wait for the platter to rotate to the right position (rotational latency). These mechanical operations take milliseconds, which is millions of times slower than electronic circuits.

The key insight is that each of these technologies occupies a distinct niche in the speed-cost spectrum. No technology dominates all others on every metric. If it did, we would simply use that one technology for everything. Instead, we stack them in a hierarchy, each technology serving its best purpose.

Researchers continue to explore new memory technologies that might narrow these gaps. MRAM (Magnetoresistive RAM) and ReRAM (Resistive RAM) promise non-volatile memory with near-DRAM speeds, which could eventually blur the line between memory and storage. 3D XPoint (marketed as Intel Optane) offered latencies around 10x faster than NAND flash, though still 10x slower than DRAM — it occupied a niche between DRAM and SSD that no previous technology had filled. HBM (High Bandwidth Memory) stacks DRAM chips vertically using through-silicon vias (TSVs) to dramatically increase bandwidth, reaching over 1 TB/s in HBM3 — crucial for GPUs and AI accelerators that need to feed vast amounts of data to thousands of parallel processing units.

These emerging technologies are adding new layers to the hierarchy rather than eliminating it. The fundamental physics — faster access requires smaller size and higher cost — is unlikely to change. What changes is the number and nature of the layers, and the specific technologies at each level. The hierarchy is not a temporary engineering compromise; it is a permanent feature of computer architecture.

Here is the complete picture in one table:

The speed hierarchy: CPU-level storage (registers, cache) is fastest; RAM is in the middle; mass storage (disk) is slowest.

Look at the cost column. SRAM used for L1 cache costs roughly $1,000 or more per gigabyte. DRAM costs about $3–5 per gigabyte — a factor of 200–300 cheaper. An HDD costs about $0.02 per gigabyte — another factor of 150 cheaper than DRAM. If we could build 64 GB of SRAM at L1 speed, it would cost tens of thousands of dollars just for the memory, and the chip would need to be impractically large to accommodate all those transistors.

The hierarchy exists because economics forces us to use a mix of technologies. A modern computer might contain: 1 KB of registers, 256 KB of L1 cache, 4 MB of L2 cache, 32 MB of L3 cache, 32 GB of DRAM, 1 TB of SSD, and perhaps an external 4 TB HDD for backups. Each level costs roughly the same amount of money (a few tens of dollars) but differs by orders of magnitude in size and speed. This is the magic of the hierarchy — equal investment at each level, but exponentially more capacity as you go down.

The memory hierarchy has evolved significantly over the decades. In the early days of computing (1950s–1960s), there was essentially no hierarchy — the processor accessed main memory directly, and that was it. The first caches appeared in the late 1960s. Two-level caches (L1 + L2) became common in the 1990s. Three-level caches (L1 + L2 + L3) became standard in the 2000s.

Meanwhile, the technologies at each level have also evolved. DRAM has gone through multiple generations: SDRAM, DDR, DDR2, DDR3, DDR4, and now DDR5 — each generation roughly doubling bandwidth over its predecessor. DDR5, for example, offers peak bandwidth of over 50 GB/s per channel, compared to roughly 25 GB/s for DDR4. However, latency has not improved nearly as much — DDR5 latency is actually slightly worse than DDR4 in absolute terms (nanoseconds), even though it is better in terms of clock cycles. This asymmetry between bandwidth improvement and latency stagnation is a recurring theme throughout the hierarchy.

Flash storage has evolved from SLC (single-level cell, 1 bit per cell) to MLC (2 bits), TLC (3 bits), and QLC (4 bits), each generation offering higher density and lower cost at the expense of endurance and speed. Modern SSDs also stack cells vertically in 3D NAND configurations with over 200 layers, dramatically increasing capacity per chip.

The trend at every level has been toward greater capacity and bandwidth, while latency improvements have been more modest — reinforcing the need for the hierarchy. The memory wall is not shrinking; if anything, it is growing wider, making caches and hierarchical design more important than ever.

As you move down the hierarchy from CPU to mass storage, bandwidth increases but latency and capacity also increase — the fundamental speed-size tradeoff.

Two metrics define how data moves through the hierarchy:

Both metrics matter, but in different scenarios. Latency dominates when you are making many small, random accesses — each one pays the full latency cost. Bandwidth dominates when you are streaming large blocks of sequential data — the initial latency is amortized over many bytes.

A useful analogy: latency is like the time it takes a delivery truck to drive from the warehouse to your house. Bandwidth is like the size of the truck. If you are ordering one small package, you care mostly about the travel time (latency). If you are moving your entire apartment, you care mostly about how much fits in each trip (bandwidth). Optimizing memory-intensive code requires thinking about which metric matters more for each access pattern.

Here are some approximate bandwidth figures for context:

Notice that the bandwidth drops by roughly an order of magnitude at each major boundary. The CPU-to-L1 connection might be 1,000 times faster than the connection to a SATA SSD. This is why keeping data in cache is so important — not just for latency, but also for throughput.

It is worth noting the recent rise of NVMe SSDs, which bypass the older SATA interface and connect directly to the CPU via PCIe lanes. This has narrowed the bandwidth gap between SSDs and RAM from roughly 100x (SATA era) to roughly 10x (NVMe era). Latency has also improved, dropping from ~100 μs for SATA SSDs to ~10–20 μs for the best NVMe drives. Modern NVMe drives using PCIe 5.0 can sustain sequential reads above 10 GB/s — fast enough that the bottleneck is shifting from the storage device to the CPU's ability to process data.

This trend is gradually blurring the boundary between storage and memory. Some systems now use persistent memory technologies that sit between DRAM and flash in both speed and cost. Others use CXL (Compute Express Link), a new interconnect standard that allows remote DRAM pools to be accessed almost as if they were local memory. The hierarchy is not static — it continues to evolve as new technologies emerge.

A detailed view of the memory hierarchy showing capacity, access time, and transfer unit at each level — from CPU registers (hundreds of bytes, <10 ns) through magnetic tape (unlimited, seconds).

You might think the memory hierarchy is purely a hardware concern and that software developers can ignore it. That would be a costly mistake. The difference between code that works with the cache hierarchy and code that works against it can be enormous — factors of 10x or more on the same hardware.

Here are the practical takeaways:

Sequential access is fast; random access is slow. Iterating through an array in order exploits spatial locality perfectly. The first access loads an entire 64-byte cache line, and the next 15 accesses (for 4-byte integers) all hit in cache. Jumping randomly through a large array defeats this — every access might be a cache miss.

Keep your working set small. If your frequently-accessed data fits in L1 (64 KB), your program will be blazingly fast. If it fits in L2 (1 MB), still fast. If it fits in L3 (32 MB), still decent. If it spills into main memory, performance drops noticeably. If it spills to disk, it drops catastrophically.

Data structure choice matters. An array of structs (where each struct contains all the fields you need) often outperforms a struct of arrays (where each field is a separate array) for sequential processing, because it keeps related data together in memory. Conversely, if you only access one field of each struct, a struct-of-arrays layout avoids loading unused fields into cache.

Reuse data while it is still in cache. If you need to process a dataset multiple times, try to do all your work on each block before moving to the next, rather than making multiple full passes over the entire dataset. This is called blocking or tiling, and it is the key optimization technique in high-performance numerical computing.

Avoid pointer-chasing patterns. Linked lists, trees, and graphs store nodes scattered across memory, connected by pointers. Following a chain of pointers — where each pointer could point anywhere in memory — produces a pattern of essentially random access that defeats both spatial and temporal locality. This is why arrays and array-based data structures (hash tables, flat vectors) tend to dramatically outperform pointer-based structures for many workloads, despite their theoretical algorithmic disadvantages in some operations.

Align data to cache line boundaries. If a 4-byte integer straddles two cache lines (partly in one, partly in the next), reading it requires loading two cache lines instead of one. Compilers generally handle basic alignment automatically, but when you are designing performance-critical data structures or working with SIMD (Single Instruction, Multiple Data) operations, manual attention to alignment can make a measurable difference.

Be aware of the "cliff" effects. Performance often degrades smoothly as your working set grows — until it exceeds a cache level's capacity, at which point there is a sudden, sharp increase in access time. A data structure that fits in L1 (32 KB) runs fast. One that grows to 40 KB and overflows L1 into L2 will see a noticeable slowdown. One that exceeds L3 and spills into main memory may become dramatically slower. Understanding where these cliffs lie for your target hardware allows you to size your data structures accordingly.

These are not abstract concerns. Database engines, game engines, scientific simulation codes, and operating system kernels are all carefully designed to work with the memory hierarchy. Professional game developers spend significant time ensuring that hot data structures fit within L1 or L2 cache. Database engines organize their on-disk data in page-sized blocks that map efficiently to memory pages and cache lines. Scientific computing libraries like BLAS implement matrix multiplication using blocking strategies that can achieve 90%+ of theoretical peak performance — versus perhaps 5-10% for a naive implementation that ignores the cache.

As a computer systems architect, understanding these tradeoffs is essential. The memory hierarchy is not just a hardware curiosity; it is one of the most important factors determining the real-world performance of the software you write.

The memory hierarchy is a brilliant engineering compromise born from a fundamental constraint: we cannot build memory that is simultaneously fast, large, cheap, and non-volatile. Instead, we layer multiple technologies, each optimized for a different point in the tradeoff space, and use locality of reference to keep the most relevant data at the fastest levels.

From the programmer's perspective, the key lessons are:

In earlier lectures, you encountered specific memory types in context: SRAM in caches (L03), DRAM in main memory (L03, L06), ROM holding the BIOS firmware (L07). You also saw the memory hierarchy table in L06, which showed how each level trades off speed, size, and cost. But we have not yet stepped back to survey the full landscape of computer memory technologies and the principles that distinguish them.

This section provides that comprehensive classification, organizing memory types by six different criteria: volatility, power consumption, read/write capability, access pattern, physical integration, and underlying technology. Understanding these distinctions is not merely academic — they explain why computer systems use a hierarchy of different memory technologies rather than a single universal one, and they will help you reason about performance trade-offs in real systems. The classification also provides vocabulary you will encounter in datasheets, technical documentation, and job interviews throughout your career.

The most fundamental division in memory technology is volatility — whether the memory retains its contents when power is removed.

Volatile memory loses all stored data the moment electrical power is cut. This category includes:

The key trade-offs between the two volatile technologies are worth summarizing:

This table explains a question students often ask: "Why not make all memory out of the faster type?" The answer is economics and physics. SRAM's six-transistor cell is roughly six times larger than DRAM's one-transistor cell, making it six times more expensive per bit and six times less dense. Building 16 GB of SRAM main memory would be prohibitively expensive and physically enormous. Conversely, DRAM is too slow for the innermost cache levels where the CPU needs data within a single clock cycle. The memory hierarchy exploits the strengths of each technology by using them in complementary roles.

The volatile nature of RAM is why your computer "forgets" everything in memory when it loses power — and why the boot process described in L07 is necessary every time you turn the machine on. The contents of DRAM cells decay within milliseconds without refresh, and SRAM cells lose their state immediately when the supply voltage drops.

Non-volatile memory retains data without power. This category is far more diverse:

The volatility divide explains the fundamental two-tier structure of every computer: fast, volatile memory for active computation, and slower, non-volatile storage for persistent data. The boot process is the bridge between them — it copies the operating system from non-volatile storage into volatile RAM so the CPU can execute it.

It is worth noting that these categories are not entirely fixed — they represent the current technological landscape, not fundamental physical laws. Emerging technologies have attempted to blur the volatile/non-volatile boundary. MRAM (Magnetoresistive RAM) stores bits as magnetic orientations in nanoscale magnetic tunnel junctions, offering non-volatility with near-SRAM speed. Intel's Optane persistent memory, based on a technology called 3D XPoint, offered non-volatile memory with latencies only 2–3x slower than DRAM — dramatically faster than Flash. While Optane was discontinued in 2022 (it proved too expensive and niche to be commercially viable at scale), the quest for "universal memory" — fast, dense, non-volatile, and cheap — continues to drive research in materials science and semiconductor engineering. If such a technology were to succeed, it could fundamentally reshape the memory hierarchy by eliminating the volatile/non-volatile divide.

Another useful distinction is between primary memory and secondary memory (sometimes called primary storage and secondary storage). Primary memory is directly addressable by the CPU — the processor can issue a MOV instruction with a memory address and access the data. This includes RAM (both SRAM and DRAM), ROM, and cache. Secondary memory is not directly addressable — the CPU cannot issue a simple load or store instruction to access data on a hard drive or SSD. Instead, it must go through an I/O controller, issue commands through a device driver, and wait for the data to be transferred into primary memory before it can be processed. This distinction matters for understanding performance: primary memory access is measured in nanoseconds and involves a single instruction, while secondary memory access is measured in microseconds or milliseconds and involves hundreds of instructions (the device driver code, the I/O controller protocol, and interrupt handling).

Different memory technologies have dramatically different power characteristics, which directly affect system design — especially in battery-powered devices like laptops and smartphones.

SRAM is the most power-hungry memory technology per bit when active. Each bit requires six transistors that must be continuously powered to maintain state. The cross-coupled inverters draw current whenever they are holding a value (this is called static power dissipation — power consumed even when no reads or writes are occurring), and switching state (writing) requires driving the bit lines against the current latch state. Modern SRAM cells in advanced process nodes (5 nm, 3 nm) also suffer from increasing leakage current — current that flows through transistors even when they are "off," a growing problem as transistors shrink. This is one reason caches are kept small: a 64 MB L3 cache would consume significant power even when idle, generating heat that would be difficult to dissipate within the processor package.

DRAM consumes less power per bit than SRAM in steady state — the single capacitor holding a charge is essentially passive storage. However, DRAM's refresh requirement adds a continuous power overhead. Every few milliseconds, the memory controller must read and rewrite every cell in the entire DRAM array, consuming power proportional to the total capacity. In a modern laptop with 16 GB of DDR5, the refresh cycles alone consume a meaningful fraction of the system's idle power budget. DRAM manufacturers continuously refine refresh algorithms (partial array refresh, temperature-compensated refresh intervals) to reduce this overhead.

Flash memory represents a dramatic improvement in idle power: a Flash chip consuming essentially zero power when not being accessed. Power is consumed only during read and write operations — and writes are particularly expensive because they involve high voltages to inject or remove electrons from the floating gate of each transistor. This near-zero idle power is why smartphones and tablets can retain their data for weeks on standby: the Flash storage consumes virtually no power between accesses.

Magnetic and optical storage have yet another power profile. HDDs consume moderate power when the platters are spinning (typically 3–8 watts), but can enter low-power standby modes where the platters stop. Optical drives consume power only when the disc is spinning and the laser is active. For tape drives, a cartridge sitting on a shelf consumes exactly zero power — the ultimate in energy-efficient storage, assuming you can tolerate the access time.

The power hierarchy — SRAM > DRAM > Flash > magnetic/optical — mirrors the speed hierarchy almost perfectly. Faster memory technologies consume more power, which is a fundamental physical trade-off: maintaining data in a rapidly accessible state requires continuous energy input. This is not a coincidence — it reflects a deep connection between thermodynamics and information theory. Keeping data "ready" (low latency) requires maintaining physical states that are easy to read quickly, and sustaining those states costs energy.

Not all memory is equally easy to read and write. The read/write characteristics of a memory technology determine how it can be used in a system.

Read/Write memory (RAM) can be freely read from and written to at the individual byte or word level, with roughly equal speed for both operations. Both SRAM and DRAM fall into this category. This is the memory your programs operate on directly — every MOV [address], value instruction you have written in assembly performs a write to RAM, and every MOV reg, [address] performs a read.

Flash memory occupies an interesting middle ground. It can be read at the individual byte level, just like RAM. But writes are constrained: Flash can only be written in blocks (typically 4 KB to 256 KB), and a block must be fully erased before any part of it can be rewritten. Erasing sets all bits to 1, and writing selectively changes some bits to 0. You cannot change a 0 back to a 1 without erasing the entire block. This asymmetry between reads and writes — and the block-level granularity of writes — is why SSDs need sophisticated firmware controllers with wear leveling, garbage collection, and write amplification management. It is also why the operating system cannot treat an SSD as simply "fast RAM" despite Flash being a semiconductor technology.

Read-only memory comes in several gradations, as described in Section 10.1:

The progression from ROM to EEPROM to Flash represents a steady increase in write flexibility, though at the cost of endurance: Flash cells can typically survive 1,000 to 100,000 write cycles before wearing out, depending on the type of Flash. This is still orders of magnitude less durable than DRAM, which can be written essentially unlimited times.

The endurance issue deserves elaboration because it has practical consequences you may encounter. Modern NAND Flash comes in several variants that trade endurance for density:

Packing more bits per cell increases density and decreases cost, but each additional bit makes the voltage levels closer together, increasing error rates and reducing how many times the cell can be reliably rewritten. SSD firmware compensates through wear leveling (distributing writes evenly across all cells), over-provisioning (reserving extra capacity to replace worn cells), and error-correcting codes (detecting and correcting bit errors). Despite these mitigations, the finite write endurance of Flash is a real constraint — a heavily written SSD will eventually wear out, though modern consumer SSDs typically last many years under normal use.

How a memory technology allows you to locate and retrieve data has profound implications for performance.

Random access means any memory location can be read or written in the same amount of time, regardless of which location was accessed previously. There is no need to traverse intervening locations to reach a target address. SRAM and DRAM are true random-access memories — the address decoding circuitry selects the target row and column directly, and every access takes the same number of clock cycles (ignoring cache effects). This is where the term "RAM" (Random-Access Memory) comes from, though in modern usage, "RAM" almost always refers specifically to DRAM.

Inside a DRAM chip, the memory cells are arranged in a two-dimensional grid of rows and columns. To access a particular bit, the memory controller first sends a row address (activating an entire row of cells, copying them into a row buffer), then sends a column address (selecting a specific cell within the activated row). This row-then-column scheme is why DRAM access has two distinct timing components: RAS (Row Address Strobe) latency and CAS (Column Address Strobe) latency. Accessing a different column within the same row is faster than accessing a new row, because the row is already activated — this is the origin of the performance benefit of sequential memory access patterns even in "random-access" memory.

Sequential access means data must be read in order — to reach a particular location, you must pass through all preceding locations. The canonical example is magnetic tape. To read data at the middle or end of a tape, the drive must physically spool through all the tape before it, which can take seconds to minutes depending on the position. This makes tape terrible for random lookups but perfectly acceptable for its primary use case: backup and archival storage, where data is written sequentially and read back in the same order (or not read at all, ideally — the best backup is one you never need to restore).

Direct access (sometimes called block access) falls between random and sequential. The device can jump directly to a general area or block, then access data sequentially within that block. Hard disk drives are the primary example: the read/write head can seek to any track on the platter (direct access to the block), but then must wait for the desired sector to rotate under the head (sequential access within the track). A typical 7,200 RPM drive completes one rotation in about 8.3 milliseconds, so the average rotational latency (waiting for the right sector) is half that — about 4.2 ms. The seek time (moving the head to the right track) averages 5–10 ms. Combined, a single random read from an HDD takes roughly 10–15 ms, which is why HDD performance is characterized by both seek time (the direct-access component) and rotational latency (the sequential component).

Optical drives (CDs, DVDs, Blu-ray) work similarly: the laser assembly can jump to any track on the spiral, but then reads sequentially along that spiral. Optical drives are generally slower than HDDs because the disc spins at lower speeds and the seek mechanism is less precise.

SSDs are an interesting case: because they are built from Flash chips that support random access at the page level, SSDs provide true random access without seek time or rotational latency. This is the single biggest performance advantage of SSDs over HDDs and explains why replacing an HDD with an SSD produces such a dramatic improvement in responsiveness — the operating system makes many small random reads (loading program files, reading configuration data, accessing metadata), and the access pattern that cripples HDDs is handled effortlessly by SSDs.

The access type of a memory device is closely related to how the CPU communicates with it. CPU-memory communication relies on three types of buses to connect the processor to memory devices. The address bus carries the address of the memory location being accessed (the width of this bus determines the maximum addressable memory — a 32-bit address bus can address 2^32 = 4 GB). The data bus carries the actual data being read or written (its width determines how many bits can be transferred per cycle — 64 bits on modern systems). The control bus carries signals that coordinate the operation: read/write commands, clock signals, chip-select lines, and acknowledgment signals. For random-access memory, the address bus selects a location, the control bus specifies the operation, and the data bus transfers the result — all in a few nanoseconds. For sequential or block-access devices, additional protocols handle the more complex access patterns.

Where memory physically resides in the system determines its speed, capacity, and role.

On-chip memory is fabricated directly on the CPU die itself. This includes:

Being on the same die as the CPU means these memories communicate over extremely short, wide internal buses with no off-chip delays. The physical distance between a register and the ALU may be less than a millimeter. This matters more than you might expect: electrical signals travel at roughly two-thirds the speed of light in copper, which is about 20 centimeters per nanosecond. At a clock frequency of 4 GHz (one cycle = 0.25 ns), a signal can travel only about 5 centimeters in a single clock cycle. This physical constraint is a fundamental reason why on-chip memory must be small and close to the execution units — the speed of light itself limits how far data can travel within one clock cycle.

On-board memory resides on the motherboard but not on the CPU die:

The communication between the CPU and DRAM occurs over the memory bus (also called the front-side bus in older systems, or the memory channel in modern architectures). This bus is much narrower and slower than the on-chip interconnects, and the signals must travel through the CPU package, across the motherboard's PCB traces, through the DIMM socket connector, and along the traces on the DIMM module itself — a total distance of 10–20 centimeters. At the speed of light in copper, this round-trip alone takes about 1–2 nanoseconds, but the protocol overhead (address setup, row activation, CAS latency, data transfer) adds much more. This physical and protocol overhead explains the large latency gap between on-chip caches and off-chip DRAM.

External memory connects to the system through I/O interfaces:

Each level outward introduces more latency, more capacity, lower cost per bit, and greater physical distance from the CPU. This is the memory hierarchy in physical form — and it is a recurring theme throughout this course. The hierarchy can be extended conceptually even further: cloud storage adds another level beyond local external storage, with latencies measured in tens of milliseconds but capacities measured in petabytes.

Cutting across the categories above, memory technologies can also be grouped by the physical phenomenon used to store data.

Semiconductor memory stores data using electronic circuits fabricated on silicon wafers, typically using CMOS (Complementary Metal-Oxide-Semiconductor) technology. The major semiconductor memory types are:

Semiconductor memories dominate the upper levels of the hierarchy because they have no moving parts, operate at electronic speeds (nanoseconds), and can be directly addressed by the CPU. All semiconductor memory chips share a common physical structure: a two-dimensional array of memory cells, with peripheral circuitry for address decoding (row and column decoders), sense amplifiers (to detect the tiny signals from individual cells), and I/O buffers. The fundamental unit is the memory cell — whether it is a six-transistor SRAM cell, a one-transistor-one-capacitor DRAM cell, or a floating-gate Flash cell.

Magnetic memory stores data as patterns of magnetization on a ferromagnetic surface:

Optical memory stores data as physical patterns on a disc surface, read by a laser:

Each optical format comes in three variants:

The naming progression — infrared to red to blue — reflects the physical principle: shorter wavelengths can resolve smaller features, enabling higher data density. Although optical media has largely been displaced by USB drives and streaming services for consumer use, it remains relevant in archival contexts: pressed Blu-ray discs are extremely durable, resistant to magnetic fields, and can be stored for decades.

Emerging technologies that do not fit neatly into the three categories above include:

These technologies represent the frontier of memory research, each attempting to combine the best properties of existing technologies — the speed of SRAM, the density of DRAM, and the non-volatility of Flash — in a single device.

A few observations about this table:

The table makes the trade-offs starkly visible: no single technology excels on all dimensions simultaneously. Fast memory is expensive and small; cheap memory is large and slow. This is precisely why the memory hierarchy exists — it allows the system to provide the illusion of fast, large, cheap memory by automatically keeping frequently used data in the faster levels and pushing rarely used data to the slower ones.

Notice also the enormous latency gaps between levels. Going from L1 cache (1 ns) to DRAM (100 ns) is a 100x slowdown. Going from DRAM to an HDD (10 ms) is a 100,000x slowdown. To put this in human terms: if an L1 cache access took 1 second (the time to grab a book from the shelf next to your desk), a DRAM access would take about 1.5 minutes (a walk to the next room), and an HDD access would take about 115 days (a voyage across the ocean and back). These dramatic ratios explain why caching and virtual memory are so critical — even a small improvement in cache hit rate or a modest reduction in page faults translates into enormous performance gains, because the penalty for a miss is so severe.

You were first introduced to virtual memory in L07, in the context of Protected Mode: the 80386 processor added a paging unit that divides memory into 4 KB pages, uses a TLB (Translation Lookaside Buffer) with a greater than 99% hit rate to speed up translations, and supports demand paging so that not all of a program's memory needs to be physically present in RAM at once. That introduction explained what paging is and why it was added to Protected Mode. Now we go deeper into the mechanics: how virtual addresses are translated to physical addresses step by step, what happens when a page is not in memory, how multi-level page tables make the scheme practical, and how the hardware and operating system cooperate to manage memory transparently. By the end of this section, you should be able to trace the complete path of a memory access from virtual address to physical location, including the exceptional case where the data is not in RAM at all.

To understand why virtual memory was invented, consider the alternative — a system without it. In the early days of computing, and in the Real Mode environment you program in during labs, programs access physical memory directly. If your program writes to address 0x8000, it writes to physical byte 0x8000 in the RAM chips. This creates several serious problems as systems grow more complex:

Problem 1: No isolation between programs. If two programs are loaded into memory simultaneously, nothing prevents one from reading or overwriting the other's data. A buggy program can corrupt the operating system itself — or worse, a malicious program can deliberately read passwords and private data from other programs' memory. In your DOSBox environment, you could (if you were careless) overwrite the interrupt vector table at address 0000:0000 and crash the entire system — and there is no mechanism to prevent this. In a multi-user system (a university server, a cloud computing platform), this is a security and reliability catastrophe.

Problem 2: Memory fragmentation. Suppose you have 16 MB of RAM and three programs are loaded: Program A occupies addresses 0–4 MB, Program B occupies 4–8 MB, and Program C occupies 8–10 MB. Now Program B exits, freeing the 4–8 MB range. You want to load Program D, which requires 6 MB of contiguous memory — but the largest contiguous free block is only 4 MB (where B was). The 6 MB region at the top (10–16 MB) is available, but Program D needs 6 MB contiguous. The total free memory (10 MB) is more than sufficient, but it is fragmented into non-contiguous pieces. Without virtual memory, you would need to physically relocate Program C in memory to consolidate the free space — a slow operation that also requires updating every pointer and address in Program C, which is essentially impossible if the program is running.

This problem is called external fragmentation, and it plagues every system that allocates contiguous physical memory to processes. Over time, as programs start and stop, memory becomes increasingly fragmented until large allocations become impossible despite ample total free memory. Virtual memory eliminates this problem entirely: since virtual-to-physical mapping is done at the page level, the OS can assemble a "contiguous" virtual allocation from physically scattered frames without the program knowing.

Problem 3: Programs limited to physical RAM size. If your machine has 512 MB of RAM, no single program can use more than 512 MB (minus what the OS and other programs consume). For applications like video editing, scientific simulation, or large databases, this is a crippling limitation. Before virtual memory, programmers had to manually divide large programs into "overlays" — sections of code that were explicitly loaded into and out of memory by the programmer. This was tedious, error-prone, and placed an enormous burden on programmers to manage memory manually. Virtual memory automates this process entirely.

Problem 4: Address conflicts. Every program must be compiled or linked to run at a specific memory address (or be position-independent, which adds complexity and overhead). If two programs are both compiled to start at address 0x10000, they cannot run simultaneously without relocation.

Problem 5: No fine-grained memory protection. Even if you could isolate programs into separate physical regions, there is no mechanism to enforce permissions at the page level. You cannot mark certain pages as read-only (to protect code from being overwritten), or as non-executable (to prevent data buffers from being executed as code — a common security exploit known as a buffer overflow attack). Without hardware-enforced page-level protection, the operating system has no way to prevent a user program from modifying kernel data structures or executing arbitrary code injected into a data buffer. The page table entries in a paging system include permission bits for each page, allowing the hardware to enforce read/write/execute restrictions on every single memory access — the foundation of modern operating system security.

Virtual memory solves all five problems through a single elegant mechanism: indirection. Instead of letting programs access physical memory directly, the system interposes a translation layer. Each program operates in its own virtual address space — a private, contiguous range of addresses that belongs to that program alone. The hardware translates every memory access from the virtual address the program uses to the physical address where the data actually resides in RAM.

From the program's perspective, it has a large, contiguous, private memory space starting at address 0 (or, more precisely, from the lowest user-accessible address — the exact layout depends on the OS). From the hardware's perspective, the program's data may be scattered across non-contiguous physical frames, or may not even be in RAM at all (it could be on disk, waiting to be loaded on demand). The translation layer maintains this illusion transparently — the program never needs to know where its data physically resides, and it cannot discover or interfere with other programs' memory.

This is the same principle of abstraction you have seen in other contexts: just as the segment:offset addressing in Real Mode abstracted physical addresses behind a segmentation layer, virtual memory abstracts physical memory behind a paging layer. The difference is that paging is far more powerful and flexible.

A brief history. The concept of virtual memory was first proposed by German physicist Fritz-Rudolf Güntsch in his 1956 doctoral dissertation. The first working implementation appeared in the Atlas Computer at the University of Manchester in 1962, designed by Tom Kilburn and his team. The Atlas had only 16,384 words of fast magnetic core memory (the equivalent of RAM) but 98,304 words of slower magnetic drum storage (the equivalent of disk). Its "one-level storage" system automatically moved data between the two, giving programmers the illusion of a single, large, fast memory. The Atlas supervisor software — one of the earliest operating systems — handled all the transfers transparently, including page replacement decisions. This was a revolutionary idea at the time, and it is essentially the same principle used by every modern operating system today, more than sixty years later. The Atlas team's insight that hardware and system software could cooperate to create a useful abstraction became one of the most influential ideas in the history of computing.

The most common implementation of virtual memory is paging, which divides both virtual and physical address spaces into fixed-size blocks. (An alternative approach, segmentation, divides memory into variable-size segments based on logical program structure — code segment, data segment, stack segment. The x86 architecture supports both, but as we discussed, modern operating systems use paging almost exclusively because fixed-size pages avoid external fragmentation.)

Pages and frames. The virtual address space is divided into pages, and the physical address space is divided into frames (also called "page frames"). Pages and frames are always the same size — on x86 systems, the standard page size is 4 KB (4,096 bytes = 2^12 bytes), though larger page sizes (2 MB "large pages" and 1 GB "huge pages") are also supported for specialized use cases.

Using 4 KB pages, a 32-bit virtual address space (4 GB) contains exactly 1,048,576 pages (2^20), and a system with 4 GB of physical RAM contains the same number of frames. But here is the key insight: the number of virtual pages can vastly exceed the number of physical frames, because not all virtual pages need to be in RAM at the same time.

Why 4 KB? The page size is a design trade-off. Smaller pages reduce internal fragmentation (wasted space within a page — on average, the last page of any allocation wastes half a page), give finer-grained memory protection, and allow the OS to load only the specific data needed. But smaller pages mean more page table entries (consuming more memory), more TLB pressure (each entry covers less address space), and higher overhead for disk I/O (each page fault loads less data per disk access). Larger pages reduce these overheads but waste more memory due to internal fragmentation. The 4 KB size, chosen by the Intel 80386 designers in the mid-1980s, represented a balanced compromise, and it has persisted as the default ever since — though, as noted above, 2 MB and 1 GB large pages are available for workloads where the trade-off favors larger granularity.

Address translation. The core of paging is the translation from virtual addresses to physical addresses. Here is how it works for a 32-bit system with 4 KB pages:

A 32-bit virtual address is split into two fields:

The page number identifies which page the address falls in. The offset identifies the specific byte within that page. Because pages are 4 KB = 2^12 bytes, the offset needs exactly 12 bits.

The CPU uses the virtual page number to look up the corresponding physical frame number in a data structure called the page table. Each process has its own page table, maintained by the operating system. The page table entry for a given virtual page contains:

Once the frame number is found, the physical address is constructed by combining the frame number with the original offset:

The offset is identical in both the virtual and physical addresses — this is a direct consequence of pages and frames being the same size. Only the page/frame number changes during translation. This is an elegant design: the offset bits pass through the translation unchanged, and only the high-order bits (the page number) need to be looked up. This means the TLB only needs to store mappings for page numbers, not for every individual byte address.

Address translation through paging: the CPU generates a logical address (page number + offset), the page table maps it to a physical frame number, and the offset selects the byte within the frame.

The paging address translation process: the CPU generates a logical address, which is split into a page number and an offset. The page number indexes into the page table to retrieve the corresponding frame number. The frame number is combined with the original offset to produce the physical address used to access main memory.

A concrete example. Suppose your program accesses virtual address 0x00403A7C. In binary, the upper 20 bits are 0x00403 (page number = 1,027) and the lower 12 bits are 0xA7C (offset = 2,684). The CPU looks up page 1,027 in the page table and finds that it maps to physical frame 5,891 (0x01703). The resulting physical address is 0x01703A7C — frame 5,891, offset 2,684. The program has no idea this translation happened; it simply issued a MOV instruction targeting 0x00403A7C, and the hardware transparently redirected it.

Multi-level page tables. There is a practical problem with the scheme described above. A flat page table for a 32-bit address space with 4 KB pages would need 2^20 = 1,048,576 entries. If each entry is 4 bytes, the page table itself consumes 4 MB of contiguous physical memory — per process. With dozens or hundreds of processes running on a modern system, this is unacceptable. The solution is multi-level page tables. Instead of one large flat table, the virtual page number is split into two (or more) indices. On the 32-bit x86, the 20-bit page number is split into a 10-bit page directory index and a 10-bit page table index. The page directory (1,024 entries) points to page tables, and each page table (1,024 entries) points to physical frames. Most page tables for unused regions of the address space do not need to exist at all — the page directory entry is simply marked "not present." This typically reduces the total page table memory from 4 MB to a few tens of kilobytes for a typical process that uses only a small fraction of its 4 GB virtual address space.

On 64-bit systems (x86-64), the address space is so vast (2^48 bytes = 256 TB in current implementations) that four levels of page tables are used: PML4 (Page Map Level 4), PDPT (Page Directory Pointer Table), PD (Page Directory), and PT (Page Table). Each level adds 9 bits of indexing, plus 12 bits for the offset, giving 9+9+9+9+12 = 48 bits. Recent Intel and AMD processors support a fifth level (PML5), extending the virtual address space to 2^57 bytes = 128 PB, though no current system has anywhere near that much physical memory. The multi-level scheme ensures that even with these enormous virtual address spaces, the actual memory consumed by page tables remains proportional to the memory the process actually uses, not the size of the address space it could theoretically address.

The translation described above works smoothly when the target page is present in physical RAM. But what happens when it is not?

The page fault. When the CPU looks up a virtual page in the page table and finds the present bit is 0, the page is not currently in physical memory. The CPU raises a page fault exception — a special type of interrupt (specifically, exception 14 on x86). This transfers control to the operating system's page fault handler, a kernel routine specifically designed to handle this situation.

The page fault handler determines what to do based on the faulting address and the process's memory map:

Walking through a page fault step by step. Consider what happens when your program calls a function that has not been executed yet. The function's code resides on a page that was never loaded into RAM (demand paging means the OS does not load pages until they are actually needed):

From the program's perspective, nothing unusual happened — the function call just took a bit longer than usual (perhaps 100 microseconds instead of a few nanoseconds, but the program has no way to notice this unless it is explicitly measuring time). From the OS's perspective, it performed a complex sequence of operations involving exception handling, disk I/O scheduling, context switching, and page table manipulation, all triggered by a single instruction fetch. This is demand paging in action — the OS loads pages only when they are actually needed, avoiding the overhead of loading an entire program into memory at startup. Most programs have large portions of code that are rarely or never executed (error handlers, seldom-used features), and demand paging ensures those pages never consume physical memory.

Swapping and page replacement. When the OS needs a free physical frame to satisfy a page fault, but all frames are occupied, it must evict an existing page. The OS selects a victim page using a page replacement algorithm — commonly an approximation of LRU (Least Recently Used), which evicts the page that has not been accessed for the longest time. The "ideal" algorithm (known as Belady's optimal algorithm) would evict the page that will not be used for the longest time in the future — but this requires knowledge of future memory accesses, which is obviously impossible in practice. LRU approximates this by using the past as a predictor of the future: a page that has not been used recently is likely to not be needed soon.

The hardware assists the OS in making this decision through the accessed bit and dirty bit in each page table entry. The CPU automatically sets the accessed bit whenever a page is read or written, and sets the dirty bit whenever a page is written. The OS periodically clears the accessed bits and checks which pages have been re-accessed — those that have not are candidates for eviction. Pages with the dirty bit clear are cheaper to evict because they do not need to be written back to disk.

If the victim page has been modified since it was loaded (the dirty bit is set), it must be written back to disk before the frame can be reused. If it has not been modified (the dirty bit is clear), the frame can be immediately reused because the disk still has a valid copy.

The region of disk used to store evicted pages is called the swap space (on Linux) or pagefile (on Windows). On Linux, swap is typically a dedicated disk partition, though it can also be a file. On Windows, the pagefile (pagefile.sys) is a hidden file on the system drive. In both cases, the swap area acts as an extension of physical memory, albeit one that is thousands of times slower.

Thrashing. If the system has too little physical RAM for the combined working sets of all running processes, it can enter a pathological state called thrashing: the OS spends most of its time moving pages between RAM and disk, and very little time actually executing program instructions. Each page loaded for one process evicts a page that another process (or even the same process) will need almost immediately, triggering yet another page fault. The system appears to freeze — the disk light stays on continuously, but nothing gets done.

You can observe thrashing on a real system: open far more applications than your RAM can support, and the computer becomes sluggish to the point of being nearly unusable. On Linux, the vmstat command shows page-in and page-out rates; on Windows, Task Manager's "Memory" tab shows the commit charge relative to physical memory. When page faults per second spike and the disk I/O light stays on, you are witnessing thrashing.

The solutions to thrashing include:

The TLB (Translation Lookaside Buffer). If every memory access required a page table lookup, the overhead would be devastating — each instruction fetch and each data access would require an additional memory read just to consult the page table, effectively halving the CPU's speed. The solution is the TLB, a small, fast hardware cache built into the CPU's Memory Management Unit (MMU) that stores recent page-table translations.

When the CPU needs to translate a virtual address, it first checks the TLB. If the translation is found there (a TLB hit), the physical frame number is available immediately — no memory access needed. Only on a TLB miss does the CPU walk the page table in memory, and it then caches the result in the TLB for future use.

The TLB is remarkably effective. A typical TLB holds 64 to 1,024 entries. With 4 KB pages, even a 512-entry TLB covers 2 MB of memory, and because programs exhibit strong locality of reference (they tend to access the same pages repeatedly over short time intervals), the TLB achieves hit rates above 99% in practice. This means the translation overhead is negligible for the vast majority of memory accesses.

There is one important complication: because each process has its own page table, the TLB entries from one process are invalid for another. When the OS performs a context switch (switching from running Process A to running Process B), it must either flush the entire TLB or use Address Space Identifiers (ASIDs) to tag TLB entries with the process they belong to. Modern x86 processors support ASIDs (called "Process Context Identifiers" or PCIDs on Intel), allowing TLB entries from multiple processes to coexist, which significantly reduces the performance cost of context switches. Without ASIDs, every context switch would cause a burst of TLB misses as the new process rebuilds the TLB from scratch — a measurable performance penalty on systems that switch frequently between many processes.

This is also why large pages (2 MB or 1 GB instead of 4 KB) are useful for certain workloads: a single TLB entry covers a much larger memory region, so the same number of TLB entries can cover far more memory. Databases and virtual machines that work with large, contiguous memory regions often benefit significantly from large pages.

Virtual memory maps each process's virtual address space to physical RAM — some pages reside in RAM, others are swapped to disk.

Virtual memory mapping: each process has its own virtual address space, and the operating system maps virtual pages to physical frames in RAM. Pages not currently needed can reside on disk. The mapping is maintained per-process, providing complete isolation — Process A's page 0 and Process B's page 0 can map to entirely different physical frames.

Virtual memory is one of the most important abstractions in computer architecture, and it is realized through a partnership between hardware and software:

The hardware provides:

The operating system provides:

This hardware-software partnership is a recurring theme in computer architecture. The hardware provides fast mechanisms (translation, caching, exception generation), while the software provides flexible policies (which pages to load, which to evict, how to handle errors). Neither alone would be sufficient — pure hardware could not make complex policy decisions, and pure software could not intercept every memory access without catastrophic performance overhead.

Virtual memory enables several powerful techniques beyond basic isolation and swapping.

Copy-on-write (COW). When a process creates a child process (via the fork() system call on Unix/Linux), the OS does not immediately copy all of the parent's memory. Instead, it sets up the child's page table to point to the same physical frames as the parent, but marks all shared pages as read-only in both page tables. Both processes continue executing, reading from the shared frames with no overhead. Only when one of them tries to write to a shared page does the hardware trigger a page fault (because the page is marked read-only). The OS then copies that single page to a new frame, updates the writing process's page table to point to the copy, and marks it read/write. This "lazy copying" approach is far more efficient than duplicating all memory upfront, because many pages (especially code pages) are never modified by the child process.

Memory-mapped files and shared memory. The OS can also map a file on disk directly into a process's virtual address space, so that reading from those virtual addresses reads the file and writing to them writes the file — no explicit read() or write() system calls needed. Similarly, two processes can have page table entries that point to the same physical frames, allowing them to share data through a common region of memory. Both techniques are widely used in modern operating systems and would be impossible without virtual memory. The OS can map a file on disk directly into a process's virtual address space, so that reading from those virtual addresses reads the file and writing to them writes the file — no explicit read() or write() system calls needed. Similarly, two processes can have page table entries that point to the same physical frames, allowing them to share data through a common region of memory. Both techniques are widely used in modern operating systems and would be impossible without virtual memory.

The x86 paging history illustrates this evolution. The original 8086 (1978) and 80286 (1982) had no paging support at all — they relied on segmentation for memory management. The Intel 80386 (1985) was the first x86 processor to include a paging unit, adding it on top of the existing segmentation mechanism. In Protected Mode on the 386, an address goes through two translations: first segmentation (logical address to linear address), then paging (linear address to physical address). Modern x86-64 operating systems effectively bypass segmentation (by setting all segment bases to 0 and limits to maximum) and rely entirely on paging for memory management. The segmentation legacy remains in the architecture for backward compatibility, but paging does all the real work. This is an instructive example of how hardware architectures evolve: features that seemed essential in one era (segmentation in the 80286) can become vestigial in the next, while newer features (paging in the 80386) prove so successful that they become the foundation of all modern memory management.

Virtual memory also closes the loop on the memory hierarchy from L06. Recall the hierarchy: registers, L1 cache, L2 cache, L3 cache, main memory (DRAM), storage (SSD/HDD). Virtual memory is the mechanism that connects the bottom two levels — DRAM and disk. Just as the cache hardware automatically moves data between DRAM and the cache levels, the virtual memory system automatically moves data between disk and DRAM. At every level, the principle is the same: keep frequently used data close to the CPU, and let rarely used data sink to slower, cheaper, larger storage. The result is a system that behaves almost as fast as the fastest level and almost as large as the largest level — the best of both worlds, achieved through the elegant use of indirection and caching at every layer.

Consider the complete path of a single memory access in a modern system. Your program executes MOV EAX, [address]. The CPU first checks the L1 data cache — hit? Done in 1 ns. Miss? Check L2 — done in 5 ns. Miss? Check L3 — done in 15 ns. Miss? Access DRAM — done in 100 ns. But the MMU is operating in parallel: the virtual address must be translated to a physical address. The TLB handles this in one cycle for the 99%+ common case. On a TLB miss, the CPU walks the multi-level page table (additional DRAM accesses). On a page fault, the OS loads the page from disk — microseconds for an SSD, milliseconds for an HDD. Each layer is a cache for the one below, each managed by progressively more complex mechanisms (hardware logic for L1–L3, hardware-software cooperation for virtual memory). The beauty of this design is that the program sees none of this complexity — it simply accesses a memory address, and the system delivers the data as quickly as the hardware allows. This layered transparency, where each level of the hierarchy is invisible to the one above it, is one of the great achievements of computer systems architecture and a principle you will encounter repeatedly throughout your career in computing.

In Part 3 of this lecture, we shift from hardware back to software. The memory hierarchy we just explored — registers, cache, DRAM, disk — is the stage on which all your data lives. Now the question becomes: how do you organize that data? High-level languages give you arrays, structs, linked lists, and hash maps with convenient syntax. But underneath, every one of these structures is just a pattern of bytes laid out in contiguous or pointer-linked memory. In this section and the three that follow, we will implement these patterns directly in x86 assembly, so you can see exactly what the compiler does on your behalf.

We start with arrays — a topic you already encountered in L05. This section is intentionally brief: we review the essentials, then extend to two-dimensional arrays with a concrete access formula and a word-sized matrix example.

In L05, you learned that an array in assembly is simply a sequence of values stored at consecutive memory addresses. There is no runtime bounds checking, no .length property, no type metadata — just raw bytes. You declare an array using DB, DW, or DD, and you access elements by calculating their offset from the base address.

Static access is straightforward — the assembler computes the offset at assemble time:

Dynamic access requires a register to hold the calculated offset:

The universal formula is: address = base + index x element_size. For a byte array, element_size is 1, so the index is the offset. For a word array, element_size is 2, so element 3 lives at base + 6. This formula is worth memorizing — it reappears in every language and every architecture.

Declaring and accessing 1D arrays in TASM: byte arrays, word arrays, and direct element access with constant offsets and register indexing.

A 2D array (a matrix) adds one layer of complexity: you have rows and columns, but memory is one-dimensional. How do you flatten a 2D structure into a 1D sequence of bytes?

The standard solution is row-major order — you store row 0 first, then row 1, then row 2, and so on. This is the convention used by C, Python (NumPy), and most other languages. (Fortran famously uses column-major order instead, storing column 0 first, then column 1, etc. The formulas differ, but the underlying idea is the same.)

Declaring 2D arrays in TASM: a 3x4 byte matrix, a 2x3 word matrix, and an uninitialized buffer — all stored in row-major order.

In memory, these 12 bytes are stored contiguously. If we visualize the raw memory starting at the base address of matrix, we see:

Row 0 occupies offsets 0-3, row 1 occupies offsets 4-7, and row 2 occupies offsets 8-11. The row boundaries are invisible to the hardware — the CPU sees a flat sequence of 12 bytes. It is your code that interprets them as a 3x4 grid. To access element matrix[row][col], you need the formula:

address = base + (row x COLS + col) x element_size

The expression row x COLS converts the 2D coordinates into a linear index — it skips over all complete rows before the one you want. Adding col then indexes within that row. For byte-sized elements, element_size is 1, so you can drop the final multiplication. For word-sized elements, you multiply by 2 at the end.

Let us access matrix[1][2], which should be 70 (second row, third column, counting from zero):

Accessing matrix[1][2]: compute the offset as base + (row x COLS) + col, then load the value.

Walk through it step by step: MUL CL computes 1 x 4 = 4 in AX. Adding that to BX moves us past row 0 (4 bytes) to the start of row 1. Adding 2 moves us to column 2 within row 1. The byte at that address is 70 — exactly matrix[1][2].

To verify manually: the linear index is 1 x 4 + 2 = 6. Looking at our memory layout above, offset 6 holds the value 70. The formula works.

For word-sized elements, you would multiply by 2 at the end (or shift left by 1), since each element occupies 2 bytes. The formula is the same; only the element size changes.

One subtlety worth noting: the MUL instruction always uses AL (or AX) as an implicit operand and stores the result in AX (or DX:AX). This means you must load the row number into AL and clear AH before multiplying. If you forget mov ah, 0, any leftover value in AH will corrupt the multiplication result — a common bug in student code.

For larger matrices with word-sized elements, the formula requires one more step:

The SHL AX, 1 is equivalent to multiplying by 2 but faster than a second MUL. You will see this shift-instead-of-multiply optimization frequently in hand-written assembly and even in compiler output for power-of-two element sizes. For a doubleword array (4-byte elements), you would use SHL AX, 2 instead.

A note on performance. The MUL instruction is relatively slow on the 8086 (taking 70-77 clock cycles for a byte multiply). In performance-critical code that iterates over a 2D array, programmers often avoid MUL entirely. Instead of recalculating row x COLS from scratch on each iteration, they maintain a running pointer and simply add COLS to it each time they advance to the next row. This transforms a multiplication into a single addition — a technique you will see in optimized inner loops.

This is all the machinery behind statements like int matrix[3][4]; ... matrix[1][2] = 70; in C. The compiler generates exactly this kind of address computation every time you index into a multidimensional array.

Arrays are useful when every element has the same type and size. But real-world data rarely fits that mold. A student record has a name (string), an age (byte), and a GPA (floating-point number). A pixel has red, green, and blue channels (three bytes). A network packet header has fields of different widths packed together. In high-level languages, you use a struct (C) or a class (Java/Python) to group these heterogeneous fields. In assembly, the TASM assembler provides the STRUC / ENDS directive pair to accomplish the same thing.

A STRUC definition tells the assembler "here is a blueprint for a data layout." It does not allocate any memory — it only defines field names and their offsets within the structure.

Defining a Person struct with STRUC/ENDS, creating two instances, and accessing fields with dot notation — person1.age and person1.height.

This is analogous to writing struct Person { uint8_t age; uint16_t height; }; in C. The assembler records three things: (1) the total size of Person is 3 bytes, (2) the field age lives at offset 0 within the structure, and (3) the field height lives at offset 1. No bytes are emitted into the program yet — the STRUC block is purely a compile-time construct.

The ? initializer means "uninitialized" — the fields have no default value. You could also specify defaults: age DB 0 would make zero the default age for any instance that does not override it. When you later create an instance with Person <> (empty angle brackets), the default values are used. When you write Person <25, 180>, you override the defaults with specific values.

To actually allocate memory for a Person, you declare an instance in the .data segment:

The angle-bracket syntax < > provides initial values for each field, in order. Using ? means "leave uninitialized" (the assembler may fill in zeros or whatever the default is). You could also write person1 Person <25, 180> to initialize age to 25 and height to 180 at assemble time.

Here is where the struct syntax pays off. Instead of remembering that height lives at offset 1 from the start of the structure — and manually adding that offset every time — you write the field name directly:

Key distinction: the STRUC block is only a template (no memory allocated); the instance declarations (person1, person2) are what actually reserve bytes in memory.

The dot notation person1.age is purely an assembler convenience — syntactic sugar, if you like. At the machine level, person1.age resolves to the address of person1 plus the offset of the age field (which is 0), and person1.height resolves to the address of person1 plus the offset of height (which is 1). The CPU sees ordinary memory addresses — it has no concept of "fields" or "structures." If you disassemble the compiled code, you will see plain MOV instructions with numeric addresses, with no trace of the dot notation.

The benefit is entirely for the programmer: field names make your intent clear and prevent the kind of off-by-one errors that arise when you manually compute offsets.

Let us trace exactly what sits in memory after executing the code above. Suppose person1 is allocated at address 0100h:

Notice that height occupies two bytes in little-endian order (low byte first), as is standard on x86. The value 180 is 00B4h: the low byte B4h is stored first (at the lower address 0101h), and the high byte 00h follows at 0102h. If you ever inspect memory in a debugger and see B4 00 where you expected to see 180, this is why — little-endian byte ordering.

The two Person instances are packed contiguously with no padding — unlike C compilers on 32-bit or 64-bit platforms, which often insert alignment padding between fields or between structures to ensure that multi-byte fields start at addresses that are multiples of their size. In 16-bit Real Mode with TASM, what you declare is exactly what you get. The assembler does not add any hidden bytes.

When you have a pointer to a struct (for example, the address of a node in a linked list), you access fields using the bracket-dot syntax:

This pattern is essential for the linked list traversal we will write in Section 14. When BX holds the address of any Person, [bx].age always reads the right byte, regardless of which instance BX points to.

This distinction trips up many students, so let us be explicit. The STRUC/ENDS block is a template — think of it as a cookie cutter. It defines a shape but produces nothing tangible. The person1 Person <?, ?> declaration is an instance — it is the actual cookie, the physical bytes in your program's data segment.

You can define as many templates as you like without increasing your program's memory footprint by a single byte. Only instances consume memory. This is exactly how struct definitions work in C: struct Person { ... }; defines a type but allocates nothing; struct Person p1; allocates memory.

One more practical point: you can use SIZE Person to get the total byte count of the structure (3 in our case) and OFFSET Person.height to get the field's offset within the template (1 in our case). These are assembler-time constants — they do not generate any runtime instructions. They are useful when writing general-purpose routines that need to iterate over arrays of structures, where you advance a pointer by SIZE Person bytes per iteration.

With structs in hand, we can tackle a more ambitious data structure: the linked list. In a linked list, each element (called a node) contains some data and a pointer to the next node. Unlike an array, the nodes do not need to be contiguous in memory — they can be scattered anywhere, connected only by their pointers.

Why does this matter at the assembly level? Arrays have a fixed size determined at declaration time. Inserting an element in the middle of an array requires shifting all subsequent elements — and in assembly, that means writing a byte-by-byte or word-by-word copy loop, which is both tedious and slow for large arrays. Linked lists, by contrast, allow O(1) insertion and deletion at any point — you just adjust a few pointers. The tradeoff is that you lose O(1) random access: to reach the n-th element, you must traverse the list from the beginning, following each next pointer one by one.

In high-level languages, the standard library hides these details. In assembly, you see every pointer dereference, every NULL check, and every field offset calculation. This transparency is the point of the exercise.

In assembly, a "pointer" is simply a 16-bit value — an offset within the data segment. There is no special pointer type, no automatic dereferencing, and no garbage collector. You manage every pointer manually. This makes linked list code in assembly more verbose than in C, but it also makes every step completely transparent — there is no hidden machinery.

We build on the Person struct from Section 13 by embedding it inside a Node that also contains a next pointer:

Defining a Node struct that contains a Person value (3 bytes) and a next pointer (2 bytes), with three statically allocated nodes.

The Person <> syntax creates an embedded instance with default values. Notice that TASM allows you to nest structures: the Node template contains a Person template as its first field. This is conceptually identical to embedding one struct inside another in C (struct Node { struct Person value; uint16_t next; };).

The next field is a word-sized offset that will hold the address of the following node in the list, or a sentinel value (-1, i.e., FFFFh) to mark the end of the list.

Since we are working in a simple .COM-style program without dynamic memory allocation (no malloc), we pre-allocate the nodes in the .data segment:

The SIZE Node operator returns 5 (the number of bytes in one Node), so each declaration allocates 5 zero-initialized bytes. We use DB ... DUP(0) rather than Node <...> here to show that the allocation is really just a block of bytes — the Node label is a human convenience, not a hardware feature. You could equally write head Node <, ?> using the struct syntax — the result in memory is identical.

Now we fill in the data and wire up the next pointers to form the chain: head -> node2 -> node3 -> NULL. This is the assembly equivalent of building a linked list by hand in C — assigning values to fields and setting each node's next pointer to the address of the following node.

Initializing the linked list: setting each node's data fields and linking them with OFFSET — head -> node2 -> node3 -> NULL (-1).

A few things to notice. The nested dot syntax head.value.age works because TASM resolves it step by step: head gives the base address of the first node, .value adds the offset of the embedded Person (which is 0, since it is the first field), and .age adds the offset of age within Person (also 0). So head.value.age resolves to the very first byte of head. Meanwhile, head.value.height resolves to head + 0 + 1 = head + 1, and head.next resolves to head + 3 (skipping the 3-byte Person).

We use -1 (FFFFh) as the NULL sentinel rather than 0 because address 0 could theoretically be a valid offset in the data segment. Using FFFFh as an invalid marker is a common convention in 16-bit code, though you could use any agreed-upon sentinel. In 32-bit and 64-bit C, NULL is typically 0, but the concept is the same: a reserved value that means "this pointer does not point to any valid node."

The canonical linked list operation is traversal: start at the head, process each node, follow the next pointer, and stop when you hit NULL. This is the assembly equivalent of a while loop that walks through every node. Here we increment the age of every person in the list:

Traversing the linked list: follow the next pointer from node to node, incrementing each person's age, until reaching the NULL sentinel.

Step through this mentally. On the first iteration, BX points to head. We read head.value.age (25), add 1 to get 26, and store it back. Then mov bx, [bx].next loads the word at head + 3, which is the offset of node2. BX now points to node2. On the second iteration, we increment node2's age from 30 to 31 and load node2.next, which is the offset of node3. On the third iteration, we increment node3's age from 22 to 23 and load node3.next, which is FFFFh. The CMP sets the zero flag, JNE does not jump, and the loop ends.

This is exactly the pattern that a C compiler generates for:

The only difference is that in C, the compiler manages the pointer arithmetic and the NULL check for you. In assembly, you do it all explicitly — which is precisely why studying this builds deep understanding of how data structures actually work at the machine level.

Let us visualize the complete memory layout after initialization. Suppose head starts at offset 0200h:

Each node is exactly 5 bytes: 1 for age, 2 for height, and 2 for the next pointer. The nodes happen to be contiguous here (because we declared them one after another in .data), but that is not a requirement. The beauty of the linked list is that head.next could point to any address in the data segment — the nodes could be scattered across memory, and the traversal code would work identically. The pointer is what creates the logical sequence, not the physical address order.

Notice also that if you wanted to insert a new node between head and node2, you would only need to change two pointers: set the new node's next to point to node2, and set head.next to point to the new node. No data needs to be shifted. Compare this to inserting into the middle of an array, where every element after the insertion point must move one position — an O(n) operation that the linked list avoids entirely.

Two mistakes appear frequently in student linked list code:

These are not just assembly pitfalls — they are the exact same bugs that plague C programs with linked lists (null pointer dereference, use-after-free). Seeing them at the byte level here will make you a better debugger in any language.

Our final data structure combines two ideas you already know — strings and pointers — into a pattern that appears everywhere in systems programming: an array of pointers to strings.

Suppose you want to store five strings. If you try to put them in a regular array, you immediately hit a problem: strings have variable length. "Hi" is 2 characters; "Assembly Language" is 17 characters. You cannot declare stringArray DB 5 DUP(...) because each element would need a different number of bytes.

You could pad every string to some maximum length — say, 32 bytes — and store them in a flat array of 5 x 32 = 160 bytes. But this approach wastes memory (short strings leave most of their slot empty) and complicates processing (you need to know the maximum length everywhere). Worse, if even one string exceeds the maximum, you have to resize the entire array. There is a better way.

The standard solution — used by C, by operating systems, and by virtually every language runtime — is indirection: store the strings themselves somewhere in memory, and then create a separate array of pointers (offsets) that point to the start of each string. Each pointer is a fixed-size word (2 bytes in 16-bit mode), so the pointer array is a regular, uniform array that you can index with the formulas from Section 12. The strings can be any length — the pointer array does not need to know or care.

Declaring an array of string pointers and the string data itself — each string is $-terminated for DOS INT 21h output.

The strings are declared separately, each terminated with $ (the DOS string terminator for INT 21h, function 09h). Note that $ is not the same as C's null terminator (\0); it is a DOS convention specific to the print-string function. The pointer array strArray is initially filled with zeros — we will set the pointers in the .code segment at runtime.

Each entry in strArray must hold the offset of the corresponding string:

Initializing the pointer array: each word-sized entry receives the OFFSET of its corresponding string.

Notice the indices: strArray[0], strArray[2], strArray[4], and so on — incrementing by 2 because each pointer is a word (2 bytes). This is the base + index x element_size formula from Section 12 in action, with element_size = 2.

After initialization, the pointer array in memory looks like this:

Each 2-byte slot contains the starting address of the corresponding string. The strings themselves can be anywhere in the data segment and can have wildly different lengths — the pointer array does not care.

Now we can iterate through the pointer array and print each string using DOS interrupt 21h, function 09h (which expects the string address in DX and prints characters until it encounters the $ terminator):

Printing all strings: loop through the pointer array, load each address into DX, and call DOS INT 21h function 09h.

The output looks like this:

The output: five strings printed sequentially, demonstrating that the pointer array successfully references each string's location in memory.

Notice how the loop body is agnostic to string lengths. It does not need to know that "Hello, World!" is 13 characters and "TASM Program" is 12. It simply loads a pointer and hands it to DOS. The INT 21h handler walks through memory starting at that pointer, printing characters until it hits $. The variable-length complexity is entirely hidden behind the pointer indirection.

This pattern — an array of pointers to variable-length data — is one of the most fundamental structures in systems programming. Consider:

The key insight is that indirection (storing a pointer to data rather than the data itself) solves the variable-size problem. Each pointer is a uniform 2 bytes, so the pointer array is easy to index. The actual strings can be any length, stored anywhere in memory. This separation of "index structure" from "payload data" appears at every level of computer systems, from file system inodes to database indexes to virtual memory page tables.

Look at how much ground we have covered in this part of the lecture. We started with flat arrays (Section 12), where every element has the same size and lives at a predictable offset. We then introduced structs (Section 13) to group fields of different sizes under named offsets. We combined structs with pointers to build linked lists (Section 14), where nodes reference each other through stored addresses. And finally, we used pointer arrays (this section) to manage variable-length data by adding a layer of indirection.

Each of these structures exists in memory as nothing more than a sequence of bytes. The CPU does not know whether a particular byte is an age field, a pointer to the next node, or the letter 'H' in a string. All the meaning comes from how your code interprets those bytes — which offsets you read, how many bytes you fetch, and which address you follow next.

That is the fundamental lesson of low-level data structures: structure is not a property of the data, but of the code that reads it. The same 5 bytes could be a Node (with age, height, and a next pointer) or 5 unrelated byte values — the CPU cannot tell the difference. It is the programmer (or the compiler, on the programmer's behalf) who imposes meaning on raw memory. Understanding this dissolves the mystery of how high-level data structures "really work" — they work exactly like this, just with more syntactic sugar on top.

What you'll do:

Connection to this lecture: