Operating systems
Processes, scheduling, deadlock, memory and the Linux layer under your containers, split into what SDE-1 asks and what waits.
The operating system is the layer that shares one CPU, one memory space and one disk between many programs. Interviewers ask about it because every backend bug eventually lands there: a slow request, a full connection pool, a process killed at 3 a.m.
It is also the vocabulary for concurrency and for every container question. Read this page in weeks 1 to 2, then keep answering the questions out loud until they are boring.
Priority for SDE-1
The must list is short: process vs thread, context switch, scheduling, synchronisation, deadlock, paging and the page fault walk. Page cache internals, cgroups, the OOM killer and io_uring are SDE-2 reading. Skip them until you have an offer.
SDE-1 scope vs SDE-2 scope
| Area | SDE-1 must | SDE-2 later |
|---|---|---|
| Processes and threads | What is shared, what is private, why a thread is cheaper | Thread pool sizing under real load, stack growth |
| CPU scheduling | FCFS, SJF, round robin, and the waiting-time arithmetic | CFS and EEVDF, nice values, CPU affinity |
| Synchronisation | Mutex vs semaphore, race condition, condition variable | Futex internals, lock contention profiling |
| Deadlock | The four Coffman conditions, and how to break each | Lock graphs in production, deadlock detection at scale |
| Memory | Paging, page table, TLB, page fault, virtual vs physical | Huge pages, NUMA, memory fragmentation |
| Files and I/O | write, fsync, buffered vs direct I/O, sequential vs random | Page cache reclaim, readahead, io_uring, kernel bypass |
| Linux basics | fork, execve, signals, exit codes, kill -9 | cgroup limits, OOM killer, namespaces at depth |
| Containers | A container is a process with namespaces and cgroup limits | Image layers, runtime isolation, seccomp profiles |
Roadmap
Stage 1: the basics (weeks 1 to 2)
Read OSTEP chapters on processes, threads, scheduling, locks and memory. Do not read the whole book. Five chapters are enough to answer everything in this stage: processes, threads, CPU scheduling, address spaces, and paging.
Draw the process state diagram once, by hand. Then draw a page table walk for a two-level table. Those two drawings carry most of the round.
Stage 2: interview core (weeks 3 to 7)
This stage runs beside Ledgerline, so use the build as your examples. When you add ordered locking in week 3, write down what would deadlock without it. When you rely on Postgres for durability, write down what a crash between the write and the fsync would lose.
Answer the SDE-1 questions below out loud, recorded, once a week. Reread nothing. If an answer stalls, note the gap and fix only that gap.
Stage 3: SDE-2 depth (weeks 8 to 15, optional now)
The second half of the fundamentals block has room for the parts an SDE-2 interview probes: how the page cache behaves under write load, what a cgroup memory limit actually does, when the OOM killer fires, and why io_uring exists.
Treat this as reading, not drilling. If a subject from stage 2 is still shaky at week 8, spend the time there instead.
Core topics, in order
| Topic | What to be able to explain | Level |
|---|---|---|
| Process vs thread | Address space, file descriptors and signal handlers are per process; heap and code are shared by threads. | SDE-1 must |
| Context switch | Saving one thread's registers and stack pointer, loading another's, and why the TLB reload makes it costly. | SDE-1 must |
| CPU scheduling | Run FCFS, SJF and round robin over three jobs by hand and compare average waiting time. | SDE-1 must |
| Synchronisation | A mutex guards a critical section; a semaphore counts permits; a condition variable waits for a state change. | SDE-1 must |
| Deadlock | Mutual exclusion, hold and wait, no preemption, circular wait. Break one and the deadlock cannot form. | SDE-1 must |
| Paging | Virtual pages map to physical frames through a page table; the TLB caches recent translations. | SDE-1 must |
| Page fault | The CPU traps, the kernel finds or creates the frame, updates the table, reloads the instruction. | SDE-1 must |
| Virtual memory | Each process sees a private, contiguous address space that may exceed physical RAM. | SDE-1 must |
| File systems and I/O | write copies into the page cache; fsync forces it to disk. Sequential beats random. | SDE-1 nice |
| Signals | Asynchronous notifications: SIGTERM asks, SIGKILL cannot be caught, SIGSEGV means a bad access. | SDE-1 nice |
fork and execve | fork clones the process, execve replaces its image. A shell is a loop of both plus wait. | SDE-1 nice |
| Page cache behaviour | Reads and writes hit RAM first; dirty pages flush later. This hides disk latency and sets your durability window. | SDE-2 later |
| cgroups and OOM | A cgroup caps memory for a group of processes; going over the limit triggers reclaim, then the OOM killer. | SDE-2 later |
| Containers | Namespaces hide PIDs, mounts and network; cgroups cap CPU and memory. There is no virtual machine. | SDE-2 later |
| io_uring and kernel bypass | Submission and completion queues avoid a syscall per operation; bypass skips the kernel entirely. | SDE-2 later |
Four commands worth typing once
You do not need to be a systems programmer for SDE-1. You need to have seen these run once.
ps -eLfshows threads as separate rows under one process. That is the process vs thread answer, visible.strace -f -e trace=openat,write,fsync ./yourservershows every file call your program makes. Point at the fsync.cat /proc/<pid>/statusshows voluntary and involuntary context switches, plus the resident set size in kB.free -mseparates total, used, buffered and cached. The cached column is the page cache, not wasted memory.
The page cache is not a leak
If free shows most memory as cached, nothing is wrong. The kernel reclaims that memory when a process needs it. Saying this out loud is a small, credible signal.
Resources
Learn
- Operating Systems: Three Easy Pieces: the free book this page follows. Read the process, scheduling, memory and persistence chapters in that order.
- xv6: a simple, Unix-like teaching operating system: a real kernel small enough to read in a month. Read the chapters on traps and page tables when a definition feels hollow.
- CSE 120: paging slides: worked page table arithmetic and replacement examples, which is exactly the exam shape interviewers use.
Practice
- fork(2) Linux manual page: read the return values and the warning about threads. Write a 20-line program that forks and waits.
- signal(7) Linux manual page: the table of signals and default actions. Know
SIGTERM,SIGKILL,SIGINTandSIGSEGV. - namespaces(7) Linux manual page: the seven namespace types. Run
unshare --pid --fork --mount-proc bashonce and look around.
Long-form
- What even is a container: namespaces and cgroups: the clearest short explanation of why a container is not a small VM. Read it before the Docker question.
- Control Group v2: the official reference. Read the memory controller section only, and only at SDE-2 depth.
- Linux Page Cache Hit Ratio: how to measure cache behaviour with
cachestatand friends. Useful when Logsmith benchmarks confuse you. - io_uring(7) Linux manual page: what the queues are for. Read the introduction and stop there.
Questions to answer out loud
Answer each in 60 to 90 seconds, recorded.
- What does a thread share with its process, and what does it not share?
- Why is a context switch expensive? Name the two costs.
- Run FCFS and round robin over three jobs and say which has the lower average wait.
- Why does SJF need a prediction, and what does the scheduler use instead?
- What are the four Coffman conditions? Break one of them in a sentence.
- Mutex or semaphore for a pool of 5 database connections? Why?
- What happens on a page fault, step by step?
- What is the TLB, and what happens on a TLB miss?
- What is the difference between a process and a container?
- What does
fsyncguarantee thatwritedoes not?
SDE-2 questions
- How does the page cache change the cost of a 4 kB random write versus a 1 MB sequential one?
- What happens when a container crosses its cgroup memory limit?
- Why can the OOM killer pick the wrong process, and how do you influence its choice?
- When is io_uring worth the complexity over a thread pool doing blocking reads?
- How do you find which process is causing involuntary context switches on a box?
How this shows up in your projects
- Ledgerline, ordered locking. You take row locks in account ID order to prevent the circular wait. That is Coffman condition four, removed in code.
- Ledgerline, fsync and durability. Postgres commits with fsync by default. The gap between
writeandfsyncis the window where a power cut loses the last transfer. - Ledgerline, connection pools. Each request holds a connection, which is a thread plus a socket plus kernel buffers. An exhausted pool is a scheduling and blocking story, not just a config value.
- Logsmith, append-only files. Sequential appends land in the page cache and flush later. Your three fsync modes are a direct trade of throughput against the durability window.
Common mistakes
- Answer with the mechanism, then the name. "The CPU traps and the kernel loads the frame" before "demand paging".
- Use one project example per answer, in one sentence.
- Draw the page table walk on paper until you can do it without notes.
- Say plainly what you have not used. Claiming cgroup tuning you never did is worse than a gap.