SDE-2 project options
Detailed backend and systems project briefs with algorithms, scaling questions, milestones, and diagrams.
This page is a project menu. It does not add these builds to the 34-week calendar. Finish the core portfolio first, then choose one option that fits the roles you want.
Each brief starts with a working single-node version. The SDE-2 depth comes from making trade-offs explicit, testing failure cases, measuring a workload, and explaining how the design changes as traffic grows. Do not claim production scale from a local demo.
How to use this menu
For one project, build the core path, add tests for its hardest failure, and publish a short capacity note. Treat every stretch item as optional. A diagram, benchmark, and demo should describe the code you actually shipped.
The existing Ledgerline, Logsmith, Shiplane, and Faultline remain available. These options are additional choices, not required work.
Distributed job queue
Build a queue that accepts jobs, assigns them to workers, records acknowledgements, and makes failed jobs visible. A worker lease lets another worker recover a job after the first worker disappears.
Core algorithms: visibility leases, bounded exponential backoff, deduplication keys, and fair worker selection. Explain at-least-once delivery and why job handlers need idempotency.
Build first: one queue, a worker pool, ack, retry, dead-letter inspection, and restart recovery. Add partitions and delayed jobs only after the basic failure path works.
Measure: enqueue and completion throughput, queue wait time, retry count, and recovery time after killing a worker. Test one slow worker and a burst of jobs.
Design discussion: partition key choice, ordering scope, poison jobs, backpressure, and the cost of a centralized scheduler.
URL shortener and redirect platform
Build a service that creates short links, redirects requests, expires links, and reports aggregate clicks. The redirect path needs low latency; analytics should not slow it down.
Core algorithms: compare random IDs with database-generated IDs, encode IDs in base 62, and use a cache policy for popular links. Explain collision handling and hot-key behavior.
Build first: create, resolve, expire, and disable links. Send click events asynchronously and show approximate aggregate counts. Add tenant quotas and abuse controls before adding custom domains.
Measure: redirect p50/p95/p99, cache hit rate, ID collision retries, and click-event lag under a reproducible workload.
Design discussion: read/write ratio, cache invalidation, analytics consistency, shard keys, and how to prevent one popular link from overloading a shard.
Distributed cache
Build a key-value cache with a small client library. Let users choose expiration and eviction settings, then compare behavior under different workloads.
Core algorithms: LRU or LFU eviction, consistent hashing, TTL expiration, and request coalescing to limit cache stampedes. Compare a single-node cache with a sharded version.
Build first: get, set, delete, TTL, bounded memory, and one eviction policy. Add replication or invalidation as a stretch feature, with clear consistency guarantees.
Measure: hit rate, operations per second, p99 latency, memory per key, and behavior when a node leaves the ring.
Design discussion: stale reads, hot keys, replication lag, key skew, and whether the cache can lose data without harming correctness.
Search and indexing service
Index a real, bounded dataset and return ranked results with an explanation of why each result matched. Make indexing and querying separate paths.
Core algorithms: inverted indexes, term-frequency ranking such as BM25, top-k selection, and segment merging. Compare a full rebuild with incremental indexing.
Build first: tokenize documents, search terms and phrases, return snippets, and show score contributions. Add updates and deletions through tombstones before adding shards.
Measure: indexing throughput, query p95/p99, index size, and ranking quality on a small set of hand-labeled queries.
Design discussion: shard by document or term, handle skewed terms, merge segments without blocking searches, and trade freshness for query speed.
Notification delivery platform
Build a service that accepts notification requests and delivers them to mock email, SMS, and push providers. Give operators a view of pending, sent, retried, and failed notifications.
Core algorithms: deduplication, bounded retries with jitter, priority scheduling, per-tenant quotas, and fair queue selection. Separate provider failures from invalid requests.
Build first: request validation, user preferences, one mock provider, retry history, and a dead-letter view. Add multiple channels and tenant fairness next.
Measure: delivery lag, success rate by provider, retry volume, queue age, and per-tenant fairness during a burst.
Design discussion: provider rate limits, duplicate delivery, priority inversion, retry storms, and how to prevent one tenant from starving others.
Distributed rate limiter
Build an API that decides whether a request fits a user or tenant quota. Include an in-memory mode and a shared-store mode so you can show the trade-offs.
Core algorithms: token bucket, fixed window, and sliding window. Compare burst allowance, memory use, and boundary effects. Explain atomic updates and clock assumptions.
Build first: configure limits by key, return remaining quota, and test concurrent requests. Add local token reservations to reduce shared-store calls as a stretch.
Measure: decision latency, allowed and denied counts, approximation error under concurrency, and the effect of shared-store delay.
Design discussion: strict versus approximate quotas, hot tenants, fail-open versus fail-closed behavior, and how regional limits differ from global limits.
Durable workflow engine
Build a small engine for multi-step jobs that may wait, retry, pause for approval, or resume after a process restart. Store each state transition so the engine can recover work.
Core algorithms: durable state transitions, lease-based workers, retry scheduling, event replay, and idempotent activities. The durable execution model records history so a restarted worker can rebuild workflow state, as described in Temporal's documentation.
Build first: one workflow definition, persisted events, a timer, retry, and resume after restart. Add human approval and cancellation before considering parallel workflow branches.
Measure: recovery correctness, timer delay, workflow throughput, and storage growth per execution.
Design discussion: deterministic replay, schema evolution, side-effect safety, long histories, and task ownership after worker failure.
Metrics and log ingestion platform
Build a small telemetry service that accepts timestamped measurements and log events, stores them by time, and answers dashboard queries. Keep ingestion separate from query serving.
Core algorithms: time partitioning, downsampling, bounded-cardinality aggregation, and retention cleanup. Explain why unbounded labels can make a metrics store expensive.
Build first: ingest counters and logs, query a time range, and render a basic dashboard. Add rollups and retention before attempting distributed ingestion.
Measure: ingest events per second, query latency at different time ranges, storage per event, and dropped events under backpressure.
Design discussion: hot partitions, late events, cardinality limits, compression, and how to keep dashboards useful during an ingest spike.
Inventory reservation system
Build a reservation service for a limited stock of items. Hold stock for a fixed time, confirm a purchase, and prove concurrent requests cannot oversell inventory.
Core algorithms: atomic conditional updates, reservation expiry, idempotency, and contention control. Compare row locking with optimistic concurrency under a popular item.
Build first: reserve, confirm, cancel, and expire holds. Add a waitlist or multi-region inventory only as a stretch.
Measure: oversell count under concurrent load, reservation latency, lock waits, and stock recovery after worker restart.
Design discussion: inventory ownership, hot SKUs, checkout timeouts, duplicate confirmations, and whether stock is globally or regionally allocated.
Stream processing engine
Build a processor that consumes ordered events and calculates rolling counts or sums. Let users replay a time range and inspect late or duplicate events.
Core algorithms: event-time windows, watermarks, deduplication, checkpointing, and partition assignment. Show how late data changes a previously emitted result.
Build first: consume events, calculate a tumbling-window aggregate, persist offsets and state, and recover after a crash. Add sliding windows or replay next.
Measure: processing throughput, end-to-end lag, duplicate output after restart, and the effect of late events.
Design discussion: exactly-once claims, checkpoint consistency, rebalancing, state growth, and replay cost.
Feature flag and configuration service
Build a control plane for defining flags and a fast evaluation path for applications. Support gradual rollout and explain why a particular user received a value.
Core algorithms: deterministic cohort hashing, rule precedence, local caching, and version comparison. Keep evaluation in the application process so the request path does not depend on the control plane.
Build first: boolean flags, percentage rollout, targeting by a stable user key, audit history, and a local SDK. Add streaming updates and rollback next.
Measure: local evaluation latency, configuration propagation delay, cache staleness, and consistency across SDK instances.
Design discussion: safe defaults, stale configuration, targeting privacy, rule conflicts, and how to roll back a bad change quickly.
API gateway with overload protection
Build a gateway that routes requests to mock services, applies authentication and quotas, and protects callers when an upstream service slows down.
Core algorithms: circuit breaker state, bounded retries, load shedding, and connection pooling. Demonstrate why retries can amplify overload when callers and gateways retry together.
Build first: routing, timeout, rate limit, and health-based removal of a mock upstream. Add weighted routing and circuit-breaker half-open probes next.
Measure: p95/p99 latency, upstream concurrency, rejected requests, and recovery time after an upstream outage.
Design discussion: retry budgets, partial failure, connection limits, health-check accuracy, and whether the gateway is a bottleneck.
Multipart object storage service
Build an object API that accepts large files in chunks, resumes interrupted uploads, and verifies the final object before making it available.
Core algorithms: chunk checksums, content hashes, resumable upload manifests, and background cleanup of abandoned uploads. Compare whole-file upload with multipart retry costs.
Build first: start, upload, resume, complete, download, and delete. Use local disk as the blob store first; add replication or erasure coding only as a design exercise.
Measure: upload throughput, resume cost after a failed chunk, checksum failures caught, and metadata growth from abandoned sessions.
Design discussion: metadata consistency, partial uploads, deduplication safety, object durability, and how the API handles a blob-store failure during completion.
How to make the work interview-ready
For whichever project you choose, publish four artifacts:
- A short demo that shows a normal request and a failure case
- A design diagram that matches the running implementation
- A benchmark with workload, machine details, and raw results
- A design note on capacity limits, bottlenecks, and the next scaling step
Practice explaining one decision you changed after measurement. That story is stronger than claiming every stretch feature or an unsupported scale figure.