Interactive course · about 10 hours

Computer Networks

Here is everything you get: one wire between two machines, and the ability to hold your end at a high voltage or a low one. That is the whole vocabulary. Two levels on one piece of copper. Can you get a sentence across it, arriving whole, to the right machine, on the other side of the world?

How this works

Every step here is something you operate. You hold a wire high and low by hand to spell out a letter, and get it wrong. You push two clocks slowly out of step until the message garbles. You damage a message on purpose until the check that is supposed to catch it looks the other way, force two machines to talk over each other, then break a cable in a small network and watch the traffic find another way round. Each mechanism arrives as the fix for a failure you caused yourself.

The steps

Step 1

One wire, two voltages

Two machines, one wire. Your end can be pushed to a high voltage or left sitting low. There is no third option, no letters, no numbers. High, low, and that is it. One wire with a machine holding each end has a name that will keep coming back: it is a link. Everything in Part 1 happens on one link.

What "voltage" actually means on a wire

Voltage is electrical push. A battery pushes; a wire connected to nothing does not. When a machine "holds the wire high" it is connecting that wire to a small pushing source, usually somewhere between 1 and 5 volts. When it holds the wire low it connects the wire to zero instead.

The far end has a tiny meter on it. All it can report is roughly how hard the wire is being pushed right now. That single number is the only thing that will ever cross between these two machines.

So you write down an agreement: high means 1, low means 0. That feels like enough. It is not, and the quickest way to see the hole in it is to try sending a single byte by hand.

What a byte is, and why eight

A bit is one of those two levels, written 0 or 1. Eight bits in a row is a byte. Eight switches, each with two settings, make 2 x 2 x 2 x 2 x 2 x 2 x 2 x 2 = 256 different patterns, which is enough for every letter, digit and punctuation mark in English with room to spare.

Eight is a choice, not a law. Early machines used 6, 7 and 9. Eight won, and now every piece of equipment on the planet counts in bytes.

A row of bits is also a number. How?

You already do this in ten. In 407, the 4 is worth four hundreds, the 0 is worth no tens and the 7 is worth seven ones, because each place to the left is worth ten times the one on its right. Bits work the same way with two instead of ten. Reading a byte from the right, the places are worth 1, 2, 4, 8, 16, 32, 64, 128.

So 00000101 has a 1 in the fours place and a 1 in the ones place, which is 4 + 1 = 5. And 01101001 is 64 + 32 + 8 + 1 = 105, the byte written 0x69 further down this page. Going the other way, take the number and keep grabbing the largest place that fits: 200 is 128, leaving 72; then 64, leaving 8; then 8, leaving 0. So 200 is 11001000. That is the whole trick, and Step 9 will need it.

The receiver below looks at the wire once per tick and writes down whatever it sees. Your job is to hold the line so that the byte it writes down is the byte you were asked to send. Crank the clock by hand first. Then let it run.

Lab 1 · Send a byte with your hands
Try this firstThe byte you have been asked to send is drawn at the top, eight squares, leftmost first. Press Line is LOW (0) to flip the wire high, then press Tick (read the wire) to make the receiver look at it once and write down what it saw. Do that eight times, flipping the line whenever the next square differs from the last, and the count at the bottom should read byte matches.
Notice what the receiver does not know. It has no idea what you meant. At each tick it reads a voltage and writes a bit. If your hand was late, the bit is simply wrong, and nothing anywhere reports a problem. The byte that arrives is a perfectly valid different byte. Then press Flip the receiver's convention: the voltages on the wire do not change at all, and the message becomes something else.
A 1 is an agreement, not a fact

There is nothing about 3.3 volts that means "one". Both ends have to have agreed in advance: which level is which, which bit of the byte goes first, and hardest of all, when to look. Everything in this course is one of those agreements, written down once and then obeyed by every machine on Earth.

People keep saying "layer". A layer of what?

Getting a message across the world is too big a job to solve in one go, so it is cut into jobs that stack. The bottom job is only about voltages on one wire and nothing else. The job above it worries about where a message starts and stops, and simply assumes the bottom one is doing its part. Above that, one about who the message is for, and so on.

Each of those jobs is called a layer, and each one is allowed to be ignorant of everything above and below it. That is why a wire can work perfectly and still deliver nonsense: the bottom layer did its job exactly, and nothing above it had been agreed yet. This course climbs the stack one layer at a time, and each step is the next layer up.

Bytes keep being written like 0x69. What is that?

Writing out 01101001 every time is tiring, so bytes are usually written in hexadecimal: counting 0 to 9 and then A to F, sixteen digits instead of ten. One hex digit covers exactly four bits, so two digits describe one byte and nothing is left over. The 0x in front is just a label saying "the next digits are hex".

To read 0x69, split the byte down the middle. The 6 is the left four bits, 0110, and the 9 is the right four, 1001. Put them together and you have the byte: 01101001.

You send 8 bits. The receiver reads 8 bits. Someone watched the wire with an instrument that draws the voltage as a moving line on a screen, and the picture was exactly what you intended, high and low at all the right moments. Yet the receiver writes down 0x96 where you sent 0x69. What went wrong?
You agreed about the wire and nothing else. 0x69 is 01101001 and 0x96 is 10010110, every bit inverted. Either the receiver takes low as 1, or it is reading your bits in the opposite order. The physical layer worked perfectly and delivered nonsense, which is the most useful thing to learn from this step: a working wire is not a working link.
Step 2

Where does a bit end

Hand-cranking the clock made that easy, because you and the receiver ticked together. Real links do not have that. The sender has its own oscillator and the receiver has another one, and no two crystals in the world run at exactly the same rate.

What an oscillator is, and why every machine has one

An oscillator is the part that says "now" over and over at a steady rate. In almost every computer it is a sliver of quartz crystal that vibrates when electricity is applied, the same component that keeps a wristwatch honest. Each vibration is a tick, and everything the machine does is lined up with those ticks.

Quartz is steady but not identical from one sliver to the next. Two crystals cut to run at a million ticks a second will each be off by a little, in their own direction, and warming up moves them again. Nothing in this step is anyone's fault; it is what physical parts are like.

Suppose the receiver's clock is 1% slow. On the first bit that hardly matters, since it samples a fraction late and is still comfortably inside the bit. But the error does not reset. It adds up. Give the length of one bit on the wire a name: a bit-time. If the sender holds each bit for a millionth of a second, then one bit-time is a millionth of a second, and everything below is measured in those. Half a bit-time of accumulated error is all it takes to be sampling the wrong bit entirely.

Where the drift adds up, with real numbers

Call one bit-time one unit, and put numbers on it. A receiver that is 1% slow waits 1.01 units between looks, so it is 0.01 units late on the first bit, 0.02 on the second, 0.10 after ten. Nothing has gone wrong yet, because it is still looking somewhere inside the right bit.

After fifty bits it is half a bit-time late, which puts it exactly on the boundary. After a hundred it is a whole bit behind and reading its neighbour. A small steady error is harmless once and fatal when you add it to itself enough times.

Turn the drift knob and watch the sampling arrows slide off the bits underneath them.

Lab 2 · Two clocks that disagree
Try this firstHow to read this. The stepped line at the top is the voltage the sender put on the wire, one bit per dashed column. The arrows underneath are the moments the receiver chose to look, and the number under each arrow is what it wrote down: green where it matches the bit above it, red where it does not. At 0% the arrows sit dead centre in every bit. Drag the slider right to make the receiver's clock slow, so it waits slightly too long between looks, and left to make it fast. Drag it to about 5% and watch the arrows creep rightward until one of them lands in the wrong bit.
Now compare the three patterns at the same drift. With alternating bits the damage is obvious the moment the sampler crosses a boundary. With long runs of identical bits, drift builds up silently: every sample is right for ages, because reading the neighbour of a 1 that is also a 1 does no visible harm, and then the run ends and a whole group of bits is read twice or skipped. Silent accumulation is the dangerous kind.
Why nobody just uses a second wire

You could send the clock down its own wire, and short links inside a computer often do. Over any distance it stops working: two wires do not carry signals at exactly the same speed, so the clock and the data arrive with a shifting offset, and you have doubled the cost of every cable. What long links do instead is put the clock inside the data signal, which is the next step.

A link runs at 1,000,000 bits per second and the receiver's clock is 1% fast. Roughly how far into a continuous message does the sampling point slide by a whole bit?
About 100 bits. 1% of a bit-time per bit means a whole bit-time after a hundred of them, and you are already reading the wrong bit halfway there, at about fifty. The bit rate never enters the calculation, which is the surprise: a fast link and a slow link break after the same number of bits. That is why the receiver has to keep re-finding the edges instead of trusting its own counting.
Step 3

Hiding the clock in the signal

The receiver's own clock cannot be trusted for long, but the incoming signal can be. Every time the voltage changes, the receiver learns exactly where it is in time. Edges are free information. The trouble is that plain bits give you no edges when the data has a long run of the same value.

What people mean by an "edge"

Draw the voltage on the wire as a line going left to right in time. While the level holds steady the line is flat. When the sender switches from low to high the line jumps straight up, and that vertical jump is an edge. Low to high is a rising edge, high to low a falling one.

An edge is useful because it happens at a moment, not over a stretch. A flat line tells you nothing about the time; an edge tells the receiver "the sender did something right now", and the receiver can set its own idea of now to match.

So stop letting the data decide. Split every bit-time in half and force a transition in the middle of it, with the direction of that transition carrying the value. The scheme has a name, Manchester encoding, and here it is in full:

I have never read code before. Do I have to?

No. Every code block in this course is a second telling of something the words beside it already said, kept because it is the exact recipe and hides nothing. If you skip all of them you will miss nothing you need later.

If you want to try: a line like function manchester(bits) means "here is a recipe called manchester, and you hand it a list of bits". The line for (const b of bits) means "do the following once for every bit in that list, calling it b". b === 1 means "if b is exactly 1" and one equals sign would mean "make it 1", so the language uses three to mean "is it". out.push(0, 1) means "add 0 then 1 to the end of the answer list". Anything after // is a note to human readers that the machine ignores. So the whole recipe is: for each bit, add low-then-high for a 1 and high-then-low for a 0.

// one bit becomes two half-bits, so there is always an edge mid-bit
function manchester(bits) {
  const out = [];
  for (const b of bits) {
    if (b === 1) out.push(0, 1);   // 1 = low then high  (rising edge)
    else         out.push(1, 0);   // 0 = high then low  (falling edge)
  }
  return out;
}

Now the receiver never counts far. It finds the mid-bit edge, reads its direction, and resets its own idea of "now" to that edge. Drift has nothing to accumulate over. Push the same drift knob as before and compare the two lines.

Decoding an encoded line by hand

Say the sender wants 1, 0, 0. It holds low then high for the first bit, high then low for the second, high then low for the third: low, high, high, low, high, low. Six half-bits on the wire for three bits of data.

The receiver does not count half-bits. It waits for the next edge, looks at which way it went, and writes 1 for upward and 0 for downward. Upward, downward, downward gives back 1, 0, 0. The edge in the middle of a bit is the data; any edge at the join between two bits is only there to get the wire back into position, and the receiver ignores it.

Lab 3 · Raw line against an encoded one, under the same drift
Try this firstHow to read this. The two stepped lines carry the same eight bits, shown along the top. The upper one is the plain method from Step 1, one level held for the whole bit, which engineers call NRZ, short for non-return-to-zero, meaning the voltage just stays wherever the data put it. The lower one is the same bits Manchester encoded, two half-bits each. Under each line are the numbers the receiver wrote down, red where it is wrong. Start by dragging Receiver clock drift slowly to the right and watching which line breaks first.
Push the drift to 8%. The raw line is garbage; the encoded one is untouched, because each bit is decoded from an edge the sender just produced rather than from a count the receiver has been keeping. Then push on to 30% and watch the encoded line fail too. It is a much wider margin, not an unbreakable one.
What it costs

Two half-bits per bit means the wire has to switch twice as fast to carry the same data, so you get half the data rate out of the same hardware. Early Ethernet paid exactly that price at 10 Mbit/s. Ethernet is the set of agreements almost every wired network on Earth follows: what the voltages mean, how a message is wrapped, how machines take turns. It is not a company and it is not a cable, it is a rulebook, and most of the rest of this course is describing parts of it. Later, faster standards use cleverer codes, called 4B/5B, 8b/10b and 64b/66b, which guarantee an edge often enough for the same trick to work while giving up only 20%, 20% and 3% of the wire instead of half. Same idea, better arithmetic.

What"10 Mbit/s" is counting

Mbit/s means millions of bits per second. A link at 10 Mbit/s puts ten million bits on the wire each second, so one bit-time lasts a ten-millionth of a second. Gbit/s is a thousand times more than that again.

Watch the small b. Bits are written with a small b and bytes with a capital B, so 10 Mbit/s carries a little over one megabyte each second once the framing has taken its share.

What those code names mean, if you are curious

4B/5B takes every 4 bits of your data and sends a 5-bit pattern instead, chosen from a table where no pattern has an awkwardly long run of identical bits. Five bits carrying four costs you 20% of the wire, which beats the 50% Manchester charges. 8b/10b and 64b/66b are the same bargain at bigger sizes, and 64b/66b is why a 10 Gbit/s link actually clocks its wire at 10.3.

You will see 4B/5B called a 25% overhead elsewhere. Same fact, counted from the other end: one extra bit added to every four is a quarter more, and one wasted bit in every five sent is a fifth of the wire. Nothing disagrees, the two numbers just divide by different things. And nothing later in this course depends on any of it. It is here because "half the wire" is a steep price and it is worth knowing that engineers stopped paying it.

Someone tells you Manchester encoding is a waste because it halves the data rate for no benefit to the data itself. What is the reply?
You are buying timing, not data. A raw line carrying a hundred zeros in a row is a flat voltage: the receiver has no way to tell a hundred zeros from ninety-nine or a hundred and one, and its own clock will not stay accurate for a hundred bit-times. The encoded line spends half its capacity making sure the receiver is never more than half a bit from a fresh edge. And notice what it does not buy: nothing here detects or fixes a corrupted bit. That is Step 5.
Step 4

Where a message starts and stops

Bits now arrive reliably, one after another, forever. That is still not a message. A receiver holding a stream of bytes has no way to know which of them belong together, where one message ends and the next begins.

Frame, payload, header: what the words mean

A frame is one message as the link carries it: a bundle of bytes with a definite start and a definite end. The part you actually wanted to send is the payload. Everything the frame wraps around it, to say where it starts, who it is for, or whether it survived, is overhead, and the overhead written at the front is the header.

Think of a letter. The words are the payload, the envelope and the address are the overhead, and neither one is any use without the other.

Try it. Below is a stream that arrived on the wire, correct in every bit. Cut it into the messages that were sent.

Lab 4 · Cut the stream where the messages end
Try this firstNine bytes arrived, one square each, with the letter each byte stands for underneath. Between two squares sits a small · button: click it to cut the stream at that point, and click it again to undo. Cut wherever you think one message ended and the next began, watch "Your reading" change, then press Reveal what was actually sent. When you have seen how that goes, press tab 2 · One agreed flag byte.
You cannot win the first pane, and that is the lesson: the information you need is not in the bytes. The second pane adds one agreed byte, a flag, to mark both ends of every frame, and the receiver's job becomes trivial. Then type a ~ into the payload and watch a message split itself in half.
Why typing a tilde is what breaks it

Text is stored as numbers. A table called ASCII fixes which number means which character: capital A is 65, a space is 32, and the tilde ~ is 126, which written in hex is 7E. That is the same 7E the framer picked as its flag.

So the tilde is not a special character to the wire. It is an ordinary byte that happens to carry the one value the two ends promised would never appear in the middle of a frame.

Escaping, and why it always shows up

The moment you reserve a byte to mean "frame boundary", that byte can no longer be data, and of course your data will contain it eventually. The fix is an escape byte: replace the flag inside the payload with a two-byte sequence the receiver knows to undo. And since the escape byte is now special too, it has to escape itself. You have met this before without a computer: if a form says to separate your hobbies with commas, and one of your hobbies is "reading, mostly", the comma inside it breaks the list and the form has to offer you some way of saying "this comma is part of the words, not a separator". Programmers hit the same wall constantly, and put a backslash in front of a quotation mark inside a piece of text to mean "this quote is data, do not end the text here". Same problem, same fix everywhere. Replacing an awkward byte with a two-byte escape has a name of its own, byte stuffing, because the frame gets stuffed a little fatter on the way out. That is the button in the lab.

What is a protocol, exactly?

A protocol is an agreement written down in enough detail that two machines built by strangers, in different countries, years apart, can follow it and work together first time. Nothing more mysterious than that. 7E means the edge of a frame; 7D means the next byte has been altered; the first six bytes are the address. Write those rules down, get everyone to obey them, and you have a protocol.

Every agreement in this course is one: the level meaning of high and low in Step 1, Manchester encoding in Step 3, the framing here, the backoff rule in Step 7. The word is worth having because the alternative phrase,"the thing both ends promised to do", gets tiring, and because a protocol is a document you can go and read. Ethernet, IP, DNS and BGP are all just documents that enough people agreed to obey.

Is there a way to avoid escaping altogether?

Yes, and plenty of protocols use it. Instead of marking the end, write the length at the front: "the next 240 bytes are one frame". Now no byte value is reserved, so nothing has to be escaped and the frame never swells on the wire. Ethernet works this way.

The cost is that a length is a promise the receiver cannot check as it goes. Lose or corrupt those first bytes and it will cut the stream in the wrong place and keep cutting wrongly, whereas a flag byte lets a lost receiver find the next boundary by waiting for one to go past. Neither choice is free, and nothing later here depends on which one you prefer.

A framer uses 7E as the flag and 7D as the escape byte. What must it do when the payload contains a real 7D?
The escape has to escape itself. The receiver's rule is "when you see 7D, drop it and un-mangle the next byte". A real 7D in the data triggers that rule and eats the innocent byte that follows. So the sender sends 7D 5D for a data 7D and 7D 5E for a data 7E. The second byte is altered on purpose, by flipping one fixed bit, which is what turns 7E into 5E and 7D into 5D. If it went out unchanged, the frame would still contain a bare 7E and a receiver that had lost its place would still be fooled by it. Flipping the same bit back at the other end undoes it exactly. Payload of pure 7E bytes therefore doubles in size on the wire, a worst case real protocols accept, because the alternative is a protocol that mysteriously corrupts some files and not others.
Step 5

A bit flips

Wires run past motors, fluorescent lights and other wires. A voltage that left as a clean high can arrive close enough to the threshold that the receiver reads it as a low. There is no way to prevent this, only to notice it.

What the "threshold" is

The receiver does not measure the voltage precisely. It has one dividing line, the threshold, set somewhere in the middle of the range: above it counts as high, below it counts as low. That is what turns a messy analogue push into a clean 0 or 1.

Being that decisive is a strength most of the time. A signal that arrives a bit weak or a bit noisy still lands on the correct side of the line and is repaired for free. It is only when the noise is big enough to carry the level across the line that the receiver reads the opposite bit, with complete confidence.

The cheapest way to notice costs one bit. Count the 1s in the frame and add a parity bit that makes the total even. The receiver counts again: odd total means something changed on the way. One extra bit, and corruption stops being silent.

One parity bit, worked through

Take the byte 01101001. Count the 1s: there are four, which is already even, so the parity bit sent alongside it is 0 and the total stays even. Now take 01101011, which has five 1s. The parity bit is 1, bringing the count of 1s across the whole nine bits to six.

The receiver does not need to know which byte you meant. It counts the 1s in everything that arrived, parity bit included, and expects an even answer. One flip anywhere in those nine bits makes the answer odd, and it is spotted.

Before you trust it, break it. Click the bits yourself and find a corruption the parity bit does not notice.

Lab 5 · A noisy channel and one parity bit
Try this firstThe top row is the nine bits the sender put on the wire: eight data bits and the parity bit at the end. The lower row is what arrived, and every square in it is a button. Click one square in the lower row to flip it, the way noise would, and watch the check underneath change to parity check: odd → rejected. Then click a second square and watch it go back to accepted.
Two flips beat it, because they leave the count of 1s even again. Then set the slider to 8%, press Send 100 frames about ten times, and watch the counters at the bottom. Caught means the check fired. Missed means an even number of bits flipped, the count of 1s came out even anyway, and the frame was waved through. Those missed ones are frames a program would accept as true, with the wrong numbers in them.
Detecting is not fixing

Parity tells you a frame is broken; it cannot tell you which bit broke, so it cannot repair anything. That leaves two ways to survive: send enough extra bits to correct the error where it lands (used by disks, memory and deep-space links, where asking again is impossible or slow), or throw the frame away and ask for it again. Networks mostly do the second, which is why Step 14 hands you over to a whole course about asking again.

How a check can repair a bit instead of just noticing

Send several parity bits instead of one, each counting a different overlapping group of the data bits. A single flip then fails some of the checks and passes others, and the pattern of which ones failed points at exactly one bit. Flip that bit back and the frame is repaired without anyone asking for it again. Richard Hamming worked this out in 1950 and the memory in a server still uses it.

You pay in extra bits, and the arithmetic only pins down a small number of errors. Nothing later in this course needs it, so skip it happily; it is here because "notice" and "fix" feel like the same job until you see the price tag on each.

A frame arrives and its parity bit checks out. Precisely how much do you now know?
Only that the flip count is even. Parity halves the space of possible corruptions and says nothing about the half it lets through. Single flips are the most common kind, so that is worth having, but the honest statement of what a passing check means is "this frame is either clean or wrong in an even number of places". Noise on real wires tends to arrive in bursts that flip several neighbouring bits at once, which is exactly the case parity is worst at.
Step 6

Catching much more, cheaply

Parity fails because it throws the whole frame into a single yes-or-no answer, and every bit affects that answer the same way. What you want is a check where each bit's contribution depends on where it sits, so that two flips almost never cancel.

A CRC gets there with long division. The letters stand for cyclic redundancy check: redundancy because it sends a few extra bits that carry no message of their own, check because their whole job is to be compared at the other end, and cyclic for the shape of the arithmetic underneath. Treat the frame as one enormous binary number, the way the place-value note in Step 1 turns a row of bits into a number, divide it by a fixed number everyone has agreed on, and send the remainder. The receiver divides the whole thing, frame and remainder together, and expects to come out even. It is a few lines of code and a handful of shifts per byte:

Remainder, and the trick of making it come out even

Divide 47 by 10 and you get 4 with 3 left over. That 3 is the remainder. The useful part is that the remainder depends on the whole of 47: change any digit and it changes too.

Now subtract the remainder before you send: 47 becomes 44, which divides by 10 exactly. The receiver does not need to know what you started with. It divides what arrived, and a leftover of anything other than zero means the number it received is not the number you sent. A CRC is that idea with a much larger divisor and the arithmetic done in bits.

That code is nothing but symbols. What do they mean?

Four symbols do all the work, and each is a single instruction the chip already knows. crc << 1 means slide every bit one place to the left, so 00000110 becomes 00001100, and whatever falls off the left-hand end is gone. crc & 0x80 asks a yes-or-no question: 0x80 is the byte 10000000, and & keeps only the bits that are 1 in both, so the answer is non-zero exactly when the top bit of crc is a 1, which is the bit that is about to fall off.

^ is exclusive-or, the subject of the note below: it compares two bytes bit by bit, giving 1 where they differ and 0 where they agree, which in this arithmetic is what subtraction becomes. And a ? b : c means "if a, then b, otherwise c". So the whole line reads: if the bit about to fall off is a 1, shift left and then XOR in 0x07, and if it is a 0, just shift left. That is one step of long division, and the loop does it eight times per byte.

uint8_t crc8(const uint8_t *data, int len) {
    uint8_t crc = 0;
    for (int i = 0; i < len; i++) {
        crc ^= data[i];
        for (int b = 0; b < 8; b++)
            crc = (crc & 0x80) ? (crc << 1) ^ 0x07 : (crc << 1);
    }
    return crc;                 /* send this alongside the frame */
}
That code has no division sign in it

Because CRC arithmetic is not quite school arithmetic. It works on bits with no carrying between columns, so subtraction becomes XOR: 1 against 1 gives 0, 1 against 0 gives 1. Long division still works, and each step is a shift and an XOR, which is why the loop above holds the whole algorithm and why a chip can do it at wire speed with a few dozen gates.

Dropping the carries is also what makes the guarantees in the next callout provable. You can skip all of this and still use a CRC correctly; a table of tested divisors is what engineers actually reach for.

Below, both checks run on the same frame. Find a corruption that parity misses and the CRC catches, then find one that beats them both.

Lab 6 · Corrupt a frame, watch both checks argue
Try this firstThe long row is one received frame: twelve data bits, then the parity bit, then the eight CRC bits, and every square is a button you can flip. Press Flip two bits at random a few times and watch the two verdicts underneath, parity and CRC, disagree: parity keeps accepting frames the CRC rejects. The two goals under the counters tick off as you manage each one.
A corruption the CRC misses too is the honest button. It flips four bits in a pattern chosen so that the corrupted frame divides evenly as well, so the CRC reports a clean frame. Every check has a set of corruptions it cannot see. The engineering question is never "is it perfect", it is "how likely is noise to land exactly there", and the answer for a 32-bit CRC is about one in four billion.
What a CRC actually promises

A well-chosen n-bit CRC catches every single-bit error, every double-bit error in frames of practical size, every error that flips an odd number of bits, and every burst of corruption shorter than n bits, which matters because real noise comes in bursts. Beyond that it catches all but roughly one in 2n of everything else. Ethernet, Wi-Fi, USB, disk sectors and zip files all carry one, and they all carry it for the same reason: the guarantee is specific enough to design around.

What "one in 2 to the power n" comes to

2n means 2 multiplied by itself n times, and it grows startlingly fast: 28 is 256, 216 is 65,536, 232 is about 4.3 billion. Each extra bit in the check doubles the number of possible remainders and a corruption slips through only when it happens to land on the one remainder that was already there.

So a 32-bit CRC lets through roughly one bad frame in four billion. That is small, but a gigabit link can carry millions of frames a second, so "never" is still the wrong word for it.

"Ethernet frames carry a 32-bit CRC, so corrupt data can never reach my program." Where does that reasoning go wrong?
Two gaps, and the second is the bigger one. Yes, about one in four billion bad frames slips through, and at gigabit speeds that is not never. But the CRC also only covers one hop: each router checks it, strips it, and computes a fresh one before sending the frame on, so corruption inside a router's own memory is signed off as valid by the next CRC. This is exactly why the agreements in the layers above keep a check of their own, computed by the machine that sent the message and not looked at again until the machine that finally receives it, which is what people mean by an end-to-end check. Checks protect the stretch they cover and not one bit further.
Step 7

Everyone talking at once

Running a private wire between every pair of machines does not scale: ten machines would need forty-five wires, a hundred would need nearly five thousand. So machines share. One wire, several senders, and an obvious problem.

Where 45 wires for ten machines comes from

Every machine needs a wire to each of the others, so machine 1 needs 9 wires, machine 2 needs 9 as well, and so on: 10 x 9 = 90 wire-ends. Each wire has two ends, so that is 45 wires.

The same sum for 100 machines gives 100 x 99 / 2 = 4,950. Doubling the machines roughly quadruples the cabling, which is why nobody builds networks this way past about four machines in a lab.

If two of them push the line at the same moment, neither signal survives, because the voltages add up into something that is neither message. That is a collision, and both frames are lost. Force one below and watch it happen.

The fix has to be decided by each machine alone, with no coordinator to ask. When a sender detects a collision it waits a random number of slots before retrying, and doubles the range it picks from after each failure. That rule has a name worth knowing, exponential backoff. Backoff because a machine that has just failed backs away from the wire instead of pushing harder, and exponential because the range it waits within doubles every time: 2 slots, then 4, then 8. In the lab below it is the button of that name, and the slider labelled how busy each machine is sets the load, meaning the chance, in each slot, that a machine finds it has something new to send. Turn backoff off and watch what the shared wire is worth without it.

What a "slot" is

Waiting "a random amount of time" is no use unless everyone measures time in the same lumps, so the standard defines one lump and calls it a slot. It is set slightly longer than the time a signal needs to cross the whole wire and come back, which is the longest anyone can take to notice a collision.

On the original 10 Mbit/s Ethernet a slot was 51.2 millionths of a second, which is 512 bit-times. So a machine choosing to wait 3 slots is not waiting long by human standards; it is waiting long enough for the wire to settle.

Lab 7 · Four machines, one wire
Try this firstPress Force a collision now to see two machines push the wire in the same slot and lose both frames, then press Run and let it play. How to read it. Time runs in slots, one per step, drawn as the coloured strip: green means one machine had the wire to itself and its frame got through, red means two or more pushed at once and everything was lost, grey means nobody wanted it. Throughput is simply the share of all slots so far that came out green. It can never reach 100% because some slots are quiet, but while the load is light the wire carries very nearly everything the four machines offer it, which is four times the figure on the slider. Push each machine past about one slot in ten and it stops keeping up. A few per cent means the wire is being wasted.
Set the load high and switch backoff off. Throughput collapses: the same pairs collide, retry together on the very next slot, and collide again. With backoff on, the first retry is spread over 2 slots, the next over 4, then 8, so the chance of two machines colliding again shrinks fast while a quiet wire is still grabbed almost instantly.
Doubling the wait, with the actual numbers

After one collision each machine tosses a coin: wait 0 slots or 1. Two machines pick differently half the time, so half of these collisions are over immediately. After a second collision the choice widens to 0, 1, 2 or 3, and they clash only one time in four. After a third it is one in eight.

The range only grows for the machines that keep failing, which is exactly the sign that the wire is busy. A machine on a quiet wire that succeeds on its first try never waits at all, so the rule is patient when patience is needed and instant when it is not.

Why not have every machine wait a fixed short time after a collision, instead of a random time that doubles?
Because they collided, they are synchronised. Any rule both machines follow identically keeps them synchronised, so the retry collides too, and the next, and the next. Randomness is the only way for two machines with no way to talk to each other to stop agreeing. Doubling then handles the other half of the problem: the right amount of waiting depends on how many machines are competing, which nobody knows, so each sender measures it by failing, and backs off further each time.
Step 8

Names on the link, and a switch that learns

Several machines on one wire means every frame reaches everybody, so each frame has to say who it is for. Every socket a network cable plugs into, and the card behind it, is called a network interface, and each one carries a permanent 48-bit number burned in at the factory: its MAC address, short for media access control, the part of the rules that decides who gets to use the shared wire. Every frame starts with the destination's number and then the sender's own.

Why 48 bits, and what one looks like

48 bits is six bytes, usually written as six pairs of hex digits: 3c:5a:b4:01:9f:22. Six bytes hold about 281 trillion different numbers, which is deliberately far more than there will ever be network cards, because every one has to be different from every other one made anywhere.

Nobody coordinates that by hand. The first three bytes are bought by the manufacturer, and the manufacturer picks the last three, so two factories can never collide.

That works, but it wastes the wire: every machine receives every frame and throws away the ones addressed elsewhere. Put a box in the middle instead, with one wire to each machine, and it can send each frame down only the port where its destination lives. The catch is that nobody tells the box who is on which port.

What a port is here

A port is one of the sockets on the front of the box, with its own short wire going off to one machine. A switch with eight sockets has ports 1 to 8, and each is a separate, private link.

That is what lets the box be selective. A frame arrives on one port, and the box decides which other port or ports to copy it out of. The word gets reused later for something quite different in software, so keep this meaning tied to the socket you can touch.

So it learns, from a single observation: a frame arriving on port 3 proves its sender is on port 3. Watch the table fill.

Lab 8 · A switch with an empty table
Try this firstLeave the two menus on From A and To C and press Send a frame. Watch the frame light up three wires at once, because the table underneath is empty and the switch has to guess by sending it everywhere. Then set From C, To A, send again, and watch one row appear in the table and the next frame go down a single wire.
Send a first frame and watch it flood. With an empty table the switch has no choice but to send the frame everywhere except back where it came from. One frame later it has learned the sender, so the reply goes straight to one port. Then move a machine to another port and send it something: the table is now a lie, and it stays a lie until the moved machine says something of its own.
What happens when the wires between switches form a loop

Flooding assumes there is exactly one path between any two ports. Plug two switches together with two cables and a flooded frame goes round the loop, gets flooded again, and again, at wire speed, until the network is carrying nothing else. It is one of the few faults that can take a whole building down in seconds.

The cure is a protocol the switches run among themselves to agree which links to shut off, leaving a loop-free shape while keeping the spare cable ready. Nothing later in this course needs it, and routers in Step 10 avoid the problem in a different way.

You unplug a laptop from port 2 of a switch and plug it into port 4. For a few seconds it can send perfectly but receives nothing. Then everything is fine. Why?
The table was stale, and sending is what fixes it. Learning is one-directional: the switch learns where a machine is only by seeing traffic from it. Until then, frames for the laptop go confidently out of the empty old port and vanish. Two things rescue it: the laptop sending anything at all, and entries timing out after a few minutes so a silent machine's stale port is eventually forgotten and its frames get flooded instead of dropped.
Step 9

Addresses with structure

MAC addresses have one fatal property: they carry no information about where the machine is. They are assigned in the factory, so a table that maps them to locations has to list every machine individually. That is fine for the twenty machines on one link and hopeless for the billions on the planet.

What an IP address is, next to a MAC address

An IP address is 32 bits, written as four numbers from 0 to 255 with dots between them, like 10.0.4.9. Each of those four numbers is one byte of the address, and people call them octets. Unlike a MAC address it is not burned into the hardware: it is handed out by whoever runs the network the machine is plugged into, and it changes when the machine moves.

That is the whole point of having both. A MAC address says which card this is. An IP address says where in the world it currently sits.

The fix is to give addresses a shape. An IP address splits into a network part and a host part, and the split point is written after a slash: 10.0.4.9/24 means "the first 24 bits name the network, the last 8 name a machine inside it". Now one table entry can cover thousands of machines, because they share a prefix.

Show me the /24 as actual bits

Write 10.0.4.9 one byte at a time: 00001010 00000000 00000100 00001001. That is 32 bits. A /24 draws the line after 24 of them, which lands exactly at the end of the third byte, so the network part is 00001010 00000000 00000100 and the host part is the last 00001001.

Every machine on that link has an address starting with those same 24 bits, so one line in a table that says "anything beginning 10.0.4 goes this way" covers all of them at once. The slash number is just "how many bits from the left do we all have in common".

Those 32 bits are worth being able to produce yourself rather than taking on trust, because everything in this step is written in them. One octet at a time, that is the place-value trick from Step 1: eight places worth 128, 64, 32, 16, 8, 4, 2 and 1, and a number is whichever of them add up to it.

Lab 9a · One number, eight bits
Try this firstEach of the eight squares is a button, and the small number under it is what that place is worth. Click the square worth 4, then the square worth 1, and the running total below should read 5. Then type 200 into the box and press Show me to watch the squares set themselves, with the working written out underneath.
Every address in this step is four of these in a row. Type the four numbers of 10.0.4.9 in turn, 10 then 0 then 4 then 9, and you have built its 32 bits by hand, which is exactly what the next lab shades for you. The two challenge buttons are worth doing the slow way, largest place first, before you look at the working.

All of that gives a machine the one question it actually needs to answer before sending anything: is the destination on my own link, or do I have to hand it to a router? Drag the split and watch the answer change.

What is a router, and how can a frame and a message have different addresses?

A router is a machine with a leg in more than one network: two or more sockets, each on a different link, and a job of passing traffic between them. Step 10 is entirely about how it decides. For now it is the machine on your own link whose job is everywhere else.

That is why there are two addresses in flight at once. The thing you are actually sending, the one with the far-away address on it, is called a packet, and it travels the whole way. Each link it crosses, it rides inside a fresh frame addressed only to the next machine on that link. So the frame your machine puts on the wire is addressed to your router's MAC address, while the packet inside it still says the address on the other side of the world. The router opens the frame, throws it away, and builds a new one for the next hop.

Lab 9 · Same network, or a job for the router
Try this firstThe first two rows are the two addresses in the boxes above, written out as 32 bits each and dotted into four octets, the way you have just been building them by hand. Drag the Prefix length slider slowly from /24 down to /16 and watch the shading grow leftward and the verdict underneath switch between same network and different networks.
Watch the shaded part. Two addresses are on the same network exactly when every shaded bit matches. Slide the boundary left and more machines fall inside one network; slide it right and the same two addresses become strangers who need a router between them. Nothing about the machines changed, only the agreement about where the line sits. The third row is the mask: the prefix drawn as bits, with a 1 in every position that belongs to the network part and a 0 in every position that belongs to the machine part, so a /24 mask is twenty-four 1s and then eight 0s. Machines keep the split in exactly this form, because comparing two addresses then becomes one instruction: hide every position where the mask is 0, and see whether what is left is identical. That is all the two lines below do.
def same_network(a, b, prefix_len):
    mask = (0xFFFFFFFF << (32 - prefix_len)) & 0xFFFFFFFF
    return (a & mask) == (b & mask)
Reading those two lines

0xFFFFFFFF is 32 bits all set to 1, written in the hex from Step 1: each F is four 1s and there are eight of them. The << slides those bits left, throwing away whatever falls off the left end and feeding 0s in at the right. Sliding a /24 mask means sliding by 32 - 24 = 8, so the eight rightmost bits become 0 and the mask is twenty-four 1s followed by eight 0s, which is the third row in the lab above.

The & keeps a bit only where both sides have a 1, so a & mask blanks out every position the mask has a 0 in, leaving only the network part. Doing that to both addresses and asking whether the results are identical is the whole test. It is two machine instructions, which is why a router can afford it on every packet that arrives.

How many machines fit in a /24, or a /22

Count the bits the prefix leaves over. A /24 leaves 8 host bits, and 8 bits make 28 = 256 patterns, so a /24 covers 256 addresses. A /22 leaves 10, and 210 = 1024. Every bit you take off the prefix doubles the size of the network.

Two of those addresses are always spoken for: the all-zeros one names the network itself, and the all-ones one means "everybody here", which is called a broadcast address, the same idea as the flooding a switch does in Step 8 but asked for on purpose. So a /24 holds 254 usable machines rather than 256.

Are 10.1.4.9/22 and 10.1.7.200/22 on the same network?
Same network. Write the third octet in binary the way Lab 9a does it: 4 is 4, so only the fours place, giving 00000100, and 7 is 4 + 2 + 1, giving 00000111. A /22 boundary cuts two bits into the third octet, so only the bold bits are host bits and everything above them matches. The trap here is thinking in dots. The boundary does not care about them, and a /22 network holds 1024 addresses spanning four whole "third octets". Reading prefixes as bits rather than as dotted numbers is most of what makes subnetting, the business of cutting one network into smaller ones, stop being confusing.
Step 10

Carrying a packet hop by hop

Step 9 ended by handing a packet to a router. Here is everything a router does with it. When a packet arrives whose destination is not on any of its own links, it does one small thing: look up the destination's network in a table, and hand the packet to the neighbour listed there. Then forget about it.

Packet or frame? And what is in the header?

A frame is what one link carries between two machines that share a wire. A packet is what travels the whole way, from the first machine to the last, riding inside a fresh frame on each link it crosses. The router in this step unwraps a frame, looks at the packet inside, and wraps it in a new frame for the next wire.

The packet's own header is a short block of fields at the front, carrying the source and destination IP addresses, how long the packet is, and the hop limit. Those fields belong to the packet, so they survive every hop; the frame around them is thrown away and rebuilt each time.

No router knows the whole path. Each one only knows the next step, which is why the packet's header carries a hop limit: a small number, set by the sender, that every router it passes through knocks down by one before sending it on. When it reaches zero the next router throws the packet away. A packet going round in circles because two tables disagree has to die of something.

The quiz mentions a "default route". What is that?

A table cannot list every network on the internet, and most machines do not want to. So a table can hold one catch-all entry:"anything I do not recognise, hand to this neighbour". That is the default route, and it is what makes a home router's table three lines long instead of a million.

It works because the catch-all points at somebody better informed, whose own table is bigger and whose own default route points further up again. Only the routers in the middle of the internet have no default to fall back on, which is why the question of what happens without one matters at all.

You are the forwarding decision now. Read each router's table and choose the next hop. Choose badly on purpose at least once.

Lab 10 · Deliver it yourself, one hop at a time
Try this firstThe packet starts at R1, and the table underneath is the one inside whichever router the packet is sitting in right now, with the row that matches this packet highlighted. Read that row, then press the button under Hand the packet to: that matches it. Do that until the packet arrives, and the counter should say wrong turns 0.
Send it the wrong way twice and watch the hop limit run out. Notice what the network does about your mistake: nothing. No router looks at the whole path, so no router can tell that this packet is going in circles. The hop limit is not a routing mechanism, it is a way of guaranteeing that bad routing ends.
A packet reaches a router whose table has no entry matching the destination, and no default route. What happens?
Dropped, with an explanation posted back. The switch in Step 8 could afford to flood because a link has a handful of ports and a loop-free shape. Flooding between networks would send one packet down every path on Earth, and with millions of routers it would never stop. So routers refuse to guess: no route means the packet dies here, and a small complaint message travels back to whoever sent it saying destination unreachable. Those complaints have their own agreed format, ICMP, the internet control message protocol, which is the language routers use to report trouble rather than to carry anybody's data. The note below shows how a tool called traceroute turns those complaints into a map of the path.
How traceroute turns dying packets into a map

Send a packet with the hop limit set to 1. The first router knocks it down to 0, drops it, and posts an error back, and that error carries the router's own address. You have just learned hop one. Send the next with a limit of 2 and the second router reports itself. Keep going and the whole path names itself, one dying packet at a time.

It reads the network by breaking things on purpose, which is why the list sometimes has gaps: a router configured to stay quiet sends no error, and traceroute prints a row of asterisks for it. Nothing later depends on this.

Step 11

How routers learn the map

Someone has to fill those tables in, and it cannot be a person: links fail at three in the morning and traffic has to find another way within seconds. So the routers work it out themselves, from one rule that is short enough to fit in a sentence.

Every router tells its neighbours how far it is from every destination it knows. Every router listens, adds one for the hop to that neighbour, and keeps the shortest answer it heard along with who said it. That is all. Repeat until nothing changes.

What "cost" and "how far" are measuring

Not distance in metres. In the simplest version, cost is a count of links: a network on my own wire is cost 0, one my neighbour can reach is cost 1, and so on. Adding one for a hop is all the arithmetic there is.

Real networks let an operator set the cost of each link by hand, so a slow backup line can be given a cost of 10 and a fast one a cost of 1. The rule does not change, it just adds the link's cost instead of always adding one, and traffic drifts onto whatever the operator called cheap.

for dest in all_destinations:
    best, via = INFINITY, None
    for n in neighbours:
        cost = 1 + n.advertised[dest]      # one hop to n, then n's own distance
        if cost < best:
            best, via = cost, n
    table[dest] = (best, via)
Walk the rule with three routers

A, B and C sit in a line, and network N hangs off C. C knows N at cost 0 and says so. B hears "N, cost 0" from C, adds one, and writes down "N, cost 1, via C". A hears nothing useful yet, because B has not spoken since it learned anything.

Next round B advertises "N, cost 1". A adds one and writes "N, cost 2, via B". Round three changes nothing anywhere, so the tables have settled and every router knows which single neighbour to hand an N-bound packet to. No router ever learned the shape of the network, only its own next step.

Below is a network of five routers. Let their tables settle, send a probe to see the path they agreed on, then cut a link on that path and watch what happens next.

Lab 11 · Break a link and watch the tables argue their way back
Try this firstPress Exchange one round three or four times and watch the five cards fill in. How to read a card. Each one is a single router's entire knowledge of the world: a row per destination, the cost it believes that destination is at, and via, the one neighbour it would hand the packet to. That is all a router stores. The ∞ sign means it currently knows no way there at all, and a row marked * is one that changed in the round you just ran, so pressing the button repeatedly shows knowledge spreading outward one hop per round.
Do this in order. Run until stable, send a probe and note the path. Cut a link on that path and send the probe again before exchanging anything. It loops or dies, because the tables still describe a network that no longer exists. Now step the rounds one at a time and watch the costs creep upward, 3, 4, 5, 6, two routers each believing the other has a route. Then switch Advertise the whole path on, cut the link again, and watch the creeping stop.
Counting to infinity

When a destination disappears, the routers do not conclude "gone". They conclude "further away than I thought", one hop at a time, because each is hearing its own stale route reflected back by a neighbour. Left alone they would count upward forever, so the rule picks a number, 16, and declares it to mean unreachable, purely so the counting stops. The real fix is the one in the lab: send the path along with the distance, so a router can see its own name in a route and refuse it. That is what BGP, the agreement holding the real internet together, does between networks.

Is there a cheaper fix than sending the whole path?

There is a well-known partial one. Never tell a neighbour about a route that runs through that same neighbour, since it already knows about that route and can only be confused by hearing it back. That kills the version of the loop where exactly two routers quote each other, which is the one the lab shows.

It does not kill the version with three. A can hear it from B, B from C and C from A, and none of them is telling the route straight back to the router it came from, so the ratchet still turns, just around a bigger circle. That is why the real protocols ended up sending the whole path. Nothing later here depends on this.

What BGP is, and why it gets named so often

The internet is not one network but tens of thousands of separately owned ones, and BGP is the protocol they use to tell each other which blocks of addresses they can reach. Each announcement carries the list of networks it passed through, which is the path-advertising fix from the lab, so a network can refuse a route that comes back through itself.

Because the announcements are believed, a network that announces someone else's addresses by mistake can pull the world's traffic towards itself for an hour, and this has happened more than once. Skip this note without worry; nothing later needs it.

You cut a link and watch one router's cost to a lost network climb: 3, then 4, then 5, then 6, one per exchange. What is happening?
They are quoting each other. A said "I can reach it, cost 2, via B"; B said "cost 3, via A". When the real link dies, A hears B's 3 and concludes 4, B hears A's 4 and concludes 5, and the two of them ratchet upward on each other's stale news. Nothing is wrong with either router. The flaw is in what they say: a distance with no indication of the path it takes cannot be checked for whether it comes back through you.
Step 12

From a name to an address

Routing works on addresses, and nobody wants to remember addresses. Worse, addresses change: a service moves to another machine and every reference to it has to move too. So there is a layer of indirection, meaning names, and a system for turning a name into an address at the moment you need it.

"A layer of indirection" sounds like jargon

It means putting a stand-in where the real thing used to be, so the real thing can change without anybody having to be told. Your friend's phone number lives once in your contacts, and everything else refers to the name. When the number changes you fix it in one place. This is a different sense of "layer" from the stack of jobs in Step 1 and the two are not related; here it just means one thing standing in front of another.

Names on the internet do the same job for machines. Whoever runs a service can move it to a new machine with a new address and change one answer, and every reference in the world follows without anyone editing anything.

That system is the Domain Name System, DNS, and it is the one piece of networking jargon you will meet in ordinary life. One central list of every name on the internet would be a single point of failure and a single bottleneck, so the names are split into a hierarchy and read right to left. A domain is a name like example.com and a zone is the slice of names below it that one owner is the authority for. www.example.com asks: who knows about com, then who knows about example.com, then what is www inside it. Three questions, three round trips, each to a machine that only has to know the next step down.

What one "round trip" costs

A round trip is one question sent out and one answer coming back. What it costs is time, and the time is set by distance and by how many routers the packet crosses: about 10 thousandths of a second across a city, 150 or more between continents. Light in glass is quick but it is not instant.

So "three round trips" is not three packets' worth of data, it is three waits stacked one after another, each one unable to start until the last has finished.

Walk one yourself, and watch what the second lookup costs.

Lab 12 · Walk the hierarchy, then use the cache
Try this firstPress Ask the next server and keep pressing it until the answer appears. Each row of the ladder is one machine that knows only who to ask next, and the ms column in the log is how long that question took in milliseconds, thousandths of a second. The machine on your own network that does this walking on your behalf, and keeps the answers, is called a resolver, and it is the one asking every question in the log. The last row is the only one that answers rather than pointing onward, because whoever owns example.com owns that zone and is the sole authority on what is inside it.
Resolve the same name twice and compare the times. Then resolve a different name in the same domain: it is quicker than the first but not free, because the answers about com and example.com are cached while the last question is new. Age the cache past its lifetime and the cost comes back.
What a cache is, and what its "lifetime" does

A cache is a small store of answers you already went to the trouble of getting, kept nearby in case the same question comes up again. Your machine keeps one for name lookups, and so does the resolver it asks, so a popular name is often answered from a few streets away rather than from the far side of the world.

Every answer arrives stamped with a lifetime, chosen by whoever owns the name: keep this for 300 seconds, or for a day. While it lasts, the answer is reused; after that it is thrown away and asked again. A short lifetime lets an owner move a service quickly, and a long one saves everybody work.

The cost is delay, and it is in front of everything

A lookup moves almost no data, a question and an answer, a few hundred bytes. What it costs is waiting, three times, before your machine has even started the connection it wanted. That is why every layer caches the result, why the lifetime attached to each answer is a real engineering decision (short means fast failover, long means fewer lookups), and why an outage at a big name provider takes down services whose own machines are running perfectly.

The first lookup of shop.example.com takes three round trips. The second takes none. Then mail.example.com takes one. What explains the pattern?
The cache holds the signposts, not just the destinations. Each step of a lookup produces something worth keeping: "ask this machine about com", "ask that one about example.com", and finally "shop.example.com is at this address". A new hostname in a domain you have already visited reuses the first two and pays only for the third. That is the hierarchy earning its keep: the expensive, shared parts of the answer are the parts most likely to be reused.
Step 13

What the network never promised

You can now get a packet from one machine to another across several networks. It is worth being exact about what that gives you, because it is much less than it feels like.

Each packet is forwarded on its own. Two packets sent a microsecond apart can take different routes, meet different queues and arrive in the wrong order. One can be quietly dropped because a router's queue was full, or arrive twice because something retried. The network promises to try. Nothing else.

What a queue inside a router is, and why it fills

A router can only push one packet at a time down a given wire, and packets arrive whenever they arrive. So each outgoing port has a queue: a short line of packets waiting their turn, held in the router's memory. Traffic that comes in a burst is smoothed out by the waiting.

The memory is finite. When packets keep arriving for a port faster than the wire can drain them, the queue fills, and the router's only remaining option is to throw a packet away. Nothing is broken when that happens; it is what a full queue means. It is where most lost packets on the internet die.

How long is a microsecond?

A millisecond is a thousandth of a second, and a microsecond is a thousandth of that: a millionth of a second. A blink takes about 300,000 of them.

It matters here because it is the scale computers work at. Sending two packets a microsecond apart is almost the same instant to us, and yet on the way across the world they can end up on different routes and behind different queues, and arrive in whatever order those choices produce.

Send a sentence in ten packets across a link you control and read what comes out.

Lab 13 · Send order against arrival order
Try this firstPress Send the sentence, then read the two lines that appear underneath. The knobs decide how unkind the network is: loss is the chance a packet never arrives, duplication is the chance a copy turns up as well, and delay spread is how much the travel times differ from each other, which is what shuffles the order. Seed is a number that fixes which random misfortunes happen, so leaving it alone repeats exactly the same run and changing it gives a fresh one with the same settings.
Compare the two readings of the same arrivals. Reading bytes as they turn up produces a scrambled sentence. Reading them by the number written on each packet, and refusing to hand anything to the reader until the gap in front of it is filled, produces either the true sentence or a truncated one, never a wrong one. Those numbers cost four bytes on every packet, which the counter at the bottom adds up for the run you just sent, and they do all of that.
Numbering fixed the order but not the loss. Why?

Because a number on each packet only tells the receiver what it is holding and what it is missing. Packets 1, 2, 4 arriving is now readable as"3 has not turned up", which is much better than a scrambled sentence, and it is still not the sentence.

The missing bytes exist in exactly one place: the sender's memory. Nothing the receiver can do alone will produce them, which is why this step ends with a receiver that knows precisely what is wrong and cannot fix it. Step 14 adds the one thing that closes the gap.

A receiver gets packets 1, 2, 4, 5 and then 3. Someone says the network is broken. Are they right?
That is a healthy network. Packet 3 took a slower path or sat in a longer queue; every byte arrived. A network that guaranteed order would have to hold packet 4 and 5 until 3 turned up, inside the routers, at every hop, which means memory, waiting and one slow packet stalling everything behind it. The design choice is deliberate: keep the middle simple and dumb, and let the two ends sort out order, gaps and duplicates. Which is the topic of the next course.
Step 14

What is still missing

Look at what one voltage on one wire has turned into. An agreement about levels, a clock recovered from the signal itself, frames with edges, a check that catches corruption, a way to share the wire, addresses with a shape that lets one table entry cover a million machines, tables the routers build by gossiping, and names on top of all of it.

And a message still does not reliably arrive. Everything so far moves packets; nothing so far notices that one is missing. That last gap is the one below, and closing it is not the network's job. It is the job of the two machines at the ends.

Why the ends, and not the network?

Only the two ends can tell whether the whole message arrived, because only they know what the whole message was. A router in the middle sees one packet, forwards it, and forgets it; it has no idea which conversation the packet belongs to or how many more are coming.

There is a second reason, and it is the one the designers argued about. Every guarantee built into the middle has to be paid for by everybody, including the traffic that did not want it. A video call would rather lose a packet than wait for it. So the middle stays simple and each pair of ends buys only the promises it needs.

Lab 14 · Build reliability out of parts
Try this firstThe lab has already sent your message once with every mechanism off, so the two lines under the buttons show what you typed and the mangled thing the far end can hand to a program. Press Number the packets and watch the second line change. Then press the other three buttons, Drop duplicates, Hold out-of-order packets and Resend what is missing, one at a time, in any order. Seed fixes which random misfortunes happen, so leaving it alone means every press is judged on the same run of bad luck.
Watch what each one repairs and what it leaves broken. Numbering alone does not recover a lost packet. Resending alone delivers some packets twice. Only the four together turn "most of it arrived" into "exactly what you typed", and the counters show what they cost in extra bytes and extra waiting.
What each of the four buttons actually does

Number the packets puts a sequence number on each one, so the receiver can tell a duplicate from new data and a gap from the end of the message. Drop duplicates throws away a packet whose number has already been dealt with. Hold out-of-order packets keeps an early arrival to one side instead of handing it over, until the gap in front of it has been filled. Resend what is missing sends a packet again when it has not turned up.

Each one fixes a different failure from Step 13, which is why leaving any of them off leaves something broken. And notice what the fourth one quietly assumes: the sender only knows what to resend because the receiver told it what arrived. That reporting back is not a button here, because it is the whole subject of the next course.

Packets are already numbered and duplicates are already discarded. Which single addition turns "some of it arrived" into "all of it arrived"?
Feedback. Numbering lets the receiver notice a gap, but only the sender still holds the missing data, so nothing can be repaired until information travels back. More room at the receiver only lets it wait more comfortably for something that is never coming: the missing bytes are not late, they are gone, and they exist in exactly one place, the sender's memory. That single requirement drags in everything the next course is about: what the receiver says and how often, how long a sender waits before assuming the worst, how much it dares to send before hearing anything back, and what to do when the network's answer to "send more" is to start dropping.

Where to go next

  • Reliable Data Transfer. Acknowledgements, timeouts, sliding windows and congestion control, ending with a file transferred perfectly across a link you are actively sabotaging. It starts exactly where this lab stops.
  • Build a Microprocessor. If you have not, go and see what is at the other end of the wire: the gates and registers that produce those voltages in the first place.
  • Distributed Consistency. Once messages can be lost and delayed, two machines can hold different opinions about what happened. That turns out to be much harder than moving the bytes was.
Step 15 · Advanced path

Read an eye diagram and find a bad link

The first three steps used perfect square waves. A real cable rounds every edge and weakens the signal. A connector with the wrong impedance also reflects part of the wave back toward the sender. At a low bit rate the receiver may still separate high from low. Raise the rate, lengthen the cable, or add a bad termination and the safe sampling region shrinks.

An eye diagram overlays many bit periods. The open space in the middle shows where a receiver can sample without confusing one level for the other. Vertical opening is noise margin; horizontal opening is timing margin. Engineers do not diagnose this by staring at one attractive waveform. They transmit known patterns, measure the eye at the receiver, and compare it with the link specification.

Attenuation
The cable removes signal energy. Longer and higher-frequency links usually lose more.
Reflection
An impedance change sends energy backward. The echo can disturb a later bit.
Equalisation
A receiver can compensate for a known channel response, within limits. It cannot recover information buried in noise.
Lab 15 · Close and reopen the eye
Try this firstLengthen the cable, then change the load away from 100 Ω. Watch loss and reflection consume different parts of the margin. Turn equalisation on last.
A pass is evidence for this model and test pattern, not every cable. A real qualification also sweeps patterns, temperature, supply voltage, data rate and samples across devices.
A link passes at room temperature but fails with a longer cable. What is the best first engineering step?
Measure before changing the design. A checksum detects damaged frames; it does not repair the physical margin. The eye and error counters help separate attenuation, reflection, timing and noise.

For the physics behind channel response, continue to Filters and The Frequency Domain.

What this model leaves out

The lab compresses frequency-dependent loss, crosstalk, jitter and receiver behaviour into two margins. Use it to form a hypothesis, then use the electrical standard and measured eye mask for the actual interface.

Step 16

Keep a switched network from looping

Modern Ethernet links between a host and a switch are normally full duplex: each side can transmit at the same time, so the old shared-wire collision procedure is not running there. Switches still flood broadcasts and frames whose destination they have not learned. Two redundant links can therefore make the same frame circle forever, multiplying copies until the LAN is unusable.

A spanning-tree protocol keeps the physical backup links but blocks enough forwarding ports to leave one loop-free tree. If an active link fails, the switches can recalculate and open a backup. A VLAN adds a logical boundary: ports in different VLANs do not share one broadcast domain, even when they use the same switches.

Lab 16 · Stop a broadcast storm
Try this firstAdd the third link to make a triangle. Send one broadcast with spanning tree off, then turn it on. Finally split the hosts into VLANs.
Redundancy and forwarding are separate decisions. The cable may stay connected while the switch deliberately refuses to forward ordinary traffic through one port.
Why not remove every redundant cable?
The spare path is useful. The control protocol makes the forwarding topology loop-free while retaining physical redundancy.
What changes during reconvergence?

Frames may pause or take a different path while switches agree on a new tree. Measure convergence time and verify that the intended VLAN still has a valid path after each planned link failure.

Step 17

Find a neighbour and configure an address

An IP address tells a host which network a destination belongs to, but an Ethernet interface still needs a link-layer address for the next frame. For IPv4, ARP asks, “Who has this IPv4 address?” For IPv6, Neighbor Discovery uses ICMPv6 messages to find link-layer addresses, routers and reachable neighbours. If the final destination is off-link, the frame is addressed to a router, not directly to the distant server.

Configuration is another protocol problem. DHCP can supply addresses and other settings. IPv6 can also use router advertisements and SLAAC to form an address, followed by Duplicate Address Detection. NAT at many IPv4 edges rewrites addresses and ports because public IPv4 space is scarce. NAT is not the same thing as a firewall, and IPv6 does not require NAT for ordinary end-to-end addressing.

Lab 17 · Choose the next frame destination
Try this firstCompare an on-link and off-link destination in IPv4 and IPv6. Then switch address configuration between manual, DHCP and SLAAC.
Keep two questions separate: “What IP settings does this interface have?” and “Which neighbour receives the next frame?” Mixing them makes address problems hard to diagnose.

Protocol details: IPv6 Neighbor Discovery, stateless address autoconfiguration, and the IPv6 specification.

A server is on another subnet. Which Ethernet destination should the host use?
The router is the next hop. The IP destination stays the remote server, but each link gets a new frame addressed to the next local neighbour.
Why neighbour caches expire

Interfaces move, restart and change addresses. A cached mapping is evidence from an earlier time, so hosts refresh or probe it instead of treating it as permanent truth.

Step 18

Route by topology, then by policy

Step 11 used distance-vector gossip. Large networks also use link-state routing: routers advertise their local links, build a common topology database and run a shortest-path calculation such as Dijkstra's algorithm. OSPF is one example inside an organisation. Sequence numbers, areas and careful flooding keep the database manageable and reject stale advertisements.

The Internet between organisations is not one shortest-path contest. BGP announces reachable address prefixes plus path information, and each network applies policy. A customer route may be preferred over a shorter provider route because cost, contracts and security matter. This is why “the Internet chooses the shortest route” is a useful beginner model but an incorrect final model.

Lab 18 · Compare shortest path with policy
Try this firstUse link-state mode and fail a link. Then use BGP mode and rank customer, peer and provider routes. Inspect the evidence for the selected route.
A routing decision needs an explanation: the topology cost for an interior route, or the ordered policy attributes and path for an inter-domain route.

Primary specifications: OSPFv2 and BGP-4. The Graph Algorithms course derives the shortest-path machinery.

BGP selects a four-hop customer route over a one-hop provider route. Is that necessarily a fault?
Policy comes first. Path length is one input, not the only rule. Record the attributes and policy that produced the decision.
How to test a routing change safely

Replay the candidate configuration against recorded routes, check reachability and policy invariants, deploy to a limited set of routers, and keep a rollback. A syntactically valid route can still leak traffic to the wrong neighbour.

Step 19

Share a bottleneck without collapse

A receiver can say how much data it has room for. That is flow control. It does not say how much the routers between the endpoints can handle. Congestion control learns the path's available capacity from acknowledgements, delay and loss, then adjusts how much may be in flight.

If every sender keeps increasing after queues fill, delay grows, packets are discarded and retransmissions add still more load. A simple additive-increase, multiplicative-decrease model probes upward and backs off sharply on congestion. Production algorithms are more sophisticated, but the engineering questions remain: useful throughput, queueing delay, loss, stability and fairness across competing flows.

Lab 19 · Load a bottleneck
Try this firstSet four senders to 40 Mb/s each on the 100 Mb/s link. Compare uncontrolled sending with congestion control, then change the receiver window.
Do not report throughput alone. A design that gains one percent throughput by adding seconds of queueing may be worse for an interactive application.
The receiver advertises plenty of free memory, but a router queue is overflowing. Which limit should shrink?
This is path congestion. Receiver capacity and path capacity are different constraints; a sender obeys the smaller usable window.

Continue to Reliable Data Transfer for acknowledgements, timers, sliding windows and congestion control as one complete transport design.

Why a full queue is not free storage

A queue moves waiting time into the network. Large buffers can hide short bursts, but persistent queues add latency to every packet and still overflow. Record a latency distribution as well as average throughput.

Step 20 · 2026 practice

Protect and move a modern connection

Reliability is not authenticity. TLS 1.3 establishes keys, authenticates the server certificate and protects records from unnoticed modification. The application must still verify the intended hostname and trust chain. Encryption does not prove that a server is correct, safe, or authorised to receive every piece of data.

HTTP can run over TCP plus TLS. QUIC instead integrates a TLS 1.3 handshake with a transport carried in UDP. It supports independent streams, so loss in one stream need not stop delivery in the others, and connection IDs can let a connection survive a network-address change. Zero-round-trip data can reduce delay on a resumed connection, but replay risk means it is unsuitable for some actions.

Lab 20 · Lose a packet and change networks
Try this firstCompare TCP + TLS with QUIC. Lose a packet while three streams are active, then move the client from Wi-Fi to mobile data.
This model isolates two architectural differences. Real performance also depends on implementations, round-trip time, server settings, congestion control and whether the session can resume.

Primary specifications: TLS 1.3, QUIC, and TLS in QUIC.

A valid TLS connection returns the wrong bank balance. What has TLS established?
Security properties have boundaries. TLS protects the channel. Tests, access control, data validation and audit evidence are still required above it.
What zero-RTT changes

A resumed client can send selected data before the new handshake finishes. An attacker may replay that early data, so applications should restrict it to operations that remain safe when repeated.

Step 21

Share a channel you cannot hear all at once

A radio cannot reliably detect a collision while it is transmitting its own much stronger signal. Wi-Fi therefore listens before transmitting, waits a random backoff and expects an acknowledgement. This is collision avoidance, not collision detection. Interference, distance, obstacles and competing transmitters change which modulation and coding rate is dependable.

Two stations may each hear the access point but not hear each other. They are hidden terminals, so both can decide the channel is idle and transmit together. RTS/CTS can reserve airtime first and reduce that failure, at the cost of extra control frames. It helps in some workloads; it is not a free speed button.

Lab 21 · Expose hidden terminals
Try this firstAdd stations, hide them from one another and compare useful delivery with RTS/CTS off and on. Notice the overhead when there are few collisions.
Repeatable radio tests record channel, bandwidth, power, station positions, traffic pattern and interference. “The Wi-Fi was slow” is an observation, not yet a diagnosis.
Why does Wi-Fi use acknowledgements even though Ethernet frames have a checksum?
Detection is not delivery. A missing acknowledgement tells the sender to treat the attempt as unsuccessful, whether collision, interference or fading caused it.
Why the fastest rate is not always fastest

A denser modulation carries more bits per symbol but needs a cleaner signal. Repeated failures and retransmissions can make a lower, more error-tolerant rate deliver more useful data.

Step 22

Prove where a packet stopped

“The network is down” can describe a loose cable, a missing neighbour entry, a wrong route, DNS failure, a blocked transport port, an invalid certificate or an application that never answered. Good troubleshooting narrows the fault one boundary at a time. Start with the smallest failing path, make a prediction, collect one relevant measurement and update the prediction.

Link
Carrier state, signal margin, errors, negotiated speed and duplex.
Local delivery
Address, prefix, neighbour table, VLAN and default gateway.
Path
Route selection, hop probes, loss and latency at controlled points.
Name and transport
DNS answer, handshake, retransmissions, TLS identity and application response.

Packet captures show what crossed one observation point, not what happened everywhere. A missing response could mean the request never arrived, the response took another path, or the capture filter was wrong. In 2026, tools can summarise captures and suggest likely causes. Treat those suggestions as hypotheses: retain the raw timestamps, packets, configuration and commands that let another engineer check the claim.

Lab 22 · Choose the next measurement
Try this firstSelect a failure report. Turn on the measurements that can distinguish its competing explanations; the lab rejects evidence that is merely nearby.
Write the final record as: symptom, scope, hypothesis, measurement, result, change and verification. That structure prevents a lucky reboot from becoming a false explanation.
A packet capture on the client shows a request and no reply. What can you conclude?
State only what the evidence supports. Inspect the destination, route, server-side observation and return path before choosing a cause.

Continue the network path

What to save from an investigation

Keep clocks and time zones, interface and route state, commands with their output, capture locations, filters, configuration versions and the exact verification after the fix. Remove secrets before sharing the record.