Short answer: an encoder throws away most of the video and reconstructs what's missing. Raw 1080p30 video runs about 1.5 Gbps — roughly 300 times more than you'd stream. Compression closes that gap by exploiting two redundancies: within a frame, neighbouring pixels are similar, and between frames, most of the picture doesn't change. The encoder stores full pictures occasionally and, in between, stores only descriptions of what moved and what changed. Everything else — bitrate control, keyframe intervals, latency — follows from that basic trade.
This explains the mechanism. For what to actually configure, see the setup guide.
The size problem
Uncompressed 1080p30 is about 1.5 Gbps. Nobody streams that. Getting to 5 Mbps means discarding roughly 99.7% of the data while keeping something that still looks like the original.
That's possible because raw video is enormously redundant. Two things make it so:
Spatial redundancy. Within one frame, a pixel usually resembles its neighbours. A wall, a sky, a jersey — large regions of near-identical values.
Temporal redundancy. Between consecutive frames, most of the picture is unchanged. In a 30fps talking-head shot, 95% of frame two is identical to frame one. Storing it again is waste.
Encoders attack both. Spatial compression works like JPEG; temporal compression is what makes video codecs distinct.
Frame types and the GOP
Encoded video isn't a sequence of pictures. It's a sequence of three different things:
| Type | What it holds | Relative size | Depends on |
|---|---|---|---|
| I-frame (keyframe) | A complete picture | Largest — 10 to 100× a P-frame | Nothing |
| P-frame | Changes from the previous frame | Small | Earlier frames |
| B-frame | Changes from frames before and after | Smallest | Frames both directions |
An I-frame is a full image, independently decodable. P and B frames hold only differences, which is why they're tiny.
A GOP — group of pictures — is one I-frame plus the P and B frames that follow it until the next I-frame. A two-second keyframe interval at 30fps means a 60-frame GOP: one complete picture, 59 descriptions of change.
This explains the two-second rule in live streaming. A player can only start decoding at an I-frame, and HLS segments must begin at one. Longer GOPs compress better but make viewers wait longer to start and coarsen seeking. Two seconds is the compromise the industry settled on.
B-frames also explain a subtlety: since they reference frames that come after them, the encoder must hold frames before emitting them. That's structural latency, and it's why very low-latency configurations often disable B-frames and accept a larger file for the reduced delay.
Motion estimation, the expensive part
To encode a P-frame, the encoder splits the picture into blocks and, for each, searches the previous frame for the closest match. Finding it means the block can be stored as a motion vector — "this block moved eleven pixels left" — plus a small residual for whatever doesn't match exactly.
The search is where nearly all the computation goes, and it's what encoder presets control. A slower preset searches more candidate positions and finds better matches, producing a smaller file at the same quality. A faster preset gives up sooner and spends more bits. Nothing else about the output changes — the quality-versus-speed trade is almost entirely how hard the encoder looks for motion.
This is also why some content compresses far better than others. A locked-off camera on a news desk is nearly all motion vectors. Confetti, rain, water, fast pans, film grain — every block genuinely changes, motion estimation finds nothing reusable, and the encoder has to spend real bits. Sport at high frame rates is demanding for exactly this reason.
Transform, quantise, discard
Within each block, after motion compensation, the remaining difference goes through a frequency transform — a DCT or similar — which separates coarse structure from fine detail.
Then comes quantisation, the only genuinely lossy step. Fine detail coefficients get divided down, and many round to zero. Human vision is far more sensitive to broad structure than to fine high-frequency detail, so this discards what you're least likely to notice.
Quantisation strength is the quality dial. Heavy quantisation zeroes many coefficients and produces the familiar blocky artefacts at low bitrates. Light quantisation keeps detail and costs bits.
What survives is entropy-coded — lossless packing of the remaining numbers — and written out.
How bitrate control works
The encoder has a target and must hit it, so it adjusts quantisation continuously.
CBR holds bitrate constant and lets quality float. When a hard scene arrives, quantisation increases and the picture softens. Live streaming uses CBR because the network needs predictable throughput.
VBR holds quality roughly constant and lets bitrate float, spending more on hard scenes. Better for files, wrong for live, because a spike can exceed what the connection carries.
This is the mechanism behind the advice to encode below your upload capacity. CBR at 9 Mbps doesn't degrade gracefully on a 6 Mbps connection — it keeps producing 9 Mbps and the network drops what it can't carry.
Hardware versus software encoding
Same algorithms, different silicon.
Software runs on general-purpose CPU cores. Flexible, supports every option, and slow — motion estimation at 1080p60 is a serious workload.
Hardware uses a fixed-function block: NVENC on NVIDIA, Quick Sync on Intel, AMF on AMD. Circuitry built for this one job, so it's dramatically faster and uses almost no CPU. The trade is flexibility — fewer options, and at very low bitrates software still produces better results because it can search more exhaustively.
For live, hardware almost always wins. Real-time is a hard deadline, and an encoder that misses it drops frames. A slightly larger file that arrives on time beats a smaller one that doesn't.
Muxing and transport
The encoded video and audio are separate elementary streams. Muxing interleaves them into a container with timestamps that keep them synchronised.
Which container depends on where it's going. RTMP uses an FLV-derived format. SRT typically carries MPEG-TS. HLS and DASH use fragmented MP4 — the CMAF container — segmented at I-frame boundaries, which is why keyframe interval and segment length are linked.
The segmenting step is where a lot of latency lives. A player generally needs a complete segment before playing it, so six-second segments mean at least six seconds of delay before anything starts. Low-latency HLS addresses this with partial segments, publishing pieces of a segment before it's finished.
Where the delay comes from
Total glass-to-glass latency is a sum, and the encoder is only one term:
| Stage | Typical contribution |
|---|---|
| Capture and buffering | 1–2 frames |
| Encoding (B-frame lookahead) | 100–500 ms |
| Muxing and segmenting | 2–6 sec (standard), 200 ms–1 sec (low-latency) |
| Network and CDN | 0.5–2 sec |
| Player buffer | 2–10 sec |
The player buffer and the segmenting step dominate, which is why "reduce encoder latency" rarely helps much on its own. Cutting total delay means shorter segments and a smaller player buffer — and both reduce the cushion absorbing network trouble. That's the trade behind every low-latency configuration.
Quick answers
Why is the first frame bigger? It's an I-frame, a complete picture. The rest store only changes.
Why does sport need more bitrate? Fast motion and crowd detail defeat motion estimation, so more of each frame must be encoded outright.
What does a slower preset actually do? Searches harder for motion matches. Better compression, more computation, same output format.
Why keyframes every two seconds? Players can only start at an I-frame, and HLS segments must begin at one.
Why is hardware encoding worse at low bitrates? Fixed-function circuits can't search as exhaustively as software given unlimited time.
Why can't I get sub-second latency on HLS? Segmenting and player buffering, not encoding. Sub-second needs WebRTC.
Sources: standard video-compression references on H.264/AVC frame types, motion compensation, transform coding and rate control; published streaming-architecture documentation on containers and segmented delivery. Figures are typical values, not specification limits.