Historical evidence, superseded guidance. This is the earlier article, preserved with its original workload, measurements and revision links. Its defaults, branch/deployment labels and recommendations are not current. Read the current recipe.

AVPlumber · Mixer optimization cookbook

NVDEC arrays without a staging copy

Implementation links are pinned to develop at c20464a5 (2026-10-09). The mixer has changed since; this page was checked against develop at 8fa15c92 on 2026-10-13. Measurements retain their original workload and scope. Watch the current 192-input L4 demo.

Current scope. Current L4 SDR admission is 26 outputs at 25/30 fps and 22 at 50/60 fps. Earlier 28-output probes below are not the current full-scene ceiling: the later 30-output sweep failed. Preserve three-frame decoded queues and distinguish linear and CUarray profiles.

The mixer samples NVDEC's decoded CUDA arrays directly. At the same 192 inputs and 20 outputs, whole-GPU memory fell from 20.65 to 15.77 GiB: 4.88 GiB saved, or 23.7%. The recovered room supports 28 encoded outputs, with the decoded queues still at three frames and AUX latency still at 80 ms.

Status: Opt-in, measured on L4

The idea

The linear CUDA baseline keeps decoded output images separately from NVDEC's decode surfaces. The new path registers a fixed pool of CUDA arrays with NVDEC and publishes references to those same arrays. Matching normalization forwards the handle; the compositor reads the planes through textures while rendering its output canvas. There is no intermediate decoded-image copy.

Direct array sampling removes the separate decoded output image The linear baseline maps and copies decoder output into linear frames before composition. CUarray output uses a single fixed image pool; queues and outputs hold references to its arrays. Both paths render a new output canvas. Linear CUDA baseline NVDEC decode surfaces codec reference pictures map / copy Linear decoded frames separate image storage composite Output canvas → NVENC CUarray · fixed pool, shared references Registered CUDA arrays NVDEC writes here same handle Queues / output references no second image bank sample directly Output canvas → NVENC One decoded image can serve many outputs. Each output still renders its own canvas.
Zero-copy describes the boundary from decode to composition. Rendering output pixels still takes GPU work. Optional pad/crop/transition filters produce their own final linear output; crop copies its selected rectangle directly, without a staging image.

Measured gain

The workload has 120 independent HEVC 1080p25 clips, 32 raw inputs and 40 DMA-BUF browser inputs. The 20-output runs use identical scenes, resolutions, rates and encoder settings. Twenty outputs means two program renditions, two monitors and 16 extra AUX buses; 28 adds eight AUX buses.

60-second warmed captures on the same L4
Decode pathOutputsWhole-GPU VRAMMixer CPUGPU SM / NVDEC
FFmpeg 8.1, linear CUDA2020.65 GiB6.47 cores49.0% / 68.0%
Pinned upstream, CUarray2015.77 GiB6.01 cores43.3% / 65.8%
Pinned upstream, CUarray2818.13 GiB6.42 cores44.4% / 65.6%

The matched comparison saves 4.88 GiB and about 7.2% mixer CPU. The 28-output run has 40% more outputs while using 2.52 GiB less VRAM than the old 20-output baseline. Memory includes resident browser/GPU services; CPU sums the mixer threads in logical cores.

A fresh 28-output container also passed a five-minute quiet capture: all 120 decoders and 28 encoders advanced at 25.00–25.01 fps, with zero output drops, missed deadlines or overflows. A separate two-minute AUX page-switch repeat was clean. Cut/fade/wipe stress completed 136 cuts, 57 fades and 52 wipes with zero missed program deadlines. An earlier scheduling capture had three output drops. The later capacity sweep report retains its own failed trials and explains the current 26-output SDR25 admission ceiling.

These are tested configurations, not a universal capacity ceiling. A 32-output attempt hit the existing downstream-key fanout limit before a valid run; it did not establish a VRAM or NVENC limit.

Fixed surfaces are separate from queues

extra_hw_frames adds application headroom to the codec's reference and working surfaces. It does not change the decoded edge's capacity or the AUX deadline. Queues and independently timed outputs hold references to decoded pictures; they share the image storage but can keep different pictures alive.

Pool sizing for this HEVC workload: nine base surfaces plus the configured extras
Extra framesFixed arrays per decoderObserved result
3, source default12Exhausted during 20-output startup; rejected
817Passed 20 outputs; exhausted at 28 outputs
1221Passed the 28-output validation

The pool is allocated up front and never grows or silently switches to copying. Other bitstreams can need a different base count. Validate decoder progress as well as output deadlines: a stalled input can leave downstream outputs repeating its last picture. The earlier one-frame queue proposal was not selected; this optimization keeps all three decoded queue slots.

Since this measurement

Tone mapping reads arrays too. tonemap_cuda now reads arrays in its colour, depth and chroma conversion kernel and writes the final linear output, with no intermediate copy of the input (patch 0016 of the FFmpeg series). The format conversion that scale_cuda rejects for arrays therefore has a path. Where a source's declared colour already matches the canvas and no storage is requested, the mixer's conversion graph is setparams alone, so the frame reaches the compositor as the array NVDEC wrote: a 4:2:2 HDR compositor samples 4:2:0 arrays itself. The change ships a pixel comparison of array against linear inputs; what it saves in memory or CPU was not measured on its own.

Output ceilings from full-scene sweeps. The 28 outputs above passed a quiet capture and a cut, fade and wipe stress. Later sweeps on an L4 cut through all 256 scenes of the 192-input SDR show at 25 fps: 30 outputs had passed a six-cut probe and then failed the sweep, with 693 missed deadlines, 856 AUX output drops and NVENC at 100 % (p95); at 26 outputs the first sweep had one AUX drop and the repeat none. The sweep table has no run at 28 outputs. Setup now admits 26 encoded outputs at 25 and 30 fps on an SDR canvas and 22 at 50 and 60, programs and the reserved clean feed included. A show whose 96 decodes were all HLG ran out of memory at encoder start, and the automatic return to the previous show failed as well. capacity.md · NVIDIA L4 has every run, the 10-bit modes and what was derived instead of tested.

Fewer spare arrays: smaller, but starts failed. On 14 October, with the playout releasing frames no tick can show any more, the 192-source show at 1080p30 read 18.76 GB of GPU memory with 12 spare arrays per decoder, 17.78 GB with 8 and 16.21 GB with 6. At 6 the first start attempt failed twice with No decoder surfaces left before a retry succeeded, so the show stays at 12 (GPU memory: frames no tick can show any more).

A profile and a preset switch it on for an instance. The opt-in nvidia_l4_cuarray instance profile generates SDR and HLG 4:2:0 clips as HEVC with fixed pools of 12 extra frames and keeps that across Setup changes of frame rate and canvas format; it admits 192 sources at 25 and at 30 fps on an SDR canvas. The portable L4 preset builds the pinned FFmpeg series and selects that profile. The nvidia_l4 and tesla_t4 profiles keep linear decoding and FFmpeg 8.1.

Earlier savings already in the baseline

One decoder thread. GPU video sources use decoder options.threads=1. FFmpeg frame threading otherwise reserves additional decode surfaces per active frame thread while NVDEC does the decode work. This is separate from one thread per GPU filter graph, and does not impose a one-thread policy on CPU decoding.

Low-DPB demo assets. Generated H264/HEVC demo clips use -bf 0 -dpb_size 2 to reduce reference-picture requirements at the source. Each clip remains an independent full-resolution input processed every frame. This changes the encoded media and can trade compression efficiency for a smaller decode footprint; it cannot remove reference pictures that an incoming live bitstream requires.

Both optimizations were already present in the 20.65 GiB linear baseline. Their savings are not counted again in the CUarray result.

At a glance

Use the measured configuration

A video source entry for the tested 28-output graph uses the following settings. Twelve extras is that graph's validated budget, not a recommendation for every source or show. Build the exact FFmpeg revision and headers in the patch-series instructions.

{"id": "clip", "kind": "video", "path": "<path>",
 "decode_storage": "cuarray", "extra_hw_frames": 12}

Where in the code