Historical evidence, superseded guidance. This is the earlier article, preserved with its original workload, measurements and revision links. Its defaults, branch/deployment labels and recommendations are not current. Read the current recipe.

AVPlumber · Mixer optimization cookbook

Raw frames read straight into the memory the GPU reads

Measured on 2026-10-13 on the 16 vCPU L4 host that runs the demo. Source links are pinned to commit 8fa15c92 on develop. Watch the current 192-input L4 demo.

An uncompressed 1080p NV12 frame is 3.1 MB, and a raw source hands one to the GPU for every frame it shows. The mixer's raw file sources copied each of those frames twice on the CPU first: out of the page cache into a packet, and out of the packet into a page-locked staging buffer. A source node that reads the file straight into the page-locked buffer leaves one copy and one thread per source. On the 192-source show, 47 of them raw, the upload threads disappeared and the mixer went from 679 to 510 % of a core.

• 679 → 510 %of one core, the whole mixer, 192 sources at 1080p30
• 132 → 0 %the 47 upload threads: they no longer exist
• 1,327 → 1,186threads: three fewer for each raw source
• 322 → 123 %time threads spent runnable without a core
• 34 → 30 %each Program compositor, which the change does not touch
• 0missed deadlines, dropped frames and repeated raw frames in the first hour on the new path

Three ways to hand a frame to the GPU

canvas.raw_upload selects how NV12 and P010 file sources reach CUDA. Each step below fixed what the one before it left.

1. hwupload: FFmpeg's stock upload from ordinary memory. The frame sits in ordinary pageable memory, so the driver has to make it transferable on every frame, and traces showed upload, import and compositor threads waiting on one another inside the driver while it did. In a 68-input show at 60 fps with 17 raw sources this path missed 69 % of Program deadlines. It remains the library default; it needs no patched FFmpeg.
2. pinned: a patched FFmpeg filter with a page-locked staging buffer and its own CUDA stream. The same show, measured with the pinned uploader that preceded the filter, missed no deadline. This step removed waiting, not work: threads stopped queueing on the driver's lock, but every byte of every frame was still moved by a CPU thread, twice. On the 192-source show the 47 upload threads spent 132 % of a core doing nothing but the second copy.
3. direct: one node, raw_video_file_cuda, that owns the page-locked buffer and reads the file into it. The read is the only copy. There is no packet, no decoder that wraps the packet, no staging step, and no FFmpeg patch involved.

What a frame cost with pinned

The chain was input_rec → demux → dec_video(rawvideo) → filter_video(hwupload_cuda=pinned=1): four threads for each raw source. Per 3.1 MB frame:

• The input thread allocated a new 3.1 MB packet and called read(), which copies the frame out of the page cache: about 0.85 ms of kernel time.
• The demuxer and the raw "decoder" passed the packet on. The decoder only wraps the packet's bytes as a frame, but each is a thread, a queue and two wake-ups.
• The upload thread copied the frame row by row into its staging buffer, about 0.9 ms of user time, then asked the GPU to fetch it and slept until it had. The packet was freed on this thread, not the one that allocated it.

Two copies of 3.1 MB for each of 1,410 raw frames a second is about 9 GB a second through the CPU, and about three times that in memory traffic, since a copy reads its source and writes a destination that is not in cache. That traffic is shared with every other thread on the machine.

What the node does

1. At start it allocates one page-locked block the size of a frame through the CUDA driver, a private CUDA stream, and a pool of GPU frames.
2. For each frame it calls pread() into that block at the frame's offset in the file. The kernel's copy out of the page cache lands directly where the GPU will read.
3. It queues one transfer per plane from the block to a GPU frame on its own stream, after an event that orders the transfer behind any reader still using that pooled frame, and waits for the stream to finish. This is the sequence the pinned filter uses; only the staging copy before it is gone.
4. It stamps the timestamp and the colour tags and puts the frame on its output queue, where it blocks until the pacing node takes the previous one. The block is free again as soon as the transfer has finished.

Looping is arithmetic on a frame counter: the offset wraps, the timestamp counts on. Nothing is kept between passes. Every frame shown was read from the file and transferred for that showing, so a raw file still stands in for a live uncompressed input.

Why the upload reads 0 %

The thread meter no longer lists an upload class because no thread has that job. The upload thread did three things, and each went somewhere:

• The copy into staging, nearly all of its 132 %, is not done by anyone. The bytes are already in page-locked memory when the read returns.
• Asking the GPU to fetch the frame and waiting for it moved to the reading thread. The reading threads' user time rose from 14 to 20 % of a core for all 1,410 frames a second: about 45 µs a frame.
• The transfer itself never was CPU work. The GPU reads page-locked memory over the bus on its own; while it does, the thread sleeps.

So the upload is not free, it is small: about 6 points of a core where there were 132. What a raw source still costs is the read: about 113 % of a core of kernel time for 47 sources, the one copy that remains.

Before and after

The running 192-source show at 1080p30 (105 HEVC through NVDEC, 40 browser, 47 raw NV12; 24 encoded outputs), one 20-second reading of every mixer thread on the build before and one on the build with the node, about three minutes after it started.

• The threads that vanished account for about 148 of the 169 points: the uploads, and the 47 raw sources' decode and demux threads.
• The rest came from threads the change does not touch. The compositors, the pacing loops and the AUX buses all read lower: with 4.4 GB a second less being copied and 141 fewer threads waking every frame, they wait less for memory and for a core.
• Time spent runnable without a core fell by 62 %, far more than the CPU saved. That is the figure that says how close a 30 fps show is to missing a deadline.
• GPU memory was unchanged (19,337 and 19,291 MiB): the same number of frames is held on the GPU, and one page-locked frame per source in host memory before and after.
• An hour later the show had missed no deadline and dropped no frame, and neither Program compositor had repeated a frame of any raw source since the start.

Isolated: the upload paths alone

Eight raw NV12 1080p sources at 30 fps on a second, otherwise idle L4 host of the same shape, each paced, scaled on the GPU and discarded, 30 seconds per path (tests/cuda/benchmark_raw_upload.py). The figures are the whole process, so they include the pacing, the scaler and the sink of each source, which are the same on every row.

• pinned costs the same CPU as hwupload. With eight sources on an idle machine nothing waits on the driver, so what pinned fixed does not show; what it left, the two copies, is the whole bill on both rows.
• direct halves it: the user time, which was the second copy, is gone, and the kernel time, which is the read, is unchanged.
• A write-combined block makes no difference (16.4 and 16.2 % for the eight reading threads). The node offers it as write_combined; it stays off.
• A frame costs less here than on the busy demo host (about 0.7 ms a copy against 0.9), because nothing else is using the memory.

What it gives up

• Read-ahead in memory. The old chain kept about ten frames of packets queued in RAM ahead of pacing. The node holds two frames, both already on the GPU: one in its output queue and the one it is waiting to put there. A read that stalls for longer than a frame period now shows as a repeated frame where 300 ms of queued packets used to hide it. The demo's raw files stay in the page cache and the raw sources repeated no frame in the first hour; a show reading raw video from a slow disk should raise the output queue's capacity, not a deadline.
• A read timeout. input_rec watches its reads; a blocking local pread() has no watchdog. A file that ends inside a frame after it was opened stops the node with an error.
• Formats. NV12 and P010 only. Packed v210 still goes through hwupload_cuda=v210_width=…, because the unpack kernel lives in the FFmpeg patch, and decoded RGBA clips still use the pinned filter.

For a live uncompressed input

The upload half is its own class, CudaPinnedUploader: the page-locked block, the stream, the ordering event, the transfers and the frame pool. A source that can write its samples where it is told uses it the same way: a capture SDK given the block as its buffer, or a packet reassembler that assembles rows into it. For such a source the remaining copy does not exist either; the file read is what stands in for it here.

Since this measurement

A fourth mode, "resident", keeps a looped file's pictures in one page-locked block the size of the file: each picture is read the first time it is shown, and later passes only transfer it to the GPU. On the same show on 13 October the 152 reading threads fell from 139.7 to 27.3 % of a core and the mixer from 471 to 335 %, at the price of 8.8 GB of host memory that cannot be swapped out. It is on an avplumber branch, not yet on develop; the demo show runs it, while Setup still builds shows with direct. Every picture is still transferred for every showing. Input, decode and wake-ups has the measurement and why it still stands in for a live input.

At a glance

• WhatOne node per raw NV12 or P010 file source reads each frame straight into page-locked memory and has the GPU fetch it from there. It replaces four nodes and the staging copy between them.
• WhyThe pinned filter removed the waits on the driver's lock but still moved every frame twice on the CPU. Forty-seven upload threads spent 1.3 cores on the second copy.
• Latency0 frames added. A frame is uploaded one ahead of pacing, as before.
• Side effectsNo packets queued in RAM ahead of pacing, so a slow read is no longer absorbed. No read timeout.
• CostOne page-locked frame per source (3.1 MB for 1080p NV12), which the staging buffer already cost. No FFmpeg patch.
• RiskLow for files in the page cache. Tests cover byte-exact NV12 and P010, loop timestamps, end of file, and a frame rewritten in the file while it loops, which must show up on a later pass.
• Switch it off"canvas": {"raw_upload": "pinned"} restores the filter chain; "hwupload" the stock one. "resident", on a branch, goes one step further (above).

Related pages

• Browser frames go back one per new frame: the driver lock that uploads wait on, read directly.
• Colour tags at the fan-out: the node stamps the declared tags, so a raw source needs no colour filter.
• The compositor's wait costs the frames that arrived: the busiest single thread, and how the per-thread readings are taken.
• Real 1080p inputs, no shortcuts and Source limits by mode and frame rate: what the raw sources stand in for and how many a show carries.

Where in the code

• src/nodes/hwaccel/raw_video_file_cuda.cpp: the node: the read, the timestamps, end of file.
• src/nodes/hwaccel/CudaPinnedUploader.hpp: the page-locked block and the transfer, for any source that can fill it.
• src/nodes/hwaccel/raw_frame_index.hpp: where frame n of a file is and when it is shown.
• pyplumber/mixer/inputs.py: build_raw420_input, the three chains.
• deps/ffmpeg/9/0017-avfilter-hwupload-cuda-pinned.patch: the pinned filter, still used for v210 and RGBA.
• tests/cuda/smoke_raw_direct_input.py and tests/cuda/benchmark_raw_upload.py: what the node must keep doing, and the three paths side by side.
• doc/NODES.md: the node's parameters.

Source links are pinned to the measured commit.