A recent patent by Nintendo European Research & Development (NERD) suggests that a machine learning neural network may be used to reduce compression artifacts from GameShare/GameChat streams. Whether it’s been implemented in the Switch System, however, is unclear.
- Host device compresses gameplay video using H.264; QP is based on connection quality; poor link = higher compression
- Higher compression introduces more artifacts, such as macroblocking, chromatic aberrations, ringing effects, etc.
- Guest device uses an ML model to reduce artifacts and (optionally) upscale video
- ML model is lightweight: <10ms/frame, 14 convolution layers across 3 levels, <200k parameters
- ML model processes Y channels only (cleanup and optional upscaling); optional UV channels upscaling uses bilinear interpolation based on the processed Y (img below)
- Separate model versions may be trained to handle 1.33x, 1.5x, 2x, and 3x scaling, selected based on the output resolution
In a non-streaming use case, another ML model can be used on the game device to removing bands (stepped gradients) from pre-rendered cutscene videos. Images 1-2 below illustrate the processing steps and pseudocode; images 3-4 show a sample video frame before and after debanding (you may need to enlarge the images for a closer look).
Per usual, my summary is simplified. Click the link for more details.
[Originally posted on 2026-10-05 Pacific Time]
