Skip to content

avcdec: crop subtitles to their visible pixels - #1272

Open
kierank wants to merge 5 commits into
Upipe:masterfrom
kierank:avcdec-sub-crop
Open

kierank wants to merge 5 commits into
Upipe:masterfrom
kierank:avcdec-sub-crop

Conversation

@kierank

@kierank kierank commented Nov 5, 2026

Copy link
Copy Markdown
Contributor

avcdec composes every subtitle on a cleared full canvas, so downstream (upipe_blit, and any scaler in front of it) rescales and blends the whole, mostly transparent canvas for every frame the subtitle stays on screen. This crops the subpicture to the bounding box of its visible pixels and describes where the box sits in the canvas with the pic padding attributes, which upipe_blit's flow format provider already uses to reconstruct the virtual canvas, so the cropped picture keeps the scale and position of the full one.

  • avcdec: skip subtitle rects without a bitmap: a non-bitmap rect, or one without its index or palette plane, was composed from a NULL pointer.
  • upipe_blit: scale the bottom padding by the bottom padding: same commit as upipe_blit: scale the bottom padding by the bottom padding #1241, included because the crop depends on it.
  • upipe_blit: scale the paddings by the fitted size: the paddings were scaled by dest/src size, but the source is fitted into the destination by DAR, so a 4:3 canvas on a 16:9 picture got the wrong size and position. The vertical position was also rounded by the horizontal rounding.
  • avcdec: crop subtitles to the bounding box of their rects, then to the visible pixels: teletext renders the whole page into one rect, so the rects alone crop nothing there. An empty subtitle still emits a minimal transparent picture so it replaces the previous one.

Checked with upipe_blit's flow format provider on a 1920x1080 destination: a 520x60 box at 100,500 in a 720x576 4:3 canvas is placed at 440,936 with size 1040x112, where the full canvas would put it; on master any padded subpicture currently gets a wrapped vertical size. Builds without warnings; upipe_blit_test passes

The vertical scaling of the destination rectangle derived bpad from tpad, so
an asymmetric padding put the picture at the wrong size and position.  Only
subpictures whose flow definition carries pic padding attributes are
affected; for everything else both are zero.
A rect of another type, or a bitmap rect without its index or palette
plane, was composed from a NULL pointer.  Skip it with a warning.
The padding attributes of a subpicture flow were scaled by
dest_size / src_size, but the source is fitted into the destination
rectangle by DAR, so when their aspect ratios differ the suggested size
and position were wrong in the non-fitted axis.  Scale them by the fitted
size instead.  The vertical position was also rounded by the horizontal
rounding parameter.
The subpicture was composed on a cleared full canvas, but everything
downstream works on the whole picture for every frame the subtitle stays on
screen, so the transparent part of the canvas was rescaled and alpha blended
at every frame.  Crop to the bounding box of the rects instead, and describe
where the box sits in the canvas with padding attributes: flow format
providers such as upipe_blit reconstruct the virtual canvas from size +
padding, so the cropped bitmap gets the same scale factor and screen
position as the full canvas would.

An empty subtitle keeps a minimal transparent picture, since it must still
replace the previous subpicture downstream.  Offsets and sizes are kept even
for the chroma siting of a possible 4:2:0 conversion, and rects with negative
coordinates are dropped like the ones past the canvas - they used to write
before the start of the plane.
The bounding box of the rects crops nothing for teletext: the libzvbi
decoder renders the whole page into a single rect, whose transparent part is
then rescaled and blended for every frame the page stays on screen - the very
thing the box was meant to avoid.  The rects are no bound on what is visible,
so scan the pixels: an index of the palette is visible if its alpha is not
zero, and the box is the union of the boxes of the visible pixels.

The box can now be tighter than a rect, so clip the rects to it when
composing; the rows and columns kept by the rounding to even offsets and
sizes are transparent and blend as before.  Subtitle pages come out a couple
of rows tall instead of a full page, and dvb subtitles lose the transparent
part of their regions as well.

The dimensions of a teletext subpicture are no longer constant, so a
downstream scaler is reconfigured once per page.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant