Skip to content

Adjust speed features for RTC speed >= 6. - #5300

Merged
urvangjoshi merged 1 commit into
AOMediaCodec:av2-encfrom
marpan-max:tx_size_rtc
Aug 21, 2026
Merged

Adjust speed features for RTC speed >= 6.#5300
urvangjoshi merged 1 commit into
AOMediaCodec:av2-encfrom
marpan-max:tx_size_rtc

Conversation

@marpan-max

@marpan-max marpan-max commented Aug 20, 2026

Copy link
Copy Markdown
Contributor

Set tx_size_search_level = USE_FAST_RD, and add level 3 to restrict_tx_partition_type_search to only test
TX_PARTITION_NONE and TX_PARTITION_SPLIT.

Also switch cdef_pick_method to CDEF_FAST_SEARCH_LVL3, until the CDEF_PICK_FROM_Q model is further evaluated/improved.

Only affects realtime mode speed >= 6.

This brings back some quality with small speed loss: on average over rtc set of vga and 720p clips, ~10% bdrate gain with ~4% encoder speed slowdown. This is with the command line settings below with the qp range {70 80 90 100 110 120 130 150 170}:

--passes=1 --bit-depth=8 --obu --lag-in-frames=0 --kf-min-dist=1000 --kf-max-dist=1000 --auto-alt-ref=0 --enable-tpl-model=0 --enable-keyframe-filtering=0 --enable-deblocking=1 --enable-restoration=0 --enable-intra-edge-filter=0 --enable-flip-idtx=1 --enable-masked-comp=0 --enable-onesided-comp=0 --enable-interintra-comp=0 --enable-smooth-interintra=0 --enable-diff-wtd-comp=0 --enable-interinter-wedge=0 --enable-interintra-wedge=0 --enable-global-motion=0 --enable-warped-motion=0 --enable-smooth-intra=0 --enable-paeth-intra=0 --enable-cfl-intra=0 --force-video-mode=1 --enable-overlay=0 --enable-angle-delta=0 --enable-trellis-quant=0 --enable-qm=0 --use-intra-dct-only=1 --coeff-cost-upd-freq=2 --mode-cost-upd-freq=2 --mv-cost-upd-freq=3 --frame-parallel=0 --aq-mode=0 --deltaq-mode=0 --frame-boost=0 --noise-sensitivity=0 --cdf-update-mode=1 --reduced-reference-set=1 --enable-ref-frame-mvs=0 --erp-pruning-level=0 --use-ml-erp-pruning=0 --enable-ext-partitions=0 --enable-mrls=0 --enable-pc-wiener=0 --enable-wiener-nonsep=0 --enable-tip=0 --enable-bawp=0 --enable-cwp=0 --enable-imp-msk-bld=0 --enable-ist=0 --enable-inter-ist=0 --enable-inter-ddt=0 --enable-cctx=0 --enable-ibp=0 --max-drl-refmvs=0 --max-drl-refbvs=0 --enable-refmvbank=0 --enable-opfl-refine=0 --enable-ccso=0 --enable-lf-sub-pu=0 --enable-adaptive-mvd=0 --enable-flex-mvres=0 --enable-joint-mvd=0 --enable-refinemv=0 --enable-mvd-sign-derive=0 --enable-parity-hiding=0 --enable-warp-delta=0 --enable-warp-extend=0 --tile-columns=0 --gf-min-pyr-height=0 --gf-max-pyr-height=0 --end-usage=q --use-fixed-qp-offsets=0 --psnr --threads=1 --enable-sdp=0 --min-gf-interval=8 --max-gf-interval=8 --explicit-ref-frame-map=0 --enable-bru=0 --enable-ext-seg=0 --enable-six-param-warp-delta=0 --enable-cdef=1 --enable-cdef-on-skip-txfm=0 --dpb-size=8 --use-inter-dct-only=0 --enable-fsc=1 --enable-idtx-intra=1 --enable-palette=1 --enable-intrabc-ext=0 --enable-intrabc=0 --tune-content=0 --rt --cpu-used=6 --enable-gdf=0 --max-reference-frames=3 --max-q=qp ---min-q=qp

@urvangjoshi

urvangjoshi commented Aug 20, 2026

Copy link
Copy Markdown
Contributor

Also switch cdef_pick_method to CDEF_FAST_SEARCH_LVL3, until the CDEF_PICK_FROM_Q model is further evaluated/improved.

@marpan-max
Optional suggestion (as this seems like a temporary change):

Note that @chengchen-google found CDEF_FAST_SEARCH_LVL3 method to not work well: 7c9dfba

I also independently found that CDEF_FAST_SEARCH_LVL2 doesn't work well. I believe CDEF_FAST_SEARCH_LVL1 is the one used for speed >= 2 currently -- so maybe better to use that.

@marpan-max

Copy link
Copy Markdown
Contributor Author

Also switch cdef_pick_method to CDEF_FAST_SEARCH_LVL3, until the CDEF_PICK_FROM_Q model is further evaluated/improved.

@marpan-max Optional suggestion (as this seems like a temporary change):

Note that @chengchen-google found CDEF_FAST_SEARCH_LVL3 method to not work well: 7c9dfba

I also independently found that CDEF_FAST_SEARCH_LVL2 doesn't work well. I believe CDEF_FAST_SEARCH_LVL1 is the one used for speed >= 2 currently -- so maybe better to use that.

Thanks, for now prefer to keep it at L3 for rtc.

Set tx_size_search_level = USE_FAST_RD, add level 3
to restrict_tx_partition_type_search to only test
TX_PARTITION_NONE and TX_PARTITION_SPLIT.

Switch cdef_pick_method to CDEF_FAST_SEARCH_LVL3,
until the QP model is improved.

This brings back some quality with small speed loss:
~10% bdate gain with ~4% speed slowdown, on rtc set
(vga and 720p clips).
@urvangjoshi
urvangjoshi merged commit 7361d54 into AOMediaCodec:av2-enc Aug 21, 2026
8 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants