Skip to content

Handle near zero coefficients in tcq - #5261

Open
jingninghan wants to merge 4 commits into
AOMediaCodec:av2-encfrom
jingninghan:tcq1
Open

Handle near zero coefficients in tcq#5261
jingninghan wants to merge 4 commits into
AOMediaCodec:av2-encfrom
jingninghan:tcq1

Conversation

@jingninghan

@jingninghan jingninghan commented Aug 13, 2026

Copy link
Copy Markdown
Contributor

Add functions to handle the coefficients that are quantized to 1.
In such cases, there is no need to check upper abs levels in the
trellis framework, and hence reducing the complexity.

Anchor: b5bb354

Test condition: CTC, RA, 33 frames, speed 1

+---------+--------+--------+--------+--------+----------+
| Summary |   Y    |   U    |   V    |  YUV   | Enc-time |
+---------+--------+--------+--------+--------+----------+
| A1      |  0.03% |  0.06% |  -0.12% |  0.02% | 96.12%   |
| A2      |  0.06% |  0.31% |  -0.19% |  0.06% | 96.18%   |
| A3      |  0.11% |  -0.21% | -0.42% |  0.06% | 97.11%   |
| A4      |  0.16% |  0.76% |  -0.91% |  0.14% | 97.20%   |
| A5      | -0.12% | -0.17% | -1.18% | -0.20% | 96.32%   |
| B1      | -0.12% |  0.03% |  0.03% | -0.11% | 96.15%   |
|avg wo b2|  0.03% |  0.16% | -0.32% |  0.01% | 96.42%   |
+---------+--------+--------+--------+--------+----------+
STATS_CHANGED

@joeyoung-yt

Copy link
Copy Markdown
Contributor

Is there a description for this change (or bug#)? How about coding gain?

@jingninghan
jingninghan marked this pull request as draft August 17, 2026 18:28
@jingninghan
jingninghan force-pushed the tcq1 branch 3 times, most recently from fb7db6f to f62f020 Compare August 18, 2026 21:32
@jingninghan
jingninghan marked this pull request as ready for review August 18, 2026 21:34
@jingninghan

Copy link
Copy Markdown
Contributor Author

Is there a description for this change (or bug#)? How about coding gain?

Added both. ptal thanks

@joeyoung-yt

Copy link
Copy Markdown
Contributor

I'm curious about the BDrate change. It's minor, but I'd expect it to be closer to neutral.
Is it due to the q1 change (checking just 0 and 1)?
The speedup looks good, with just a minor change in bdrate.

@jingninghan

Copy link
Copy Markdown
Contributor Author

e q1 change (ch

Yes, i think that's mainly due to constraining it to only 0 and 1 levels.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants