Skip to content

Android: extracted waveform does not match the audio — 8-bit, 16-bit and float PCM samples are decoded incorrectly #512

Description

@victorrss

Describe the bug

On Android, WaveformExtractor mis-reads the decoder's PCM output in all three of its bit-depth paths, so the extracted waveform does not represent the audio. This is a data-correctness bug, not a crash, which is probably why it has gone unnoticed: the waveform still renders, it just renders the wrong shape.

The most visible case is 16-bit, the default and by far the most common path (handle16bit):

val first = buf.get().toInt()
val second = buf.get().toInt() shl 8
val value = (first or second) / Constants.SIXTEEN_BITS

buf.get() returns a signed Byte. When the low byte is above 0x7F, .toInt() widens it to a negative Int (all upper bits set), and those sign bits then swallow the high byte through the or. The low byte needs and 0xFF; only the high byte carries the sample's sign.

Because roughly half of all samples have a low byte ≥ 0x80, about half of them collapse toward zero regardless of their real magnitude. The result is a waveform that is systematically attenuated and noisy rather than obviously broken.

The other two paths:

  • handle8bitAudioFormat.ENCODING_PCM_8BIT is unsigned, with 128 as silence, but the byte is read as a signed Byte and never centered. Silence decodes as full-scale negative.
  • handle32bitpcmEncodingBit is only ever set to 32 for AudioFormat.ENCODING_PCM_FLOAT (see onOutputFormatChanged), so those samples are IEEE-754 floats already in [-1.0, 1.0]. They are instead assembled byte-by-byte as an integer and divided by 2^31.

To Reproduce

  1. Extract a waveform on Android, e.g. PlayerController.preparePlayer(..., shouldExtractWaveform: true) or AudioFileWaveforms.
  2. Compare the resulting amplitudes against the same file decoded by any other tool (or against the same file on iOS).
  3. The Android waveform is attenuated and noisier, and it does not track the audio's loudness.

For the 8-bit and float paths, an easier check is to feed a file that decodes to ENCODING_PCM_8BIT or ENCODING_PCM_FLOAT and observe that silent regions are not flat.

Expected behavior

Decoded sample values should match the audio, so that the extracted waveform has the same shape as the signal, on every bit depth and on both platforms.

Additional context

Decoding the same input under the current arithmetic versus the corrected arithmetic, normalized to [-1.0, 1.0]:

path input current expected
8-bit raw 128 (silence) -1.0000 0.0000
8-bit raw 255 (full scale) -0.0078 +0.9922
16-bit +32767 (full scale) -0.0000 +1.0000
16-bit +128 -0.0039 +0.0039
16-bit +16384 +0.5000 +0.5000
float +1.0 -0.0039 +1.0000
float -1.0 -0.0039 -1.0000
float -0.25 -0.0039 -0.2500

Note the 16-bit +16384 row: it decodes correctly because its low byte is 0x00. That is what makes the bug intermittent per sample instead of total.

I have a fix ready for all three paths and will open a PR referencing this issue.

Smartphone (please complete the following information):

  • Device: reproducible on any Android device; the arithmetic is device-independent
  • OS: Android
  • Version: master

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions