Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
4 changes: 2 additions & 2 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -34,7 +34,7 @@ A beta channel is available on [Google Play](https://play.google.com/apps/intern
## Features

- **Video:** RTSP playback, hardware H.264/H.265 decoding via MediaCodec
- **Audio:** AAC (RFC 3640 `mpeg4-generic`, ADTS), G.711 μ-law/A-law, L16 PCM (mono/stereo)
- **Audio:** AAC (RFC 3640 `mpeg4-generic`, ADTS, MP4A-LATM), Opus, G.711 μ-law/A-law, L16 PCM (mono/stereo)
- **Display:** single camera, or Quad mode with 4 cameras at once
- **Controls:** pinch-to-zoom, pan, double-tap to reset, Android TV remote (Leanback launcher)
- **Network:** Basic and Digest authentication, RTSP keepalive, automatic reconnection with exponential back-off, connection-quality indicator
Expand Down Expand Up @@ -93,7 +93,7 @@ Shows all configured slots in a 2×2 grid. Each cell has its own connection and
>
> - **RTSP over TCP only.** No UDP or multicast — [#7](https://github.com/OpenIPC/decoder/issues/7)
> - **Only 4 camera slots** — [#13](https://github.com/OpenIPC/decoder/issues/13)
> - **Audio formats not supported:** MP4A-LATM, Opus, G.726, and AAC frames split across RTP packets — [#14](https://github.com/OpenIPC/decoder/issues/14)
> - **G.726 audio is not supported** — [#14](https://github.com/OpenIPC/decoder/issues/14)
> - **Possible dependency on Google services (GAPPS)** on some devices; under investigation, so disable GAPPS with care — [#15](https://github.com/OpenIPC/decoder/issues/15)
> - **Credentials are stored and sent in clear text** (plain RTSP/HTTP; Basic auth is sent unless the camera asks for Digest) — [#16](https://github.com/OpenIPC/decoder/issues/16)
>
Expand Down
4 changes: 2 additions & 2 deletions README.ru.md
Original file line number Diff line number Diff line change
Expand Up @@ -34,7 +34,7 @@
## Возможности

- **Видео:** воспроизведение RTSP, аппаратное декодирование H.264/H.265 через MediaCodec
- **Звук:** AAC (RFC 3640 `mpeg4-generic`, ADTS), G.711 μ-law/A-law, L16 PCM (моно и стерео)
- **Звук:** AAC (RFC 3640 `mpeg4-generic`, ADTS, MP4A-LATM), Opus, G.711 μ-law/A-law, L16 PCM (моно и стерео)
- **Режимы:** одна камера или Quad — 4 камеры на одном экране
- **Управление:** масштаб двумя пальцами, перемещение, сброс двойным касанием, пульт Android TV (Leanback)
- **Сеть:** авторизация Basic и Digest, RTSP keepalive, автоматическое переподключение с растущей задержкой, индикатор качества связи
Expand Down Expand Up @@ -93,7 +93,7 @@
>
> - **Только RTSP поверх TCP.** UDP и multicast не поддерживаются — [#7](https://github.com/OpenIPC/decoder/issues/7)
> - **Только 4 слота для камер** — [#13](https://github.com/OpenIPC/decoder/issues/13)
> - **Не поддерживаются форматы звука** MP4A-LATM, Opus, G.726, а также AAC-кадры, разбитые на несколько RTP-пакетов — [#14](https://github.com/OpenIPC/decoder/issues/14)
> - **Не поддерживается звук G.726** — [#14](https://github.com/OpenIPC/decoder/issues/14)
> - **Возможная зависимость от сервисов Google (GAPPS)** на некоторых устройствах; изучается, поэтому отключайте GAPPS осознанно — [#15](https://github.com/OpenIPC/decoder/issues/15)
> - **Логин и пароль хранятся и передаются в открытом виде** (RTSP/HTTP без шифрования; Basic отправляется, если камера не запросила Digest) — [#16](https://github.com/OpenIPC/decoder/issues/16)
>
Expand Down
4 changes: 2 additions & 2 deletions README.zh-CN.md
Original file line number Diff line number Diff line change
Expand Up @@ -34,7 +34,7 @@
## 功能

- **视频:** RTSP 播放,通过 MediaCodec 进行 H.264/H.265 硬件解码
- **音频:** AAC(RFC 3640 `mpeg4-generic`、ADTS)、G.711 μ-law/A-law、L16 PCM(单声道/立体声)
- **音频:** AAC(RFC 3640 `mpeg4-generic`、ADTS、MP4A-LATM)、Opus、G.711 μ-law/A-law、L16 PCM(单声道/立体声)
- **显示:** 单路画面,或 Quad 四分屏(同时显示 4 路摄像机)
- **操作:** 双指缩放、拖动平移、双击复位,支持 Android TV 遥控器(Leanback 启动器)
- **网络:** 支持 Basic 和 Digest 认证、RTSP 保活,断线后自动重连(指数退避),连接质量指示
Expand Down Expand Up @@ -93,7 +93,7 @@
>
> - **仅支持 RTSP over TCP**,不支持 UDP 和组播 — [#7](https://github.com/OpenIPC/decoder/issues/7)
> - **仅有 4 个摄像机槽位** — [#13](https://github.com/OpenIPC/decoder/issues/13)
> - **不支持的音频格式:** MP4A-LATM、Opus、G.726,以及跨多个 RTP 包分片的 AAC 帧 — [#14](https://github.com/OpenIPC/decoder/issues/14)
> - **不支持 G.726 音频** — [#14](https://github.com/OpenIPC/decoder/issues/14)
> - **部分设备上可能依赖 Google 服务(GAPPS)**,仍在调查中,请谨慎禁用 GAPPS — [#15](https://github.com/OpenIPC/decoder/issues/15)
> - **用户名和密码以明文存储和传输**(未加密的 RTSP/HTTP;摄像机未要求 Digest 时发送 Basic) — [#16](https://github.com/OpenIPC/decoder/issues/16)
>
Expand Down
101 changes: 100 additions & 1 deletion android/app/src/main/java/com/openipc/decoder/AudioConfig.java
Original file line number Diff line number Diff line change
Expand Up @@ -13,6 +13,8 @@ class AudioConfig {
static final String PCMU = "PCMU"; // G.711 mu-law
static final String PCMA = "PCMA"; // G.711 A-law
static final String AAC = "AAC";
static final String LATM = "LATM"; // MP4A-LATM (RFC 6416): AAC in LATM framing
static final String OPUS = "OPUS";

String codec = NONE;
int pt = -1;
Expand All @@ -26,6 +28,8 @@ class AudioConfig {
int sizeLength;
int indexLength;
int indexDeltaLength;
// MP4A-LATM: "cpresent=1" puts the configuration in-band, which is not supported
boolean latmConfigInBand;

/** Fill defaults for static RTP payload types (RFC 3551), which may come without rtpmap. */
void applyStaticPayloadType() {
Expand All @@ -46,6 +50,10 @@ void applyRtpmap(String encoding) {
bigEndian = false;
if (name.equals("MPEG4-GENERIC") || name.equals("AAC") || name.equals("MP4A")) {
codec = AAC;
} else if (name.equals("MP4A-LATM")) {
codec = LATM;
} else if (name.equals("OPUS")) {
codec = OPUS;
} else if (name.equals("PCMU")) {
codec = PCMU;
} else if (name.equals("PCMA")) {
Expand All @@ -54,7 +62,7 @@ void applyRtpmap(String encoding) {
codec = PCM;
bigEndian = true;
} else {
// MP4A-LATM, OPUS, G726, ... are not supported
// G.726 and others are not supported
codec = NONE;
Comment on lines 64 to 66

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Action required

1. G.726 cameras still have no audio 📎 Requirement gap ≡ Correctness

AudioConfig.applyRtpmap() continues to classify G.726 as unsupported and sets its codec to NONE.
Any SDP advertising G.726 therefore reaches the existing unsupported-audio path instead of a
decoder, leaving that camera without playback.
Agent Prompt
## Issue description
G.726 remains explicitly unsupported, so cameras that advertise G.726 audio are still dropped instead of being played. Implement a software G.726 decoder and connect it to the audio pipeline, or otherwise provide a working decoder path for supported G.726 variants.

## Fix Focus Areas
- android/app/src/main/java/com/openipc/decoder/AudioConfig.java[64-66]
- android/app/src/main/java/com/openipc/decoder/AudioManager.java[149-173]

## Recommended Fix
Add G.726 codec recognition and SDP parameter handling, decode G.726 RTP payloads into PCM, and route the decoded samples through the existing audio output path. Add JVM coverage for codec selection and decoder input handling, and device-test playback with a G.726 camera.

ⓘ Copy this prompt and use it to remediate the issue with your preferred AI generation tools

}

Expand Down Expand Up @@ -86,6 +94,7 @@ void applyFmtp(String params) {
case "sizelength": sizeLength = Integer.parseInt(value); break;
case "indexlength": indexLength = Integer.parseInt(value); break;
case "indexdeltalength": indexDeltaLength = Integer.parseInt(value); break;
case "cpresent": latmConfigInBand = !"0".equals(value); break;
default: break;
}
} catch (NumberFormatException ignored) {}
Expand All @@ -102,6 +111,96 @@ void finish() {
|| indexDeltaLength < 0 || indexDeltaLength > 16)) {
codec = NONE;
}
if (OPUS.equals(codec)) {
// RFC 7587: always 48 kHz and "/2" in rtpmap; the decoder handles mono streams too
sampleRate = 48000;
channels = 2;
}
if (LATM.equals(codec)) {
// fmtp config= is a StreamMuxConfig here; turn it into an AudioSpecificConfig
byte[] asc = latmConfigInBand || aacConfig == null ? null : latmToAsc(aacConfig);
if (asc == null) {
codec = NONE;
} else {
int[] info = ascInfo(asc);
if (info == null) {
codec = NONE;
} else {
aacConfig = asc;
sampleRate = info[0];
channels = info[1];
}
}
}
}

private static final int[] AAC_RATES = {
96000, 88200, 64000, 48000, 44100, 32000, 24000, 22050,
16000, 12000, 11025, 8000, 7350
};

/**
* StreamMuxConfig to a 2-byte AudioSpecificConfig, or null unless it is the one layout
* the RTP path handles: audioMuxVersion 0, all streams same time framing, one subframe,
* one program, one layer, AAC-LC, a standard sampling rate, mono or stereo,
* frameLengthType 0 (byte-aligned PayloadLengthInfo), no other data and no CRC.
*/
static byte[] latmToAsc(byte[] smc) {
BitReader r = new BitReader(smc);
if (r.bits(1) != 0) return null; // audioMuxVersion 1
if (r.bits(1) != 1) return null; // allStreamsSameTimeFraming
if (r.bits(6) != 0) return null; // numSubFrames: exactly one
if (r.bits(4) != 0) return null; // numProgram: exactly one
if (r.bits(3) != 0) return null; // numLayer: exactly one
// AudioSpecificConfig
int objectType = r.bits(5);
int freqIndex = r.bits(4);
int channelConfig = r.bits(4);
Comment thread
qodo-free-for-open-source-projects[bot] marked this conversation as resolved.
if (objectType != 2) return null; // AAC-LC only (decoder profile)
if (freqIndex >= AAC_RATES.length) return null; // reserved / explicit rate
if (channelConfig < 1 || channelConfig > 2) return null; // playback is mono/stereo
// GASpecificConfig
// The 2-byte ASC built below carries neither setting, so streams using them are refused
if (r.bits(1) != 0) return null; // frameLengthFlag: 960 samples
if (r.bits(1) != 0) return null; // dependsOnCoreCoder
if (r.bits(1) != 0) return null; // extensionFlag
// Back in StreamMuxConfig
if (r.bits(3) != 0) return null; // frameLengthType 0: PayloadLengthInfo in bytes
r.bits(8); // latmBufferFullness
if (r.bits(1) != 0) return null; // otherDataPresent
if (r.bits(1) != 0) return null; // crcCheckPresent
if (r.overrun()) return null;
return new byte[] {
(byte) ((objectType << 3) | (freqIndex >> 1)),
(byte) (((freqIndex & 1) << 7) | (channelConfig << 3))
};
}

/** {sample rate, channels} of a 2-byte AudioSpecificConfig, or null. */
static int[] ascInfo(byte[] asc) {
if (asc == null || asc.length < 2) return null;
int freqIndex = ((asc[0] & 0x07) << 1) | ((asc[1] & 0x80) >> 7);
int channelConfig = (asc[1] & 0x78) >> 3;
if (freqIndex >= AAC_RATES.length || channelConfig < 1 || channelConfig > 2) return null;
return new int[] {AAC_RATES[freqIndex], channelConfig};
}

private static final class BitReader {
private final byte[] data;
private int pos;

BitReader(byte[] data) { this.data = data; }

int bits(int n) {
int v = 0;
for (int i = 0; i < n; i++, pos++) {
int bit = pos >> 3 < data.length ? (data[pos >> 3] >> (7 - (pos & 7))) & 1 : 0;
v = (v << 1) | bit;
}
return v;
}

boolean overrun() { return pos > data.length * 8; }
}

private static byte[] parseHex(String hex) {
Expand Down
Loading
Loading