Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
3 changes: 1 addition & 2 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -84,15 +84,14 @@ The app accepts plain-text shares (`ACTION_SEND`, `text/plain`). Sharing an `rts

### Quad mode

Shows all configured slots in a 2×2 grid. Each cell has its own connection and decoder. Audio is off in this mode. Tap any cell to open the menu.
Shows all configured slots in a 2×2 grid. Each cell has its own connection and decoder. Long-press a cell to play its audio (long-press it again to mute); the selected cell shows an AUDIO badge and is remembered. Tap any cell to open the menu.

## Known limitations

> [!IMPORTANT]
> These are known gaps that we plan to address. Each one has an issue — comments, test results and pull requests are welcome.
>
> - **RTSP over TCP only.** No UDP or multicast — [#7](https://github.com/OpenIPC/decoder/issues/7)
> - **No audio in Quad mode** — [#12](https://github.com/OpenIPC/decoder/issues/12)
> - **Only 4 camera slots** — [#13](https://github.com/OpenIPC/decoder/issues/13)
> - **Audio formats not supported:** MP4A-LATM, Opus, G.726, and AAC frames split across RTP packets — [#14](https://github.com/OpenIPC/decoder/issues/14)
> - **Possible dependency on Google services (GAPPS)** on some devices; under investigation, so disable GAPPS with care — [#15](https://github.com/OpenIPC/decoder/issues/15)
Expand Down
3 changes: 1 addition & 2 deletions README.ru.md
Original file line number Diff line number Diff line change
Expand Up @@ -84,15 +84,14 @@

### Режим Quad

Показывает все настроенные слоты сеткой 2×2. У каждой ячейки своё подключение и свой декодер. Звук в этом режиме выключен. Коснитесь любой ячейки, чтобы открыть меню.
Показывает все настроенные слоты сеткой 2×2. У каждой ячейки своё подключение и свой декодер. Долгое нажатие на ячейку включает её звук (повторное — выключает); у выбранной ячейки виден значок AUDIO, выбор запоминается. Коснитесь любой ячейки, чтобы открыть меню.

## Известные ограничения

> [!IMPORTANT]
> Это известные пробелы, которые мы планируем закрыть. На каждый заведён issue — комментарии, результаты проверок и pull request приветствуются.
>
> - **Только RTSP поверх TCP.** UDP и multicast не поддерживаются — [#7](https://github.com/OpenIPC/decoder/issues/7)
> - **В режиме Quad нет звука** — [#12](https://github.com/OpenIPC/decoder/issues/12)
> - **Только 4 слота для камер** — [#13](https://github.com/OpenIPC/decoder/issues/13)
> - **Не поддерживаются форматы звука** MP4A-LATM, Opus, G.726, а также AAC-кадры, разбитые на несколько RTP-пакетов — [#14](https://github.com/OpenIPC/decoder/issues/14)
> - **Возможная зависимость от сервисов Google (GAPPS)** на некоторых устройствах; изучается, поэтому отключайте GAPPS осознанно — [#15](https://github.com/OpenIPC/decoder/issues/15)
Expand Down
3 changes: 1 addition & 2 deletions README.zh-CN.md
Original file line number Diff line number Diff line change
Expand Up @@ -84,15 +84,14 @@

### Quad 四分屏模式

以 2×2 网格显示所有已配置的槽位。每个画面都有独立的连接和解码器。此模式下没有声音。点击任意画面即可打开菜单。
以 2×2 网格显示所有已配置的槽位。每个画面都有独立的连接和解码器。长按某个画面即可播放其声音(再次长按则静音);被选中的画面显示 AUDIO 标记,选择会被记住。点击任意画面即可打开菜单。

## 已知限制

> [!IMPORTANT]
> 以下是我们计划解决的已知不足,每一项都有对应的 issue。欢迎留言、提交测试结果和 pull request。
>
> - **仅支持 RTSP over TCP**,不支持 UDP 和组播 — [#7](https://github.com/OpenIPC/decoder/issues/7)
> - **Quad 模式下没有声音** — [#12](https://github.com/OpenIPC/decoder/issues/12)
> - **仅有 4 个摄像机槽位** — [#13](https://github.com/OpenIPC/decoder/issues/13)
> - **不支持的音频格式:** MP4A-LATM、Opus、G.726,以及跨多个 RTP 包分片的 AAC 帧 — [#14](https://github.com/OpenIPC/decoder/issues/14)
> - **部分设备上可能依赖 Google 服务(GAPPS)**,仍在调查中,请谨慎禁用 GAPPS — [#15](https://github.com/OpenIPC/decoder/issues/15)
Expand Down
92 changes: 70 additions & 22 deletions android/app/src/main/java/com/openipc/decoder/AudioManager.java
Original file line number Diff line number Diff line change
Expand Up @@ -88,8 +88,43 @@ void shutdown() {
}
}

// Audio routes: every source (single view, the audible Quad cell) gets a route number.
// Frames, configs and track creation of an older route are ignored, so a stopped source
// cannot leak into the next one.
private final Object routeLock = new Object();
private int route;

/** Start a new route: closes current audio and invalidates all older producers. */
int newRoute() {
int r;
synchronized (routeLock) {
r = ++route;
pcmQueue.clear();
aacQueue.clear();
}
closeAudio();
return r;
}

/** Number of decoded frames waiting for playback (for tests and diagnostics). */
int queuedPcmFrames() { return pcmQueue.size(); }

private boolean isRoute(int r) {
synchronized (routeLock) {
return r == route;
}
}

/** Audio parameters of route {@code r}; ignored if that route is no longer current. */
void configure(AudioConfig audio, int r) {
synchronized (routeLock) {
if (r != route) return;
configure(audio);
}
}

/** Called on every RTSP (re)connect, before PLAY. */
void configure(AudioConfig audio) {
private void configure(AudioConfig audio) {
config = audio;
// Parameters may differ after a reconnect: the AAC decoder is recreated on the next packet
releaseAacDecoder();
Expand All @@ -100,8 +135,9 @@ void configure(AudioConfig audio) {
}
}

/** Called from the network thread to deliver an audio RTP packet. */
void onAudioFrame(Frame frame) {
/** Called from a network thread to deliver an audio RTP packet of route {@code r}. */
void onAudioFrame(Frame frame, int r) {
if (!isRoute(r)) return;
byte[] data = frame.data();
int start = RtspSession.rtpHeaderLength(data, frame.length());
if (start < 0) return;
Expand All @@ -111,16 +147,16 @@ void onAudioFrame(Frame frame) {
AudioConfig c = config;
switch (c.codec) {
case AudioConfig.AAC:
processAac(c, data, start, end);
processAac(c, data, start, end, r);
break;
case AudioConfig.PCMU:
offerPcm(decodeG711(data, start, end, ULAW_TABLE));
offerPcm(decodeG711(data, start, end, ULAW_TABLE), r);
break;
case AudioConfig.PCMA:
offerPcm(decodeG711(data, start, end, ALAW_TABLE));
offerPcm(decodeG711(data, start, end, ALAW_TABLE), r);
break;
case AudioConfig.PCM:
offerPcm(decodeL16(data, start, end, c.bigEndian));
offerPcm(decodeL16(data, start, end, c.bigEndian), r);
break;
default:
if (!c.codec.equals(lastUnsupported)) {
Expand Down Expand Up @@ -165,10 +201,14 @@ void playPcmFrame() {

// ====================== PCM ======================

private void offerPcm(byte[] pcm) {
private void offerPcm(byte[] pcm, int r) {
if (pcm.length == 0) return;
if (!pcmQueue.offer(new Frame(pcm, pcm.length))) {
Log.w(TAG, "Audio queue full, frame dropped");
synchronized (routeLock) {
// Checked under the lock that newRoute() clears the queue with: no leftovers
if (r != route) return;
if (!pcmQueue.offer(new Frame(pcm, pcm.length))) {
Log.w(TAG, "Audio queue full, frame dropped");
}
}
}

Expand Down Expand Up @@ -242,6 +282,10 @@ private void playAudio(Frame data) {

private AudioTrack createPcmAudio(int sample, int channels) {
if (!enabled) return null;
final int trackRoute;
synchronized (routeLock) {
trackRoute = route;
}
int mask = channels == 2 ? AudioFormat.CHANNEL_OUT_STEREO : AudioFormat.CHANNEL_OUT_MONO;

Log.d(TAG, "Create PCM audio (" + sample + "hz, " + channels + "ch)");
Expand Down Expand Up @@ -283,8 +327,8 @@ private AudioTrack createPcmAudio(int sample, int channels) {

AudioTrack old;
synchronized (trackLock) {
// Paused while the track was being created: do not publish it
if (!enabled) {
// Paused or switched to another source while the track was being created
if (!enabled || !isRoute(trackRoute)) {
try { track.stop(); } catch (Exception ignored) {}
track.release();
return null;
Expand Down Expand Up @@ -313,7 +357,7 @@ private void releaseAacDecoder() {
}
}

private void processAac(AudioConfig c, byte[] data, int start, int end) {
private void processAac(AudioConfig c, byte[] data, int start, int end, int r) {
List<int[]> units;
if (c.sizeLength > 0) {
ensureAacDecoder(c, false);
Expand All @@ -328,7 +372,7 @@ private void processAac(AudioConfig c, byte[] data, int start, int end) {
units = Collections.singletonList(new int[] {start, end - start});
}
for (int[] u : units) {
offerAac(Arrays.copyOfRange(data, u[0], u[0] + u[1]));
offerAac(Arrays.copyOfRange(data, u[0], u[0] + u[1]), r);
}
}

Expand Down Expand Up @@ -388,9 +432,12 @@ static int readBits(byte[] data, int bitPos, int count) {
return value;
}

private void offerAac(byte[] au) {
if (!aacQueue.offer(au)) {
Log.w(TAG, "AAC audio queue full, frame dropped");
private void offerAac(byte[] au, int r) {
synchronized (routeLock) {
if (r != route) return;
if (!aacQueue.offer(au)) {
Log.w(TAG, "AAC audio queue full, frame dropped");
}
}
}

Expand Down Expand Up @@ -450,11 +497,12 @@ private void ensureAacDecoder(AudioConfig c, boolean adts) {
aacExecutor = Executors.newSingleThreadExecutor(
r -> new Thread(r, "rtsp-aac-decode"));
}
aacExecutor.execute(() -> aacLoop(codec));
final int codecRoute = route;
aacExecutor.execute(() -> aacLoop(codec, codecRoute));
}
}

private void aacLoop(MediaCodec codec) {
private void aacLoop(MediaCodec codec, int codecRoute) {
MediaCodec.BufferInfo info = new MediaCodec.BufferInfo();
byte[] pending = null;
while (aacRunning && !Thread.currentThread().isInterrupted()) {
Expand Down Expand Up @@ -486,7 +534,7 @@ private void aacLoop(MediaCodec codec) {
}
}
// Drain on idle iterations too, so the last decoded frame is not held back
drainAacOutput(codec, info);
drainAacOutput(codec, info, codecRoute);
} catch (Exception e) {
Log.e(TAG, "AAC decode error", e);
pending = null;
Expand All @@ -495,7 +543,7 @@ private void aacLoop(MediaCodec codec) {
}
}

private void drainAacOutput(MediaCodec codec, MediaCodec.BufferInfo info) {
private void drainAacOutput(MediaCodec codec, MediaCodec.BufferInfo info, int codecRoute) {
int outId;
while ((outId = codec.dequeueOutputBuffer(info, 0)) != MediaCodec.INFO_TRY_AGAIN_LATER) {
if (outId == MediaCodec.INFO_OUTPUT_FORMAT_CHANGED) {
Expand All @@ -510,7 +558,7 @@ private void drainAacOutput(MediaCodec codec, MediaCodec.BufferInfo info) {
byte[] pcm = new byte[info.size];
outBuf.position(info.offset);
outBuf.get(pcm);
offerPcm(pcm);
offerPcm(pcm, codecRoute);
}
codec.releaseOutputBuffer(outId, false);
}
Expand Down
Loading
Loading