From 8979dc2baa44cb546a23c66aa79255392a2b492d Mon Sep 17 00:00:00 2001 From: nibarAhmed Date: Mon, 9 Nov 2026 00:17:17 +0100 Subject: [PATCH] Say the volume as soon as it changes TalkBack waited until the volume keys had been still for 2 seconds, and didn't say the media volume while media played. Backtalk now says each new level right away, says a held key's level when it's released, keeps changes that aren't the user's quiet, such as the level headphones bring when they connect, and nudges the accessibility volume on Bluetooth so the announcement plays at the new level. "Backtalk off" is said at the accessibility volume on Samsung phones and on each output. Fixes #295 Co-Authored-By: Claude Opus 5.5 --- differences.md | 12 +- .../talkback/FinalAnnouncementVolume.kt | 72 ++- .../talkback/TalkBackService.java | 17 + .../talkback/gesture/GestureController.java | 6 + .../monitor/QuietBeforeAnnouncement.kt | 338 +++++++++++++ .../talkback/monitor/VolumeAnnouncer.kt | 274 +++++++++++ .../talkback/monitor/VolumeMonitor.java | 99 ++-- .../talkback/selector/SelectorController.java | 5 + .../talkback/FinalAnnouncementVolumeTest.kt | 31 ++ .../monitor/QuietBeforeAnnouncementTest.kt | 141 ++++++ .../talkback/monitor/VolumeMonitorTest.kt | 451 ++++++++++++++++++ .../compat/media/AudioManagerCompatUtils.java | 4 + .../utils/output/LowLatencyAudio.kt | 41 +- .../utils/output/SpeechController.java | 6 +- 14 files changed, 1443 insertions(+), 54 deletions(-) create mode 100644 talkback/src/main/java/com/google/android/accessibility/talkback/monitor/QuietBeforeAnnouncement.kt create mode 100644 talkback/src/main/java/com/google/android/accessibility/talkback/monitor/VolumeAnnouncer.kt create mode 100644 talkback/src/test/java/com/google/android/accessibility/talkback/monitor/QuietBeforeAnnouncementTest.kt create mode 100644 talkback/src/test/java/com/google/android/accessibility/talkback/monitor/VolumeMonitorTest.kt diff --git a/differences.md b/differences.md index 8e2f3c6ae..7375449c5 100644 --- a/differences.md +++ b/differences.md @@ -190,7 +190,17 @@ TalkBack ignored the order for a row whose checkbox or switch has no text of its ### "Backtalk off" at the right volume -When you turn Backtalk off, it says "Backtalk off" at the accessibility volume, using your device's real volume levels. TalkBack only approximated that volume. +When you turn Backtalk off, it says "Backtalk off" at the accessibility volume, using your device's real volume levels. This includes Samsung phones, and stays right when you connect or disconnect headphones while Backtalk is on. On Bluetooth headphones that control their own volume, it can be one volume step louder or quieter, about 3 dB, because those headphones can set the media volume between steps and Android doesn't tell apps where. TalkBack only approximated that volume. + +### Volume said as it changes + +When you press a volume key, Backtalk says the new volume right away and stops what it was saying. During continuous reading it waits for the current item to finish instead, so reading goes on. A newer level replaces one that hasn't finished. The first change names the volume, such as "Music volume set to 50 percent", and changes to the same volume less than 2 seconds apart say only the percentage. If you hold the key down, Backtalk says the first level, stays quiet while the volume keeps changing, and says the level you end at when you let go. TalkBack waited until the keys had been still for 2 seconds, and after the first press it waited 4 seconds. Backtalk also says the media volume while media plays, where TalkBack said it only when nothing was playing. + +Each output, such as the phone speaker or a pair of headphones, keeps its own media volume, and Android switches to it when media moves there. Backtalk doesn't say a media change that comes with a new output unless a volume key made it, so connecting or disconnecting headphones stays quiet. Headphones can report the same level again a moment later, and Backtalk doesn't say a media level that hasn't changed unless a volume key reported it. TalkBack said the level headphones set. If the new output's level is the one you already had, nothing changes, so the first press on the headphones' own controls after that isn't said. + +On Bluetooth headphones that control their own volume, Android raises speech to make up for the media volume, but it works out how much only when speech starts or when the accessibility volume changes. An announcement that cut into speech still playing kept the level for the old media volume, so it sounded louder or quieter than speech after it. When the media volume changes, Backtalk sets the accessibility volume one step down and back, which makes Android work out the level at once, and says the new level right away. When none of Backtalk's audio has played for 2 seconds, it first plays 0.15 seconds of silence, because Android applies the level up to 0.09 seconds after audio starts. With the screen off, Android 16 doesn't let Backtalk change the accessibility volume, so Backtalk instead stops speech, waits until none of its audio has played for 0.35 seconds, up to 0.6 seconds, and plays the silence first. During continuous reading it doesn't stop, so the level can be off then. Speech already on its way to the headphones when you turn the volume up, about 0.2 seconds of it, still plays at the old level, so it can sound loud for a moment. + +When you change the volume with a slider, Backtalk says only the slider's value. To stop hearing the volume, turn off **Verbosity** > **Speak volume changes**. ### Table column headers diff --git a/talkback/src/main/java/com/google/android/accessibility/talkback/FinalAnnouncementVolume.kt b/talkback/src/main/java/com/google/android/accessibility/talkback/FinalAnnouncementVolume.kt index 299f35ef1..61816d968 100644 --- a/talkback/src/main/java/com/google/android/accessibility/talkback/FinalAnnouncementVolume.kt +++ b/talkback/src/main/java/com/google/android/accessibility/talkback/FinalAnnouncementVolume.kt @@ -25,12 +25,13 @@ import kotlin.math.pow /** * Works out how loud to speak "Backtalk off". * - * Android shuts the accessibility audio stream down before Backtalk speaks that last announcement, - * so it plays at the media volume. To keep it at the accessibility volume, the speech is made - * quieter by the difference between the two. Android 9 and later report each volume step's real - * loudness in decibels for the current output, which is exact. Before that, the difference is - * guessed from where the two volume sliders are, which lowers the speech too little at the - * quietest accessibility volumes, where Android's volume steps are furthest apart. + * Android turns the accessibility volume up to the media volume's step before Backtalk speaks that + * last announcement. To keep it at the accessibility volume, the speech is made quieter by the + * difference between the two steps. Android 9 and later report each step's real loudness for the + * current output, which is exact. The accessibility stream's own loudness is used for both, + * because the media stream's can differ from it. Before Android 9, the difference is guessed from + * where the two volume sliders are, which lowers the speech too little at the quietest + * accessibility volumes, where Android's volume steps are furthest apart. */ object FinalAnnouncementVolume { /** @@ -59,7 +60,7 @@ object FinalAnnouncementVolume { return 0f } val gain = - decibelGain(audioManager, accessibilityIndex, musicIndex) + reportedGain(audioManager, accessibilityIndex, accessibilityMaxIndex, musicIndex, musicMaxIndex) ?: sliderGain( accessibilityIndex.toFloat() / accessibilityMaxIndex, musicIndex.toFloat() / musicMaxIndex, @@ -67,6 +68,43 @@ object FinalAnnouncementVolume { return speechVolume * gain } + /** + * The accessibility step that Android sets for media step [musicIndex], the way it matches the + * steps of two streams: in tenths of a step, from each stream's lowest step to its highest, then + * rounded. Bluetooth headphones can set the media volume between steps, which apps can't see, + * so there this can be a step off. + */ + @JvmStatic + fun accessibilityStepAtMediaVolume( + musicIndex: Int, + musicMinIndex: Int, + musicMaxIndex: Int, + accessibilityMinIndex: Int, + accessibilityMaxIndex: Int, + ): Int { + val musicRange = (musicMaxIndex - musicMinIndex) * 10 + val accessibilityRange = (accessibilityMaxIndex - accessibilityMinIndex) * 10 + val tenths = + accessibilityMinIndex * 10 + + ((musicIndex - musicMinIndex) * 10 * accessibilityRange + musicRange / 2) / musicRange + return (tenths + 5) / 10 + } + + /** + * The gain that turns a sound at the media step into one at the accessibility step, from 0 to 1, + * from the loudness Android reports for each. That is usually in decibels, which are at most + * 0 at full volume, but Samsung phones report a fraction of full loudness instead, from 0 to 1. + * Their fractions are above 0 at every step but the lowest, which is never asked for, so two + * values above 0 are fractions. + */ + @JvmStatic + fun gainFromReportedVolumes(accessibility: Float, music: Float): Float { + if (accessibility > 0f && music > 0f) { + return (accessibility / music).coerceIn(0f, 1f) + } + return gainFromDecibels(accessibility, music) + } + /** The gain that turns a sound at [musicDb] into one at [accessibilityDb], from 0 to 1. */ @JvmStatic fun gainFromDecibels(accessibilityDb: Float, musicDb: Float): Float { @@ -88,23 +126,37 @@ object FinalAnnouncementVolume { return 10f.pow((accessibilityFraction - musicFraction) / 0.4f) } - private fun decibelGain( + /** + * The gain from the accessibility stream's loudness at [accessibilityIndex] and at the step + * Android sets it to for media step [musicIndex]. + */ + private fun reportedGain( audioManager: AudioManager, accessibilityIndex: Int, + accessibilityMaxIndex: Int, musicIndex: Int, + musicMaxIndex: Int, ): Float? { if (Build.VERSION.SDK_INT < Build.VERSION_CODES.P) { return null } + val mediaStep = + accessibilityStepAtMediaVolume( + musicIndex, + audioManager.getStreamMinVolume(AudioManager.STREAM_MUSIC), + musicMaxIndex, + audioManager.getStreamMinVolume(AudioManager.STREAM_ACCESSIBILITY), + accessibilityMaxIndex, + ) val deviceType = outputDeviceType(audioManager) return try { - gainFromDecibels( + gainFromReportedVolumes( audioManager.getStreamVolumeDb( AudioManager.STREAM_ACCESSIBILITY, accessibilityIndex, deviceType, ), - audioManager.getStreamVolumeDb(AudioManager.STREAM_MUSIC, musicIndex, deviceType), + audioManager.getStreamVolumeDb(AudioManager.STREAM_ACCESSIBILITY, mediaStep, deviceType), ) } catch (e: IllegalArgumentException) { null diff --git a/talkback/src/main/java/com/google/android/accessibility/talkback/TalkBackService.java b/talkback/src/main/java/com/google/android/accessibility/talkback/TalkBackService.java index cbe9292e1..bba484153 100644 --- a/talkback/src/main/java/com/google/android/accessibility/talkback/TalkBackService.java +++ b/talkback/src/main/java/com/google/android/accessibility/talkback/TalkBackService.java @@ -1427,6 +1427,12 @@ protected boolean onKeyEventInternal(KeyEvent keyEvent) { if (adjustSliderWithVolumeKey(keyEvent, triggerModifierPressed)) { return true; } + // Backtalk: a held volume key is said once, when it's released. This follows the keys that + // Backtalk routes to its own volume too, so the release is seen even if routing changes. + if (volumeMonitor != null + && (keyCode == KeyEvent.KEYCODE_VOLUME_DOWN || keyCode == KeyEvent.KEYCODE_VOLUME_UP)) { + volumeMonitor.onVolumeKeyEvent(keyEvent); + } if (keyAction == KeyEvent.ACTION_DOWN) { boolean handleVolumeKeyInTalkBack = isTouchInteracting @@ -2576,6 +2582,15 @@ public long getSpeechCacheWindowCounts() { focuser.setMenuManager(menuManager); volumeMonitor = new VolumeMonitor(pipeline.getFeedbackReturner(), this, callStateMonitor); + // Backtalk: a focused slider that the user just moved says its own value, and continuous + // reading must not be cut off. A change from elsewhere, such as headphones, is still said. + volumeMonitor.setSliderFocused( + () -> + RecentUserInput.isRecent() + && Role.getRole(accessibilityFocusMonitor.getSupportedAdjustableNode()) + == Role.ROLE_SEEK_CONTROL); + volumeMonitor.setContinuousReading( + () -> pipeline.getActorState().getContinuousRead().isActive()); selectorController = new SelectorController( this, @@ -3193,6 +3208,7 @@ private void resumeInfrastructure() { if (volumeMonitor != null) { ContextCompat.registerReceiver( this, volumeMonitor, volumeMonitor.getFilter(), RECEIVER_EXPORTED); + volumeMonitor.startWatchingAudio(); if (FeatureSupport.hasAccessibilityAudioStream(this)) { // Cache the initial volume in case that the volume is never changed during runtime. volumeMonitor.cacheAccessibilityStreamVolume(); @@ -3345,6 +3361,7 @@ private void suspendInfrastructure() { if (volumeMonitor != null) { volumeMonitor.releaseControl(); volumeMonitor.clearVolumeChangedListener(); + volumeMonitor.stopWatchingAudio(); } if (headphoneStateMonitor != null) { diff --git a/talkback/src/main/java/com/google/android/accessibility/talkback/gesture/GestureController.java b/talkback/src/main/java/com/google/android/accessibility/talkback/gesture/GestureController.java index 0a0a92556..1d2cf16e1 100644 --- a/talkback/src/main/java/com/google/android/accessibility/talkback/gesture/GestureController.java +++ b/talkback/src/main/java/com/google/android/accessibility/talkback/gesture/GestureController.java @@ -1043,6 +1043,12 @@ private void speak(String text) { private final VolumeChangedListener a11yVolumeChangedListener = new VolumeChangedListener() { + // Backtalk: this listener says the new accessibility volume, so VolumeMonitor doesn't. + @Override + public boolean announcesVolume(int type) { + return type == AudioManager.STREAM_ACCESSIBILITY; + } + @Override public void onVolumeChanged(int type, int value, int prevValue, int percent) { if (type != AudioManager.STREAM_ACCESSIBILITY) { diff --git a/talkback/src/main/java/com/google/android/accessibility/talkback/monitor/QuietBeforeAnnouncement.kt b/talkback/src/main/java/com/google/android/accessibility/talkback/monitor/QuietBeforeAnnouncement.kt new file mode 100644 index 000000000..dd37abb96 --- /dev/null +++ b/talkback/src/main/java/com/google/android/accessibility/talkback/monitor/QuietBeforeAnnouncement.kt @@ -0,0 +1,338 @@ +/* + * Copyright 2026 Backtalk contributors + * + * Licensed under the Apache License, Version 2.0 (the "License"); you may not + * use this file except in compliance with the License. You may obtain a copy of + * the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, WITHOUT + * WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the + * License for the specific language governing permissions and limitations under + * the License. + */ + +package com.google.android.accessibility.talkback.monitor + +import android.annotation.SuppressLint +import android.content.Context +import android.media.AudioAttributes +import android.media.AudioDeviceInfo +import android.media.AudioManager +import android.media.AudioPlaybackConfiguration +import android.os.Build +import android.os.Handler +import android.os.Looper +import android.os.SystemClock +import android.util.Log +import com.google.android.accessibility.utils.output.LowLatencyAudio + +/** + * Makes a new media volume on Bluetooth headphones sound as loud as the speech after it. + * + * Headphones that control their own volume turn everything down by the media volume, and Android + * raises accessibility audio to make up for it. It works out that boost again at once when the + * accessibility volume changes, and a little after accessibility audio starts, but not when more + * audio joins audio that already plays. So Backtalk sets the accessibility volume one step away and + * back. While its audio plays, that is enough and it speaks at once. When nothing plays, it first + * plays [PRIME_MS] of silence, so the boost is in place before speech starts. If the accessibility + * volume can't be changed, speech is stopped and the announcement waits until nothing of Backtalk's + * has played for [DRAIN_MS], up to [MAX_WAIT_MS], and then plays the silence. + */ +class QuietBeforeAnnouncement +@JvmOverloads +constructor( + private val context: Context, + // Tests set this, because Robolectric has no Bluetooth audio. + private val onBluetooth: () -> Boolean = { accessibilityOnBluetooth(context) }, +) { + private val audioManager = context.getSystemService(AudioManager::class.java) + private val handler = Handler(Looper.getMainLooper()) + private var watching = false + private var playing = false + // When accessibility audio last stopped, in uptime milliseconds. + private var stoppedAt = 0L + private var pending: Runnable? = null + // When the silence that plays before an announcement ends, in uptime milliseconds. + private var silenceEnds = 0L + // The accessibility volumes Backtalk's own nudge is about to report, in order. + private val ownChanges = ArrayDeque() + // The media volume the last nudge was made for. Android reports each media change once for media + // and once for each stream that shares its volume, and one nudge covers all of them. + private var nudgedForMedia = -1 + // When that nudge was made, in uptime milliseconds. The same level later on is nudged again, + // since the boost can have changed in between without an announcement, such as during a call. + private var nudgedAt = 0L + + private val playbackCallback = + object : AudioManager.AudioPlaybackCallback() { + override fun onPlaybackConfigChanged(configs: MutableList) { + update(accessibilityAudioPlays(configs)) + } + } + + private val drained = Runnable { finish() } + private val timeout = Runnable { finish() } + private val primed = Runnable { speakNow() } + + /** Starts following accessibility audio, so that audio which just ended is waited out too. */ + fun start() { + if (watching) return + watching = true + audioManager.registerAudioPlaybackCallback(playbackCallback, handler) + playing = accessibilityAudioPlays(audioManager.activePlaybackConfigurations) + } + + /** Stops following accessibility audio and drops an announcement still waiting. */ + fun stop() { + cancel() + nudgedForMedia = -1 + // The receiver is gone, so the changes still on their way won't be seen. + ownChanges.clear() + if (!watching) return + watching = false + audioManager.unregisterAudioPlaybackCallback(playbackCallback) + } + + /** + * Runs [speak] for a change of [streamType], once Backtalk's audio is quiet if it has to be. + * [stopAudio] stops what Backtalk is saying, unless [keepSpeech] is set: then [speak] runs at + * once. Call [cancel] first to drop an announcement still waiting. + */ + fun announce(streamType: Int, keepSpeech: Boolean, stopAudio: Runnable, speak: Runnable) { + if (!needsQuiet(streamType)) { + speak.run() + return + } + if (nudgeAccessibilityVolume(streamType)) { + // Audio that plays takes the new boost at once, and audio that starts takes it only a little + // later, so silence goes first. Speech that joins the silence waits for it too. + if (playing && SystemClock.uptimeMillis() >= silenceEnds) { + speak.run() + } else { + pending = speak + speakAfterSilence() + } + return + } + if (keepSpeech) { + speak.run() + return + } + // Also while draining: speech still queued would otherwise start during the wait. + stopAudio.run() + LowLatencyAudio.pauseWhenQuiet() + pending = speak + handler.postDelayed(timeout, MAX_WAIT_MS) + if (!playing) waitOutDrain() + } + + /** + * Whether a change of [streamType] to [volume] is one of the two that Backtalk's own nudge makes, + * which aren't said. + */ + fun isOwnChange(streamType: Int, volume: Int): Boolean { + if (streamType != AudioManager.STREAM_ACCESSIBILITY) return false + // A change Android never reported mustn't hide one the user makes later. + val now = SystemClock.uptimeMillis() + ownChanges.removeAll { now - it.madeAt > OWN_CHANGE_MS } + // Android sometimes reports only the last of two quick changes, so skip any it left out. + val own = ownChanges.indexOfFirst { it.volume == volume } + if (own >= 0) { + repeat(own + 1) { ownChanges.removeFirst() } + return true + } + // Someone else changed it: the rest of the nudge won't be reported in this order. + ownChanges.clear() + return false + } + + /** + * Sets the accessibility volume one step away and back, which makes Android work out the boost + * again at once, even while speech plays. Setting it to the volume it has is ignored. Returns + * false if the volume could not be changed, and then the announcement waits for quiet instead. + */ + private fun nudgeAccessibilityVolume(streamType: Int): Boolean { + val media = audioManager.getStreamVolume(streamType) + val now = SystemClock.uptimeMillis() + if (media == nudgedForMedia && now - nudgedAt < SAME_CHANGE_MS) return true + val stream = AudioManager.STREAM_ACCESSIBILITY + val index = audioManager.getStreamVolume(stream) + val min = + if (Build.VERSION.SDK_INT >= Build.VERSION_CODES.P) audioManager.getStreamMinVolume(stream) + else 0 + // Down when it can, so the moment between the two changes is quieter rather than louder. + val nudged = if (index > min) index - 1 else index + 1 + if (nudged > audioManager.getStreamMaxVolume(stream)) return false + // Android reports each change later on this thread, so queuing it after the change is in time. + // Changes from an earlier nudge can still be on their way, so these go after them. + try { + audioManager.setStreamVolume(stream, nudged, 0) + if (audioManager.getStreamVolume(stream) != nudged) { + // Android ignored it, or someone else changed the volume first. + return false + } + ownChanges.addLast(OwnChange(nudged, now)) + audioManager.setStreamVolume(stream, index, 0) + ownChanges.addLast(OwnChange(index, now)) + if (audioManager.getStreamVolume(stream) != index) { + // Leaving it a step down would change the user's volume, so try once more. + audioManager.setStreamVolume(stream, index, 0) + Log.w(TAG, "Had to set the accessibility volume back to $index again") + } + nudgedForMedia = media + nudgedAt = now + } catch (e: SecurityException) { + Log.w(TAG, "Not allowed to nudge the accessibility volume", e) + return false + } + return true + } + + /** Drops an announcement still waiting. */ + fun cancel() { + pending = null + handler.removeCallbacks(drained) + handler.removeCallbacks(timeout) + handler.removeCallbacks(primed) + } + + private fun update(nowPlaying: Boolean) { + if (nowPlaying == playing) return + playing = nowPlaying + if (playing) { + handler.removeCallbacks(drained) + } else { + stoppedAt = SystemClock.uptimeMillis() + if (pending != null) waitOutDrain() + } + } + + /** + * Android marks a player stopped as soon as its app stops it, but the audio it already sent keeps + * playing for about 200 to 300 ms on Bluetooth, and the boost changes only once that has played. + */ + private fun waitOutDrain() { + handler.removeCallbacks(drained) + handler.postAtTime(drained, stoppedAt + DRAIN_MS) + } + + /** + * Once quiet, starts accessibility audio with silence: the boost is worked out again a little + * after audio starts, up to 90 ms on Galaxy Buds, and not when more audio joins it. + */ + private fun finish() { + if (pending == null) return + handler.removeCallbacks(drained) + handler.removeCallbacks(timeout) + // Only the timeout comes while audio plays. Audio that never stopped keeps its boost, so silence + // would change nothing. + if (playing) speakNow() else speakAfterSilence() + } + + /** Says [pending] once [PRIME_MS] of silence has played, starting the silence if none plays. */ + private fun speakAfterSilence() { + val now = SystemClock.uptimeMillis() + if (now >= silenceEnds) { + if (!LowLatencyAudio.playSilence(context, SPEECH_ATTRIBUTES, PRIME_MS.toInt())) { + speakNow() + return + } + silenceEnds = now + PRIME_MS + } + handler.removeCallbacks(primed) + handler.postAtTime(primed, silenceEnds) + } + + private fun speakNow() { + val speak = pending ?: return + cancel() + speak.run() + } + + /** + * Whether the announcement for [streamType] needs a nudge or a wait: the media volume changed and + * Backtalk's audio goes to Bluetooth. Other volumes don't change the boost, and an accessibility + * volume change makes Android work it out at once. + */ + private fun needsQuiet(streamType: Int): Boolean = + watching && streamType == AudioManager.STREAM_MUSIC && onBluetooth() + + private class OwnChange(val volume: Int, val madeAt: Long) + + private companion object { + /** How long Android has to report a nudge's change before it's forgotten. */ + const val OWN_CHANGE_MS = 2000L + + /** + * How long after accessibility audio stops the announcement waits. Speech the engine stopped + * played on for 200 to 282 ms on Galaxy Buds. + */ + const val DRAIN_MS = 350L + + /** The longest it waits for speech to stop and then for [DRAIN_MS], before [PRIME_MS]. */ + const val MAX_WAIT_MS = 600L + + /** How long the silence plays before the announcement. */ + const val PRIME_MS = 150L + + /** How close together reports count as the same media change, which one nudge covers. */ + const val SAME_CHANGE_MS = 500L + + /** As Backtalk's speech plays, so the silence goes to its output. */ + val SPEECH_ATTRIBUTES: AudioAttributes = + AudioAttributes.Builder() + .setUsage(AudioAttributes.USAGE_ASSISTANCE_ACCESSIBILITY) + .setContentType(AudioAttributes.CONTENT_TYPE_SPEECH) + .build() + + /** + * Whether accessibility audio plays. Android lists only the players that play to apps such as + * Backtalk. The engine's speech plays in the engine's own app, so players are told apart by + * their usage, not their app. + */ + fun accessibilityAudioPlays(configs: List): Boolean = + configs.any { it.audioAttributes.usage == AudioAttributes.USAGE_ASSISTANCE_ACCESSIBILITY } + + /** Whether Backtalk's audio goes to Bluetooth headphones. */ + @SuppressLint("InlinedApi") // The device types are plain numbers that older Android never uses. + fun accessibilityOnBluetooth(context: Context): Boolean { + val audioManager = context.getSystemService(AudioManager::class.java) + // Not media's output: Backtalk's audio can be sent to the phone speaker instead. + val devices = audioManager.outputsFor(SPEECH_ATTRIBUTES) ?: return false + return devices.any { it.type in BLUETOOTH_MEDIA_TYPES } + } + } +} + +private const val TAG = "BacktalkVolumeQuiet" + +/** + * The devices audio with [attributes] plays on, or null when Android can't say. Before Android 13 + * apps can't ask, so the connected outputs stand in for them. + */ +internal fun AudioManager.outputsFor(attributes: AudioAttributes): List? = + try { + if (Build.VERSION.SDK_INT >= Build.VERSION_CODES.TIRAMISU) { + getAudioDevicesForAttributes(attributes) + } else { + getDevices(AudioManager.GET_DEVICES_OUTPUTS).toList() + } + } catch (e: RuntimeException) { + // Some phones throw here, as AudioDeviceRouter found. + Log.w(TAG, "Could not find where audio plays", e) + null + } + +/** Bluetooth devices that play media. Older versions of Android never report the newer types. */ +@SuppressLint("InlinedApi") +private val BLUETOOTH_MEDIA_TYPES = + setOf( + AudioDeviceInfo.TYPE_BLUETOOTH_A2DP, + AudioDeviceInfo.TYPE_BLE_HEADSET, + AudioDeviceInfo.TYPE_BLE_SPEAKER, + AudioDeviceInfo.TYPE_BLE_BROADCAST, + AudioDeviceInfo.TYPE_HEARING_AID, + ) diff --git a/talkback/src/main/java/com/google/android/accessibility/talkback/monitor/VolumeAnnouncer.kt b/talkback/src/main/java/com/google/android/accessibility/talkback/monitor/VolumeAnnouncer.kt new file mode 100644 index 000000000..96060c549 --- /dev/null +++ b/talkback/src/main/java/com/google/android/accessibility/talkback/monitor/VolumeAnnouncer.kt @@ -0,0 +1,274 @@ +/* + * Copyright 2026 Backtalk contributors + * + * Licensed under the Apache License, Version 2.0 (the "License"); you may not + * use this file except in compliance with the License. You may obtain a copy of + * the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, WITHOUT + * WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the + * License for the specific language governing permissions and limitations under + * the License. + */ + +package com.google.android.accessibility.talkback.monitor + +import android.content.BroadcastReceiver +import android.content.Context +import android.content.Intent +import android.content.IntentFilter +import android.media.AudioAttributes +import android.media.AudioManager +import android.os.Handler +import android.os.Looper +import android.os.SystemClock +import android.view.KeyEvent +import androidx.core.content.ContextCompat +import com.google.android.accessibility.talkback.Feedback +import com.google.android.accessibility.talkback.Pipeline.FeedbackReturner +import com.google.android.accessibility.talkback.R +import com.google.android.accessibility.utils.Performance.EVENT_ID_UNTRACKED +import com.google.android.accessibility.utils.compat.media.AudioManagerCompatUtils +import com.google.android.accessibility.utils.output.FeedbackItem +import com.google.android.accessibility.utils.output.SpeechController +import java.util.function.BooleanSupplier + +/** + * Says the new volume as soon as it changes, for [VolumeMonitor], and interrupts other speech, + * except during continuous reading, which an interruption would stop. TalkBack waited until the + * keys had been still for 2 seconds, and 2 seconds more after the first press. Each new level + * replaces the one before it, if that one hasn't finished. Media is said while it plays too, where + * TalkBack said it only when nothing played. + */ +class VolumeAnnouncer( + private val context: Context, + private val pipeline: FeedbackReturner, + private val callStateMonitor: CallStateMonitor?, + private val monitor: VolumeMonitor, +) { + private val audioManager = context.getSystemService(AudioManager::class.java) + private val handler = Handler(Looper.getMainLooper()) + private val quietBeforeAnnouncement = QuietBeforeAnnouncement(context) + + /** Whether a volume key is held, so that its repeated changes are kept quiet. */ + private var volumeKeyHeld = false + + /** The change held back while a volume key was held, said when the key is released. */ + private var heldChange: HeldChange? = null + + /** The stream that changed last, and when, to keep runs of changes short. */ + private var lastChangedStream = -1 + private var lastChangeTime = 0L + + /** When a volume key last went down, including the repeats of a held key. */ + private var lastVolumeKeyDownTime = -KEY_CHANGE_WINDOW_MS + + /** Where media played at the last media change, to tell when an output brings its own level. */ + private var lastMediaOutput: Set>? = null + + /** The media volume at the last media change, to tell a repeat of it from a change. */ + private var lastMediaVolume = -1 + + /** + * Keeps the accessibility volume of the output in use, which Backtalk needs when it turns off. + * Each output has its own, and Android says when the accessibility stream moves to another one, + * after the move, where a device callback comes before it. + */ + private val outputReceiver = + object : BroadcastReceiver() { + override fun onReceive(context: Context, intent: Intent) { + val streamType = intent.getIntExtra(AudioManagerCompatUtils.EXTRA_VOLUME_STREAM_TYPE, -1) + if (streamType == AudioManager.STREAM_ACCESSIBILITY) monitor.cacheAccessibilityStreamVolume() + } + } + + /** Whether accessibility focus is on a slider, which says its own value. */ + var sliderFocused = BooleanSupplier { false } + + /** Whether continuous reading is on, which an interruption would stop. */ + var continuousReading = BooleanSupplier { false } + + private val announceAfterHold = Runnable { + val change = heldChange + forgetHold() + if (change != null) announceVolumeNow(change.streamType, change.brief) + } + + /** Starts following accessibility audio, for announcements made on Bluetooth, and outputs. */ + fun start() { + lastMediaOutput = mediaOutput() + lastMediaVolume = audioManager.getStreamVolume(AudioManager.STREAM_MUSIC) + ContextCompat.registerReceiver( + context, + outputReceiver, + IntentFilter(AudioManagerCompatUtils.STREAM_DEVICES_CHANGED_ACTION), + ContextCompat.RECEIVER_NOT_EXPORTED, + ) + quietBeforeAnnouncement.start() + } + + /** Stops following audio, and drops a held key's volume still to be said. */ + fun stop() { + context.unregisterReceiver(outputReceiver) + lastMediaOutput = null + lastMediaVolume = -1 + forgetHold() + quietBeforeAnnouncement.stop() + } + + /** + * Whether a change of [streamType] to [volume] isn't the user's, so it's ignored: Backtalk's own, + * or the level of another output, such as headphones that connect. Each output keeps its own + * media volume, so a media change that comes with a new output and that no volume key made is the + * output's. Android can report that change before it says the device connected, so the output is + * looked up when the change comes. Headphones can then report the same level again, which isn't a + * change either, though a volume key that can't go further is still said. + */ + fun isIgnoredChange(streamType: Int, volume: Int): Boolean { + if (quietBeforeAnnouncement.isOwnChange(streamType, volume)) return true + if (streamType != AudioManager.STREAM_MUSIC) return false + val output = mediaOutput() + // Where media played isn't known before start or when Android couldn't say. + val newOutput = lastMediaOutput != null && output != null && output != lastMediaOutput + lastMediaOutput = output + val repeat = volume == lastMediaVolume + lastMediaVolume = volume + return (newOutput || repeat) && !changedByVolumeKey() + } + + /** + * Says a volume change right away, unless a listener says it ([listenerAnnounces]), a focused + * slider says it, or a volume key is held. While the key is held, the change is said once the key + * is released. A change soon after the last one of the same stream says only the percentage. + */ + fun onVolumeChanged(streamType: Int, listenerAnnounces: Boolean) { + val byKey = changedByVolumeKey() + // The key check comes first because finding the focused node asks the app. + if (listenerAnnounces || (!byKey && sliderFocused.asBoolean)) return + // Nothing is said, so the next change after the call names the volume again. + if (isInCall()) return + val now = SystemClock.uptimeMillis() + val brief = streamType == lastChangedStream && now - lastChangeTime < SAME_STREAM_WINDOW_MS + lastChangedStream = streamType + lastChangeTime = now + // A held key repeats many times a second, so a missed release doesn't keep later changes quiet. + if (volumeKeyHeld && byKey) { + heldChange = HeldChange(streamType, brief) + return + } + announceVolumeNow(streamType, brief) + } + + /** + * Follows a volume key that goes to apps. Holding it makes Android repeat the press, which changes + * the volume several times a second, so those changes are kept quiet and the volume is said once + * when the key is released. + */ + fun onVolumeKeyEvent(event: KeyEvent) { + if (event.action == KeyEvent.ACTION_DOWN) { + lastVolumeKeyDownTime = SystemClock.uptimeMillis() + if (event.repeatCount == 0) { + // A new press says its own change, which replaces anything a hold had left to say. + forgetHold() + } else { + volumeKeyHeld = true + } + } else if (event.action == KeyEvent.ACTION_UP && volumeKeyHeld) { + // The volume broadcast for the last repeat can still be on its way, so wait for it. + handler.removeCallbacks(announceAfterHold) + handler.postDelayed(announceAfterHold, HOLD_RELEASE_SETTLE_MS) + } + } + + private fun forgetHold() { + handler.removeCallbacks(announceAfterHold) + volumeKeyHeld = false + heldChange = null + } + + private fun changedByVolumeKey(): Boolean = + SystemClock.uptimeMillis() - lastVolumeKeyDownTime < KEY_CHANGE_WINDOW_MS + + /** @param brief whether to say only the percentage, without the stream name */ + private fun announceVolumeNow(streamType: Int, brief: Boolean) { + // A newer change replaces an announcement still waiting for quiet. + quietBeforeAnnouncement.cancel() + if (!monitor.speakVolumeChanges() || isInCall()) return + // On Bluetooth, a new media volume is said at the level speech has from now on, after a nudge + // of the accessibility volume or once earlier speech has stopped. Continuous reading is not + // stopped for it. + quietBeforeAnnouncement.announce( + streamType, + /* keepSpeech= */ continuousReading.asBoolean, + { + pipeline.returnFeedback( + EVENT_ID_UNTRACKED, + Feedback.Part.builder().setInterruptAllFeedback(true), + ) + }, + { speakVolume(streamType, brief) }, + ) + } + + private fun speakVolume(streamType: Int, brief: Boolean) { + val text = + if (brief) briefAnnouncement(streamType) + else monitor.getAnnouncementForStreamType(R.string.template_stream_volume_set, streamType) + val speakOptions = + SpeechController.SpeakOptions.create() + .setQueueMode( + if (continuousReading.asBoolean) SpeechController.QUEUE_MODE_QUEUE + else SpeechController.QUEUE_MODE_INTERRUPT + ) + .setFlags( + FeedbackItem.FLAG_SOURCE_IS_VOLUME_CONTROL or + FeedbackItem.FLAG_FORCE_FEEDBACK_EVEN_IF_AUDIO_PLAYBACK_ACTIVE or + FeedbackItem.FLAG_CLEAR_QUEUED_UTTERANCES_WITH_SAME_UTTERANCE_GROUP or + FeedbackItem.FLAG_INTERRUPT_CURRENT_UTTERANCE_WITH_SAME_UTTERANCE_GROUP + ) + .setUtteranceGroup(SpeechController.UTTERANCE_GROUP_VOLUME) + pipeline.returnFeedback(EVENT_ID_UNTRACKED, Feedback.speech(text, speakOptions)) + } + + /** + * The volume of the stream as only a percentage, for changes that follow one that already named + * the stream. The ringer still says when it's silent or vibrating. + */ + private fun briefAnnouncement(streamType: Int): String { + if ( + streamType == AudioManager.STREAM_RING && + audioManager.ringerMode != AudioManager.RINGER_MODE_NORMAL + ) { + return monitor.getAnnouncementForStreamType(R.string.template_stream_volume_set, streamType) + } + return context.getString( + R.string.tb_template_percent, + monitor.getStreamVolume(streamType).toString(), + ) + } + + private fun isInCall(): Boolean = callStateMonitor?.isPhoneCallActive == true + + /** The devices media plays on, by ID and type. A device that reconnects gets a new ID. */ + private fun mediaOutput(): Set>? = + audioManager.outputsFor(MEDIA_ATTRIBUTES)?.map { it.id to it.type }?.toSet() + + private class HeldChange(val streamType: Int, val brief: Boolean) + + private companion object { + /** How long to wait after a held volume key is released before saying the volume. */ + const val HOLD_RELEASE_SETTLE_MS = 100L + + /** How soon a change has to follow the last one of the same stream to say only the percentage. */ + const val SAME_STREAM_WINDOW_MS = 2000L + + /** How soon a change has to follow a volume key press to count as caused by it. */ + const val KEY_CHANGE_WINDOW_MS = 1000L + + val MEDIA_ATTRIBUTES: AudioAttributes = + AudioAttributes.Builder().setUsage(AudioAttributes.USAGE_MEDIA).build() + } +} diff --git a/talkback/src/main/java/com/google/android/accessibility/talkback/monitor/VolumeMonitor.java b/talkback/src/main/java/com/google/android/accessibility/talkback/monitor/VolumeMonitor.java index 383e698c9..b886930f0 100644 --- a/talkback/src/main/java/com/google/android/accessibility/talkback/monitor/VolumeMonitor.java +++ b/talkback/src/main/java/com/google/android/accessibility/talkback/monitor/VolumeMonitor.java @@ -28,6 +28,7 @@ import android.os.Message; import android.telephony.TelephonyManager; import android.util.SparseIntArray; +import android.view.KeyEvent; import androidx.annotation.VisibleForTesting; import com.google.android.accessibility.talkback.Feedback; import com.google.android.accessibility.talkback.Feedback.Part; @@ -49,6 +50,7 @@ import java.time.Duration; import java.util.ArrayList; import java.util.List; +import java.util.function.BooleanSupplier; /** Listens for and responds to volume changes. */ public class VolumeMonitor extends SameThreadBroadcastReceiver { @@ -64,6 +66,11 @@ public class VolumeMonitor extends SameThreadBroadcastReceiver { public interface VolumeChangedListener { /** Invoked when the volume has been changed. */ void onVolumeChanged(int type, int value, int prevValue, int percent); + + /** Backtalk: whether this listener says the new volume of {@code type} itself. */ + default boolean announcesVolume(int type) { + return false; + } } private static final SparseIntArray STREAM_NAMES = new SparseIntArray(); @@ -89,6 +96,7 @@ public interface VolumeChangedListener { private final FeedbackReturner pipeline; private final AudioManager audioManager; private final CallStateMonitor callStateMonitor; + private final VolumeAnnouncer volumeAnnouncer; private int cachedAccessibilityStreamVolume = -1; private int cachedAccessibilityStreamMaxVolume = -1; @@ -117,6 +125,17 @@ public VolumeMonitor( audioManager = (AudioManager) context.getSystemService(Context.AUDIO_SERVICE); this.callStateMonitor = callStateMonitor; this.hintA11yVolumeControl = FeatureSupport.hasAccessibilityAudioStream(context); + volumeAnnouncer = new VolumeAnnouncer(context, pipeline, callStateMonitor, this); + } + + /** Backtalk: tells the monitor how to find out whether accessibility focus is on a slider. */ + public void setSliderFocused(BooleanSupplier sliderFocused) { + volumeAnnouncer.setSliderFocused(sliderFocused); + } + + /** Backtalk: tells the monitor how to find out whether continuous reading is on. */ + public void setContinuousReading(BooleanSupplier continuousReading) { + volumeAnnouncer.setContinuousReading(continuousReading); } public void addVolumeChangedListener(VolumeChangedListener listener) { @@ -131,10 +150,14 @@ public void clearVolumeChangedListener() { listeners.clear(); } - private void notifyVolumeChangedListener(int type, int value, int prevValue, int percent) { + /** Backtalk: returns whether a listener says the new volume itself. */ + private boolean notifyVolumeChangedListener(int type, int value, int prevValue, int percent) { + boolean announced = false; for (VolumeChangedListener listener : listeners) { + announced |= listener.announcesVolume(type); listener.onVolumeChanged(type, value, prevValue, percent); } + return announced; } public IntentFilter getFilter() { @@ -164,7 +187,8 @@ private boolean isSelfAdjusted(int streamType, int volume) { * @param prevVolume The previous volume. */ private void internalOnVolumeChanged(int streamType, int volume, int prevVolume) { - if (isSelfAdjusted(streamType, volume)) { + if (isSelfAdjusted(streamType, volume) + || volumeAnnouncer.isIgnoredChange(streamType, volume)) { // Ignore self-adjustments. return; } @@ -181,13 +205,17 @@ private void internalOnVolumeChanged(int streamType, int volume, int prevVolume) } } - notifyVolumeChangedListener(streamType, volume, prevVolume, getStreamVolume(streamType)); + // Backtalk: a gesture or reading control that changed the volume is listening, and says the + // new volume itself. + boolean listenerAnnounces = + notifyVolumeChangedListener(streamType, volume, prevVolume, getStreamVolume(streamType)); if (currentStream < 0) { // If the current stream hasn't been set, acquire control. currentStream = streamType; AudioManagerCompatUtils.forceVolumeControlStream(audioManager, currentStream); handler.onControlAcquired(streamType); + volumeAnnouncer.onVolumeChanged(streamType, listenerAnnounces); return; } @@ -196,9 +224,15 @@ private void internalOnVolumeChanged(int streamType, int volume, int prevVolume) return; } + volumeAnnouncer.onVolumeChanged(streamType, listenerAnnounces); handler.releaseControlDelayed(); } + /** Backtalk: follows a volume key that goes to apps, so a held key is said once. */ + public void onVolumeKeyEvent(KeyEvent event) { + volumeAnnouncer.onVolumeKeyEvent(event); + } + /** * Called after audio stream muted. Handles current stream and providing feedback. Currently, it * works with STREAM_MUSIC only. @@ -218,10 +252,6 @@ private void internalOnStreamMuted(int streamType, boolean mutedState) { text = getAnnouncementForStreamType(R.string.template_stream_volume_set, streamType); } - if (!shouldAnnounceStream(streamType)) { - handler.post(this::releaseControl); - return; - } speakWithCompletion(text, streamType, utteranceCompleteRunnable, EVENT_ID_UNTRACKED); } @@ -283,6 +313,16 @@ private void increaseAndStoreA11yVolumeControlHintTimes(int times) { } } + /** Backtalk: starts following accessibility audio, for announcements made on Bluetooth. */ + public void startWatchingAudio() { + volumeAnnouncer.start(); + } + + /** Backtalk: stops following accessibility audio. */ + public void stopWatchingAudio() { + volumeAnnouncer.stop(); + } + public void cacheAccessibilityStreamVolume() { cachedAccessibilityStreamVolume = audioManager.getStreamVolume(AudioManager.STREAM_ACCESSIBILITY); @@ -315,7 +355,7 @@ private void internalOnControlAcquired(int streamType) { * @param streamType The stream to announce. * @return The volume announcement text for the stream. */ - private String getAnnouncementForStreamType(int templateResId, int streamType) { + String getAnnouncementForStreamType(int templateResId, int streamType) { // The ringer has special cases for silent and vibrate. if (streamType == AudioManager.STREAM_RING) { switch (audioManager.getRingerMode()) { @@ -337,7 +377,8 @@ private String getAnnouncementForStreamType(int templateResId, int streamType) { /** * Called after adjustments have been made and the user has not taken any action for a certain - * duration. Announces the current volume and releases control of the stream. + * duration. Backtalk: says the accessibility volume hint, if it's still due, and releases control + * of the stream. */ private void internalOnReleaseControl() { handler.clearReleaseControl(); @@ -350,16 +391,12 @@ private void internalOnReleaseControl() { LogUtils.v(TAG, "Released control of stream %d", currentStream); - if (!shouldAnnounceStream(streamType) || !speakVolumeChanges()) { - handler.post(this::releaseControl); - return; + // Backtalk: the volume was already said as it changed, so only the hint on how to change the + // accessibility volume is left to say. + if (speakVolumeChanges() && streamType != SpeechController.DEFAULT_STREAM) { + internalSpeakA11yVolumeControlHint(); } - - final String text = - getAnnouncementForStreamType(R.string.template_stream_volume_set, streamType); - - EventId eventId = EVENT_ID_UNTRACKED; // Delayed feedback occurs after normal feedback. - speakWithCompletion(text, streamType, utteranceCompleteRunnable, eventId); + releaseControl(); } /** Releases control of the stream. */ @@ -368,14 +405,8 @@ public void releaseControl() { AudioManagerCompatUtils.forceVolumeControlStream(audioManager, -1); } - /** - * Returns whether a stream type should be announced. - * - * @param streamType The stream type. - * @return True if the stream should be announced. - */ /** Whether the user wants the new volume said after they change it. */ - private boolean speakVolumeChanges() { + boolean speakVolumeChanges() { return SharedPreferencesUtils.getBooleanPref( SharedPreferencesUtils.getSharedPreferences(context), context.getResources(), @@ -383,19 +414,6 @@ private boolean speakVolumeChanges() { R.bool.pref_speak_volume_changes_default); } - private boolean shouldAnnounceStream(int streamType) { - return switch (streamType) { - case AudioManager.STREAM_MUSIC -> - // Only announce music stream if it's not being used. - !audioManager.isMusicActive(); - default -> - // Announce all other streams by default. The VOICE_CALL and - // RING streams are handled by checking the telephony state in - // speakWithCompletion(). - true; - }; - } - /** * Speaks text with a completion action, or just runs the completion action if the volume monitor * should be quiet. @@ -502,7 +520,7 @@ public void onReceiveIntent(Intent intent) { * @return The stream volume as a percentage. */ @SuppressLint("NewApi") - private int getStreamVolume(int streamType) { + int getStreamVolume(int streamType) { int minVolume = 0; // For some stream types other than defined in // https://developer.android.com/reference/android/media/AudioManager#getStreamMinVolume(int), @@ -626,8 +644,7 @@ public void volumeButtonHintDelayed() { public void clearReleaseControl() { removeMessages(MSG_CONTROL); removeMessages(MSG_RELEASE_CONTROL); - // When we announce the volume change, the a11y volume control hint will be in the volume - // value announcement. + // Backtalk: the a11y volume control hint is said when control is released. removeMessages(MSG_VOLUME_BUTTON_HINT); } diff --git a/talkback/src/main/java/com/google/android/accessibility/talkback/selector/SelectorController.java b/talkback/src/main/java/com/google/android/accessibility/talkback/selector/SelectorController.java index 18205393f..5c61073a8 100644 --- a/talkback/src/main/java/com/google/android/accessibility/talkback/selector/SelectorController.java +++ b/talkback/src/main/java/com/google/android/accessibility/talkback/selector/SelectorController.java @@ -2492,6 +2492,11 @@ private void scrollForwardBackward(EventId eventId, boolean isNext) { private final VolumeChangedListener volumeChangedListener = new VolumeChangedListener() { + @Override + public boolean announcesVolume(int type) { + return type == listenedVolumeStream; + } + @Override public void onVolumeChanged(int type, int value, int prevValue, int percent) { if (type != listenedVolumeStream) { diff --git a/talkback/src/test/java/com/google/android/accessibility/talkback/FinalAnnouncementVolumeTest.kt b/talkback/src/test/java/com/google/android/accessibility/talkback/FinalAnnouncementVolumeTest.kt index d36b60537..04ef8aac4 100644 --- a/talkback/src/test/java/com/google/android/accessibility/talkback/FinalAnnouncementVolumeTest.kt +++ b/talkback/src/test/java/com/google/android/accessibility/talkback/FinalAnnouncementVolumeTest.kt @@ -42,6 +42,37 @@ class FinalAnnouncementVolumeTest { assertTrue(measured < guessed / 2f) } + @Test + fun decibelsAreUsedAsTheyAre() { + assertEquals(0.1f, FinalAnnouncementVolume.gainFromReportedVolumes(-40f, -20f), 1e-4f) + assertEquals(0.1f, FinalAnnouncementVolume.gainFromReportedVolumes(-20f, 0f), 1e-4f) + // A volume curve that boosts media above 0 dB is still in decibels. + assertEquals(0.1f, FinalAnnouncementVolume.gainFromReportedVolumes(-18f, 2f), 1e-4f) + assertEquals(0.7943f, FinalAnnouncementVolume.gainFromReportedVolumes(0f, 2f), 1e-4f) + } + + @Test + fun samsungsFractionsOfFullLoudnessAreDivided() { + // A Samsung phone on its speaker: accessibility volume 4 and media volume 7 out of 15. + assertEquals(0.1288f, FinalAnnouncementVolume.gainFromReportedVolumes(0.01f, 0.0776f), 1e-4f) + assertEquals(1f, FinalAnnouncementVolume.gainFromReportedVolumes(1f, 0.0776f), 0f) + assertEquals(1f, FinalAnnouncementVolume.gainFromReportedVolumes(1f, 1f), 0f) + } + + @Test + fun mediaStepsAreMatchedToAccessibilitySteps() { + // Media from step 0 and accessibility from step 1, both to 15, as on a Samsung phone, where + // media 7 set accessibility 8 and media 8 set it to 9. + assertEquals(8, FinalAnnouncementVolume.accessibilityStepAtMediaVolume(7, 0, 15, 1, 15)) + assertEquals(9, FinalAnnouncementVolume.accessibilityStepAtMediaVolume(8, 0, 15, 1, 15)) + // The emulator: media 9 set accessibility 9. + assertEquals(9, FinalAnnouncementVolume.accessibilityStepAtMediaVolume(9, 0, 15, 1, 15)) + assertEquals(15, FinalAnnouncementVolume.accessibilityStepAtMediaVolume(15, 0, 15, 1, 15)) + assertEquals(9, FinalAnnouncementVolume.accessibilityStepAtMediaVolume(9, 0, 15, 0, 15)) + assertEquals(3, FinalAnnouncementVolume.accessibilityStepAtMediaVolume(5, 0, 25, 0, 15)) + assertEquals(15, FinalAnnouncementVolume.accessibilityStepAtMediaVolume(25, 0, 25, 0, 15)) + } + @Test fun sliderGuessKeepsTalkBackBehavior() { assertEquals(1f, FinalAnnouncementVolume.sliderGain(0.6f, 0.4f), 0f) diff --git a/talkback/src/test/java/com/google/android/accessibility/talkback/monitor/QuietBeforeAnnouncementTest.kt b/talkback/src/test/java/com/google/android/accessibility/talkback/monitor/QuietBeforeAnnouncementTest.kt new file mode 100644 index 000000000..cec766db2 --- /dev/null +++ b/talkback/src/test/java/com/google/android/accessibility/talkback/monitor/QuietBeforeAnnouncementTest.kt @@ -0,0 +1,141 @@ +/* + * Copyright 2026 Backtalk contributors + * + * Licensed under the Apache License, Version 2.0 (the "License"); you may not + * use this file except in compliance with the License. You may obtain a copy of + * the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, WITHOUT + * WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the + * License for the specific language governing permissions and limitations under + * the License. + */ + +package com.google.android.accessibility.talkback.monitor + +import android.content.Context +import android.media.AudioManager +import android.media.AudioManager.STREAM_ACCESSIBILITY +import android.media.AudioManager.STREAM_MUSIC +import android.os.Looper +import java.time.Duration +import org.junit.Assert.assertEquals +import org.junit.Assert.assertFalse +import org.junit.Assert.assertTrue +import org.junit.Before +import org.junit.Test +import org.junit.runner.RunWith +import org.robolectric.RobolectricTestRunner +import org.robolectric.RuntimeEnvironment +import org.robolectric.Shadows.shadowOf +import org.robolectric.annotation.Config + +@RunWith(RobolectricTestRunner::class) +@Config(sdk = [34]) +class QuietBeforeAnnouncementTest { + private val context: Context + get() = RuntimeEnvironment.getApplication() + + private lateinit var audioManager: AudioManager + private var onBluetooth = true + private var spoken = 0 + private lateinit var quiet: QuietBeforeAnnouncement + + @Before + fun setUp() { + audioManager = context.getSystemService(AudioManager::class.java) + shadowOf(audioManager).setStreamMaxVolume(15) + audioManager.setStreamVolume(STREAM_ACCESSIBILITY, 5, 0) + audioManager.setStreamVolume(STREAM_MUSIC, 8, 0) + quiet = QuietBeforeAnnouncement(context) { onBluetooth } + quiet.start() + } + + private fun announceMediaVolume() { + quiet.announce(STREAM_MUSIC, false, {}, { spoken++ }) + shadowOf(Looper.getMainLooper()).idleFor(Duration.ofSeconds(1)) + } + + @Test + fun nudgesOneStepDownAndBack() { + announceMediaVolume() + + assertEquals(1, spoken) + assertEquals(5, audioManager.getStreamVolume(STREAM_ACCESSIBILITY)) + assertTrue(quiet.isOwnChange(STREAM_ACCESSIBILITY, 4)) + assertTrue(quiet.isOwnChange(STREAM_ACCESSIBILITY, 5)) + assertFalse(quiet.isOwnChange(STREAM_ACCESSIBILITY, 5)) + } + + @Test + fun recognizesANudgeWhenOnlyItsLastChangeIsReported() { + announceMediaVolume() + + assertTrue(quiet.isOwnChange(STREAM_ACCESSIBILITY, 5)) + assertFalse(quiet.isOwnChange(STREAM_ACCESSIBILITY, 4)) + } + + @Test + fun aChangeByAnyoneElseIsSaidAndEndsTheNudge() { + announceMediaVolume() + + assertFalse(quiet.isOwnChange(STREAM_ACCESSIBILITY, 9)) + assertFalse(quiet.isOwnChange(STREAM_ACCESSIBILITY, 4)) + } + + @Test + fun nudgesOnceForReportsOfTheSameMediaChange() { + quiet.announce(STREAM_MUSIC, false, {}, {}) + announceMediaVolume() + + assertTrue(quiet.isOwnChange(STREAM_ACCESSIBILITY, 5)) + assertFalse(quiet.isOwnChange(STREAM_ACCESSIBILITY, 5)) + } + + @Test + fun nudgesAgainForTheSameMediaVolumeLater() { + announceMediaVolume() + announceMediaVolume() + + assertEquals(2, spoken) + assertTrue(quiet.isOwnChange(STREAM_ACCESSIBILITY, 5)) + assertTrue(quiet.isOwnChange(STREAM_ACCESSIBILITY, 5)) + } + + @Test + fun forgetsTheNudgeWhenStopped() { + announceMediaVolume() + quiet.stop() + + assertFalse(quiet.isOwnChange(STREAM_ACCESSIBILITY, 4)) + } + + @Test + fun leavesTheVolumeAloneWithoutBluetooth() { + onBluetooth = false + + announceMediaVolume() + + assertEquals(1, spoken) + assertFalse(quiet.isOwnChange(STREAM_ACCESSIBILITY, 4)) + } + + @Test + fun leavesOtherVolumesAlone() { + quiet.announce(AudioManager.STREAM_RING, false, {}, { spoken++ }) + + assertEquals(1, spoken) + assertFalse(quiet.isOwnChange(STREAM_ACCESSIBILITY, 4)) + } + + @Test + fun forgetsANudgeAndroidNeverReported() { + announceMediaVolume() + shadowOf(Looper.getMainLooper()).idleFor(Duration.ofSeconds(2)) + + assertFalse(quiet.isOwnChange(STREAM_ACCESSIBILITY, 4)) + } +} diff --git a/talkback/src/test/java/com/google/android/accessibility/talkback/monitor/VolumeMonitorTest.kt b/talkback/src/test/java/com/google/android/accessibility/talkback/monitor/VolumeMonitorTest.kt new file mode 100644 index 000000000..89fab90b6 --- /dev/null +++ b/talkback/src/test/java/com/google/android/accessibility/talkback/monitor/VolumeMonitorTest.kt @@ -0,0 +1,451 @@ +/* + * Copyright 2026 Backtalk contributors + * + * Licensed under the Apache License, Version 2.0 (the "License"); you may not + * use this file except in compliance with the License. You may obtain a copy of + * the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, WITHOUT + * WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the + * License for the specific language governing permissions and limitations under + * the License. + */ + +package com.google.android.accessibility.talkback.monitor + +import android.content.Context +import android.content.Intent +import android.media.AudioAttributes +import android.media.AudioDeviceInfo +import android.media.AudioManager +import android.media.AudioManager.STREAM_ACCESSIBILITY +import android.media.AudioManager.STREAM_MUSIC +import android.os.Looper +import android.view.KeyEvent +import com.google.android.accessibility.talkback.Feedback +import com.google.android.accessibility.talkback.Pipeline +import com.google.android.accessibility.talkback.R +import com.google.android.accessibility.utils.FormFactorUtils +import com.google.android.accessibility.utils.SharedPreferencesUtils +import com.google.android.accessibility.utils.compat.media.AudioManagerCompatUtils +import com.google.android.accessibility.utils.output.SpeechController +import com.google.common.collect.ImmutableList +import java.time.Duration +import org.junit.Assert.assertEquals +import org.junit.Before +import org.junit.Test +import org.junit.runner.RunWith +import org.robolectric.RobolectricTestRunner +import org.robolectric.RuntimeEnvironment +import org.robolectric.Shadows.shadowOf +import org.robolectric.annotation.Config +import org.robolectric.shadows.AudioDeviceInfoBuilder + +@RunWith(RobolectricTestRunner::class) +@Config(sdk = [34]) +class VolumeMonitorTest { + private val context: Context + get() = RuntimeEnvironment.getApplication() + + private val spoken = mutableListOf() + private val pipeline = + object : Pipeline.FeedbackReturner { + override fun returnFeedback(feedback: Feedback): Boolean { + feedback.failovers().first().speech()?.let { spoken += it } + return true + } + } + + private lateinit var audioManager: AudioManager + private lateinit var monitor: VolumeMonitor + + @Before + fun setUp() { + FormFactorUtils.initialize(context) + audioManager = context.getSystemService(Context.AUDIO_SERVICE) as AudioManager + shadowOf(audioManager).setStreamMaxVolume(10) + // Without this, the first changes also teach how to change the accessibility volume. + SharedPreferencesUtils.putIntPref( + SharedPreferencesUtils.getSharedPreferences(context), + context.resources, + R.string.pref_a11y_volume_control_hint_times, + 3, + ) + monitor = VolumeMonitor(pipeline, context, null) + } + + @Test + fun eachChangeIsSaidRightAway() { + change(STREAM_MUSIC, 5, 4) + change(STREAM_MUSIC, 6, 5) + + assertEquals( + listOf("Music volume set to 50 percent.", "60 percent"), + texts(), + ) + spoken.forEach { + assertEquals(SpeechController.UTTERANCE_GROUP_VOLUME, it.options()!!.mUtteranceGroup) + assertEquals(SpeechController.QUEUE_MODE_INTERRUPT, it.options()!!.mQueueMode) + } + } + + @Test + fun aChangeAfterAPauseNamesTheVolumeAgain() { + change(STREAM_MUSIC, 5, 4) + shadowOf(Looper.getMainLooper()).idleFor(Duration.ofSeconds(3)) + change(STREAM_MUSIC, 6, 5) + + assertEquals( + listOf("Music volume set to 50 percent.", "Music volume set to 60 percent."), + texts(), + ) + } + + @Test + fun aChangeToAnotherVolumeNamesIt() { + change(STREAM_MUSIC, 5, 4) + change(STREAM_ACCESSIBILITY, 5, 4) + + assertEquals( + listOf("Music volume set to 50 percent.", "Accessibility volume set to 50 percent."), + texts(), + ) + } + + @Test + fun continuousReadingIsNotInterrupted() { + monitor.setContinuousReading { true } + + change(STREAM_MUSIC, 5, 4) + + assertEquals(SpeechController.QUEUE_MODE_QUEUE, spoken.single().options()!!.mQueueMode) + } + + @Test + fun aFocusedSliderSaysItsOwnValue() { + monitor.setSliderFocused { true } + + change(STREAM_MUSIC, 5, 4) + + assertEquals(emptyList(), texts()) + } + + @Test + fun theVolumeKeysAreSaidOnAFocusedSlider() { + monitor.setSliderFocused { true } + + key(KeyEvent.ACTION_DOWN, repeat = 0) + change(STREAM_MUSIC, 5, 4) + + assertEquals(listOf("Music volume set to 50 percent."), texts()) + } + + @Test + fun theMediaVolumeIsSaidWhileMediaPlays() { + shadowOf(audioManager).setIsMusicActive(true) + + change(STREAM_MUSIC, 5, 4) + + assertEquals(listOf("Music volume set to 50 percent."), texts()) + } + + @Test + fun theVolumeHeadphonesSetWhenTheyConnectIsNotSaid() { + monitor.startWatchingAudio() + // Android can report the headphones' level before it says they connected. + playMediaOn(headphones()) + + change(STREAM_MUSIC, 5, 4) + + assertEquals(emptyList(), texts()) + } + + @Test + fun theHeadphonesRepeatingTheirLevelIsNotSaid() { + monitor.startWatchingAudio() + playMediaOn(headphones()) + + change(STREAM_MUSIC, 5, 4) + change(STREAM_MUSIC, 5, 5) + + assertEquals(emptyList(), texts()) + } + + @Test + fun aVolumeKeyAtTheSameLevelIsSaid() { + monitor.startWatchingAudio() + change(STREAM_MUSIC, 10, 9) + spoken.clear() + shadowOf(Looper.getMainLooper()).idleFor(Duration.ofSeconds(10)) + + key(KeyEvent.ACTION_DOWN, repeat = 0) + change(STREAM_MUSIC, 10, 10) + + assertEquals(listOf("Music volume set to 100 percent."), texts()) + } + + @Test + fun aVolumeKeyIsSaidOnANewOutput() { + monitor.startWatchingAudio() + playMediaOn(headphones()) + + key(KeyEvent.ACTION_DOWN, repeat = 0) + change(STREAM_MUSIC, 5, 4) + + assertEquals(listOf("Music volume set to 50 percent."), texts()) + } + + @Test + fun theHeadphoneControlsAreSaidAfterTheHeadphonesSetTheirVolume() { + monitor.startWatchingAudio() + playMediaOn(headphones()) + + change(STREAM_MUSIC, 5, 4) + change(STREAM_MUSIC, 6, 5) + + assertEquals(listOf("Music volume set to 60 percent."), texts()) + } + + @Test + fun theSpeakerLevelIsNotSaidWhenHeadphonesDisconnect() { + monitor.startWatchingAudio() + playMediaOn(headphones()) + change(STREAM_MUSIC, 5, 4) + playMediaOn(speaker()) + + change(STREAM_MUSIC, 3, 5) + change(STREAM_MUSIC, 4, 3) + + assertEquals(listOf("Music volume set to 40 percent."), texts()) + } + + @Test + fun headphonesConnectedBeforeBacktalkStartsAreNotWaitedFor() { + playMediaOn(headphones()) + monitor.startWatchingAudio() + + change(STREAM_MUSIC, 5, 4) + + assertEquals(listOf("Music volume set to 50 percent."), texts()) + } + + @Test + fun theAccessibilityVolumeIsKeptWhenItMovesToAnotherOutput() { + monitor.startWatchingAudio() + monitor.cacheAccessibilityStreamVolume() + audioManager.setStreamVolume(STREAM_ACCESSIBILITY, 3, 0) + + context.sendBroadcast( + Intent(AudioManagerCompatUtils.STREAM_DEVICES_CHANGED_ACTION) + .setPackage(context.packageName) + .putExtra(AudioManagerCompatUtils.EXTRA_VOLUME_STREAM_TYPE, STREAM_ACCESSIBILITY) + ) + shadowOf(Looper.getMainLooper()).idle() + + assertEquals(3, monitor.cachedAccessibilityStreamVolume) + } + + @Test + fun nothingMoreIsSaidWhenTheKeysAreStill() { + change(STREAM_MUSIC, 5, 4) + spoken.clear() + + shadowOf(Looper.getMainLooper()).idleFor(Duration.ofSeconds(10)) + + assertEquals(emptyList(), texts()) + } + + @Test + fun aGestureThatSaysTheVolumeItselfIsNotSpokenOver() { + var heard = 0 + monitor.addVolumeChangedListener( + object : VolumeMonitor.VolumeChangedListener { + override fun onVolumeChanged(type: Int, value: Int, prevValue: Int, percent: Int) { + heard++ + } + + override fun announcesVolume(type: Int) = type == STREAM_ACCESSIBILITY + } + ) + + change(STREAM_ACCESSIBILITY, 5, 4) + + assertEquals(1, heard) + assertEquals(emptyList(), texts()) + } + + @Test + fun aListenerThatDoesNotSayTheVolumeDoesNotSilenceIt() { + monitor.addVolumeChangedListener { _, _, _, _ -> } + + change(STREAM_ACCESSIBILITY, 5, 4) + + assertEquals(listOf("Accessibility volume set to 50 percent."), texts()) + } + + @Test + fun aMediaVolumeReadingControlIsNotSpokenOver() { + monitor.addVolumeChangedListener( + object : VolumeMonitor.VolumeChangedListener { + override fun onVolumeChanged(type: Int, value: Int, prevValue: Int, percent: Int) {} + + override fun announcesVolume(type: Int) = type == STREAM_MUSIC + } + ) + + change(STREAM_MUSIC, 5, 4) + + assertEquals(emptyList(), texts()) + } + + @Test + fun anAccessibilityVolumeListenerDoesNotSilenceTheMediaVolumeKeys() { + monitor.addVolumeChangedListener( + object : VolumeMonitor.VolumeChangedListener { + override fun onVolumeChanged(type: Int, value: Int, prevValue: Int, percent: Int) {} + + override fun announcesVolume(type: Int) = type == STREAM_ACCESSIBILITY + } + ) + + change(STREAM_MUSIC, 5, 4) + + assertEquals(listOf("Music volume set to 50 percent."), texts()) + } + + @Test + fun theAccessibilityVolumeHintIsSaidWhenTheKeysAreStill() { + SharedPreferencesUtils.putIntPref( + SharedPreferencesUtils.getSharedPreferences(context), + context.resources, + R.string.pref_a11y_volume_control_hint_times, + 0, + ) + monitor = VolumeMonitor(pipeline, context, null) + change(STREAM_MUSIC, 5, 4) + spoken.clear() + + shadowOf(Looper.getMainLooper()).idleFor(Duration.ofSeconds(10)) + + assertEquals( + listOf(context.getString(R.string.hint_a11y_volume_control)), + spoken.map { it.hint().toString() }, + ) + } + + @Test + fun nothingIsSaidWhenSpeakVolumeChangesIsOff() { + SharedPreferencesUtils.putBooleanPref( + SharedPreferencesUtils.getSharedPreferences(context), + context.resources, + R.string.pref_speak_volume_changes_key, + false, + ) + + change(STREAM_MUSIC, 5, 4) + + assertEquals(emptyList(), texts()) + } + + @Test + fun holdingTheKeySaysOnlyTheLevelItEndsAt() { + key(KeyEvent.ACTION_DOWN, repeat = 0) + change(STREAM_MUSIC, 5, 4) + key(KeyEvent.ACTION_DOWN, repeat = 1) + change(STREAM_MUSIC, 6, 5) + key(KeyEvent.ACTION_DOWN, repeat = 2) + change(STREAM_MUSIC, 7, 6) + + assertEquals(listOf("Music volume set to 50 percent."), texts()) + + key(KeyEvent.ACTION_UP) + shadowOf(Looper.getMainLooper()).idleFor(Duration.ofMillis(200)) + + assertEquals( + listOf("Music volume set to 50 percent.", "70 percent"), + texts(), + ) + } + + @Test + fun aChangeThatArrivesJustAfterTheReleaseIsSaidOnce() { + key(KeyEvent.ACTION_DOWN, repeat = 0) + change(STREAM_MUSIC, 5, 4) + key(KeyEvent.ACTION_DOWN, repeat = 1) + key(KeyEvent.ACTION_UP) + change(STREAM_MUSIC, 6, 5) + shadowOf(Looper.getMainLooper()).idleFor(Duration.ofMillis(200)) + + assertEquals( + listOf("Music volume set to 50 percent.", "60 percent"), + texts(), + ) + } + + @Test + fun aHoldWithoutAChangeSaysNothingMore() { + key(KeyEvent.ACTION_DOWN, repeat = 0) + change(STREAM_MUSIC, 10, 9) + key(KeyEvent.ACTION_DOWN, repeat = 1) + key(KeyEvent.ACTION_UP) + shadowOf(Looper.getMainLooper()).idleFor(Duration.ofMillis(200)) + + assertEquals(listOf("Music volume set to 100 percent."), texts()) + } + + @Test + fun aPressAfterAHoldIsSaidRightAway() { + key(KeyEvent.ACTION_DOWN, repeat = 0) + change(STREAM_MUSIC, 5, 4) + key(KeyEvent.ACTION_DOWN, repeat = 1) + change(STREAM_MUSIC, 6, 5) + key(KeyEvent.ACTION_UP) + shadowOf(Looper.getMainLooper()).idleFor(Duration.ofMillis(200)) + key(KeyEvent.ACTION_DOWN, repeat = 0) + change(STREAM_MUSIC, 7, 6) + + assertEquals( + listOf( + "Music volume set to 50 percent.", + "60 percent", + "70 percent", + ), + texts(), + ) + } + + private fun key(action: Int, repeat: Int = 0) { + monitor.onVolumeKeyEvent(KeyEvent(0, 0, action, KeyEvent.KEYCODE_VOLUME_UP, repeat)) + } + + private fun change(stream: Int, value: Int, prevValue: Int) { + audioManager.setStreamVolume(stream, value, 0) + monitor.onReceiveIntent( + Intent(AudioManagerCompatUtils.VOLUME_CHANGED_ACTION) + .putExtra(AudioManagerCompatUtils.EXTRA_VOLUME_STREAM_TYPE, stream) + .putExtra(AudioManagerCompatUtils.EXTRA_VOLUME_STREAM_VALUE, value) + .putExtra(AudioManagerCompatUtils.EXTRA_PREV_VOLUME_STREAM_VALUE, prevValue) + ) + // Runs what is already due, without letting any time pass. + shadowOf(Looper.getMainLooper()).idle() + } + + private fun headphones(): AudioDeviceInfo = + AudioDeviceInfoBuilder.newBuilder().setType(AudioDeviceInfo.TYPE_BLUETOOTH_A2DP).build() + + private fun speaker(): AudioDeviceInfo = + AudioDeviceInfoBuilder.newBuilder().setType(AudioDeviceInfo.TYPE_BUILTIN_SPEAKER).build() + + private fun playMediaOn(device: AudioDeviceInfo) { + shadowOf(audioManager) + .setAudioDevicesForAttributes( + AudioAttributes.Builder().setUsage(AudioAttributes.USAGE_MEDIA).build(), + ImmutableList.of(device), + ) + } + + private fun texts() = spoken.map { it.text().toString() } +} diff --git a/utils/src/main/java/com/google/android/accessibility/utils/compat/media/AudioManagerCompatUtils.java b/utils/src/main/java/com/google/android/accessibility/utils/compat/media/AudioManagerCompatUtils.java index 63281b320..af38df78b 100644 --- a/utils/src/main/java/com/google/android/accessibility/utils/compat/media/AudioManagerCompatUtils.java +++ b/utils/src/main/java/com/google/android/accessibility/utils/compat/media/AudioManagerCompatUtils.java @@ -61,6 +61,10 @@ public class AudioManagerCompatUtils { public static final String STREAM_MUTE_CHANGED_ACTION = "android.media.STREAM_MUTE_CHANGED_ACTION"; + /** Backtalk: sent after a stream moves to other outputs, with {@link #EXTRA_VOLUME_STREAM_TYPE}. */ + public static final String STREAM_DEVICES_CHANGED_ACTION = + "android.media.STREAM_DEVICES_CHANGED_ACTION"; + /** The mute statefor the stream mute changed intent. */ public static final String EXTRA_STREAM_VOLUME_MUTED = "android.media.EXTRA_STREAM_VOLUME_MUTED"; diff --git a/utils/src/main/java/com/google/android/accessibility/utils/output/LowLatencyAudio.kt b/utils/src/main/java/com/google/android/accessibility/utils/output/LowLatencyAudio.kt index 72a3917e8..a19dc47c9 100644 --- a/utils/src/main/java/com/google/android/accessibility/utils/output/LowLatencyAudio.kt +++ b/utils/src/main/java/com/google/android/accessibility/utils/output/LowLatencyAudio.kt @@ -94,6 +94,10 @@ class LowLatencyAudio private constructor(context: Context, private val attribut private var watchdog: ScheduledFuture<*>? = null private var thread: Thread? = null private var idleFrames = 0 + // Set by pauseWhenQuiet: the track pauses as soon as nothing plays, not after IDLE_MS. + private var pauseSoon = false + // The last clip of silence made by playSilence, kept since every call asks for the same length. + private var silence: Clip? = null // Whether the track has audio queued since it last paused. Without the fast path, it pauses as // soon as its audio has been heard, after silentFrames of silence. private var primed = false @@ -496,6 +500,8 @@ class LowLatencyAudio private constructor(context: Context, private val attribut /** Starts the audio thread and its watchdog if they are not running. Called with [lock] held. */ private fun wake() { idleFrames = 0 + // New audio replaces a request to pause once quiet, so the track keeps IDLE_MS after it. + pauseSoon = false // A thread waiting for speech carries on with it. lock.notifyAll() if (thread != null || released || broken) return @@ -593,7 +599,9 @@ class LowLatencyAudio private constructor(context: Context, private val attribut synchronized(lock) { val current = track val idle = clips.isEmpty() && (streams.isEmpty() || held) - if (released || broken || current == null || (idle && idleFrames >= idleLimit())) { + if ( + released || broken || current == null || (idle && (pauseSoon || idleFrames >= idleLimit())) + ) { end(current) return } @@ -1023,6 +1031,37 @@ class LowLatencyAudio private constructor(context: Context, private val attribut /** Whether a player sends its track to the media output, away from the earpiece. */ @JvmStatic fun isAvoidingEarpiece(): Boolean = players().any { it.detour != null } + /** + * Makes every player pause its track as soon as nothing plays, rather than after IDLE_MS of + * silence, so that Android sees accessibility audio stop. + */ + @JvmStatic + fun pauseWhenQuiet() { + for (player in players()) { + synchronized(player.lock) { + if (player.thread != null) { + player.pauseSoon = true + player.lock.notifyAll() + } + } + } + } + + /** + * Plays [millis] of silence on the player for [attributes]' usage, which starts its track if it + * was paused. Returns false if the device cannot make a low-latency track. + */ + @JvmStatic + fun playSilence(context: Context, attributes: AudioAttributes, millis: Int): Boolean { + val player = get(context, attributes) ?: return false + val size = player.sampleRate * millis / 1000 * 2 + val clip = + player.silence?.takeIf { it.size == size && it.rate == player.sampleRate } + ?: Clip(FloatArray(size), player.sampleRate).also { player.silence = it } + player.play(clip, 0f, 0f, 1f) + return true + } + /** Stops every speech stream of every player made so far, without making new ones. */ @JvmStatic fun stopAllStreams() { diff --git a/utils/src/main/java/com/google/android/accessibility/utils/output/SpeechController.java b/utils/src/main/java/com/google/android/accessibility/utils/output/SpeechController.java index fad6e442e..300d7108e 100644 --- a/utils/src/main/java/com/google/android/accessibility/utils/output/SpeechController.java +++ b/utils/src/main/java/com/google/android/accessibility/utils/output/SpeechController.java @@ -143,6 +143,8 @@ public interface SpeechController { int UTTERANCE_GROUP_CONTENT_HINTS = 6; // Backtalk: speech that other apps ask for, so that it can be replaced or stopped on its own. int UTTERANCE_GROUP_OTHER_APPS = 7; + // Backtalk: the new volume, said each time it changes, so that a newer level replaces an older one. + int UTTERANCE_GROUP_VOLUME = 8; /** Enumeration of utterance-group values. */ @IntDef({ @@ -153,7 +155,8 @@ public interface SpeechController { UTTERANCE_GROUP_SCREEN_MAGNIFICATION, UTTERANCE_GROUP_CONTENT_CHANGE, UTTERANCE_GROUP_CONTENT_HINTS, - UTTERANCE_GROUP_OTHER_APPS + UTTERANCE_GROUP_OTHER_APPS, + UTTERANCE_GROUP_VOLUME }) @Retention(RetentionPolicy.SOURCE) public @interface UtteranceGroup {} @@ -434,6 +437,7 @@ public static boolean hasQueueModeFlagSet(int queueMode, int flag) { case UTTERANCE_GROUP_CONTENT_CHANGE -> "UTTERANCE_GROUP_CONTENT_CHANGE"; case UTTERANCE_GROUP_CONTENT_HINTS -> "UTTERANCE_GROUP_CONTENT_HINTS"; case UTTERANCE_GROUP_OTHER_APPS -> "UTTERANCE_GROUP_OTHER_APPS"; + case UTTERANCE_GROUP_VOLUME -> "UTTERANCE_GROUP_VOLUME"; default -> "(unknown utterance group)"; }; }