Fail repository-ai-translate when the non-batch report has errors by ehoogerbeets · Pull Request #1107 · box/mojito

ehoogerbeets · 2026-10-14T19:55:57Z

Summary

After a non-batch repository-ai-translate pollable finishes, the CLI always loads the import report and fails non-zero when any report line has a non-null error (timeouts, OpenAI failures, etc.), so callers do not treat skipped translations as success.
Without --download-report, prints a compact summary (errors grouped by locale with affected tmTextUnitIds). With --download-report, writes locale JSON files and lists each failed unit in detail.
Documented as a breaking change in the AI Translate guide.

Test plan

./mvnw -pl cli -am test -Dtest=RepositoryAiTranslationCommandTest -Dsurefire.failIfNoSpecifiedTests=false
Run mojito repository-ai-translate against a locale/unit that produces a report error and confirm summary (no Finished), non-zero exit
Run a successful non-batch translate and confirm Finished / exit 0
With --download-report, confirm JSON files are still written and per-unit error lines are printed

Per-unit OpenAI failures were only stored in the import report, so the CLI printed Finished and exited 0 even when translations were skipped. Co-authored-by: Cursor <cursoragent@cursor.com>

kkuliczkowski-box · 2026-10-19T15:24:28Z

+    if (!errorLines.isEmpty()) {
+      consoleWriter
+          .fg(Color.RED)
+          .newLine()
+          .a("AI translate failed for ")
+          .a(errorLines.size())
+          .a(" text unit(s):")
+          .println();
+      for (ImportReportLine line : errorLines) {
+        consoleWriter
+            .fg(Color.RED)
+            .a("- locale: ")
+            .a(line.locale())
+            .a(", tmTextUnitId: ")
+            .a(line.tmTexUnitId())
+            .a(": ")
+            .a(line.error())
+            .println();
+      }
+      throw new CommandException(
+          "AI translate completed with %d error(s)".formatted(errorLines.size()));
+    }


RepositoryAiTranslationCommand.java L369-390

This changes the exit contract for automated callers: a single OpenAI timeout now makes the whole run exit non-zero, even though every other text unit was already imported. Jobs that chain steps after this command (with set -e, for example) will stop at that point. Could we add an opt-out flag, such as --fail-on-error (default true), or a --max-errors N threshold, so pipelines can choose to tolerate occasional per-unit failures? At minimum, please call out the new non-zero exit in the release notes.

This is on purpose. We want to know when to retry. If it doesn't exit with an exit code, we'll never know.

Documented the non-zero exit as a breaking change in the AI Translate guide (docs/_docs/guides/014-ai-translate.md). We're intentionally not adding an opt-out: callers need the exit code to know when to retry.

kkuliczkowski-box · 2026-10-19T15:25:03Z

+    for (String reportLocaleUrl : report.reportLocaleUrls()) {
+      RepositoryAiTranslateClient.ProtoAiTranslateGetReportLocaleResponse reportLocale =
+          repositoryAiTranslateClient.getReportLocale(reportLocaleUrl);
+
+      if (downloadReport) {
+        Path path = Path.of(reportLocaleUrl + ".json");
+        Files.createDirectories(path.getParent());
+        Files.write(path, reportLocale.content());
+        consoleWriter.a("- ").a(path.toAbsolutePath().toString()).println();
+      }
+
+      List<ImportReportLine> lines =
+          reportMapper.readValueUnchecked(
+              reportLocale.content(), new TypeReference<List<ImportReportLine>>() {});


RepositoryAiTranslationCommand.java L348-361 (payload: AiTranslateService.java L591-592)

Before this PR, the report was only fetched with --download-report. Now every non-batch run downloads every locale report in full. Each ImportReportLine includes the whole ResponsesRequest (prompt, glossary terms, base64 screenshot data URL), repeated on every line of the same screenshot group, only so the CLI can read error. On screenshot-heavy runs that can mean many MB per locale. Could the server expose an error summary instead (an error count plus the failed tmTextUnitIds, either on the report endpoint or in the pollable task output)? Then the full report could stay behind --download-report.

Agreed that downloading the full locale report (with ResponsesRequest / screenshot payloads) just to read error is heavy — but detecting failures still requires fetching the report today, and we're keeping this PR CLI-only.

Without --download-report, the CLI now prints only a compact summary (errors grouped by locale with affected tmTextUnitIds) instead of dumping per-unit detail. With --download-report, behavior is unchanged (write full JSON + list each failed unit).

A server-side error summary / mid-run signal is tracked separately: LXP-2810.

kkuliczkowski-box · 2026-10-19T15:26:46Z

RepositoryAiTranslationCommandTest.java L59-111

Could we add two cases:

downloadReport = true combined with a report error: assert that the locale JSON file is still written (to a temp dir) before the CommandException is thrown. The write block moved into the new loop and isn't tested right now.

Two locales with a mix of failed and successful lines: assert that the count adds up across locales ("completed with N error(s)") and that lines without errors aren't listed.

Added both cases in RepositoryAiTranslationCommandTest:

downloadReport = true + report error — asserts the locale JSON is written before CommandException

Two locales with mixed success/failure — asserts the aggregate error count and that successful lines are not listed

kkuliczkowski-box · 2026-10-19T15:27:55Z

+      for (ImportReportLine line : errorLines) {
+        consoleWriter
+            .fg(Color.RED)
+            .a("- locale: ")
+            .a(line.locale())
+            .a(", tmTextUnitId: ")
+            .a(line.tmTexUnitId())
+            .a(": ")
+            .a(line.error())
+            .println();
+      }
+      throw new CommandException(
+          "AI translate completed with %d error(s)".formatted(errorLines.size()));


RepositoryAiTranslationCommand.java L377-389

If OpenAI is down, every unit fails with the same message, which prints hundreds of nearly identical lines. Could we group by (locale, error) and list the affected tmTextUnitIds, or cap the output (for example, the first 20 lines plus "... and N more")? Please also put the pollable task id in the CommandException message, e.g. "AI translate completed with %d error(s) (task id: %d)", so the report is easy to find.

Without --download-report, output is now grouped by (locale, error) with the affected tmTextUnitIds on one line. The CommandException message also includes the pollable task id.

We're not capping line count here — timeouts are slow and these runs rarely have hundreds of units. Early abort when OpenAI is repeatedly unreachable (and a cooperative pollable cancel-request) needs server changes; filed as LXP-2810 and LXP-2811.

Print a compact error summary by default, keep full per-unit output with --download-report, document the non-zero exit, and expand tests. Server-side OpenAI abort/cancel is tracked separately (LXP-2810/LXP-2811). Co-authored-by: Cursor <cursoragent@cursor.com>

Co-authored-by: Cursor <cursoragent@cursor.com>

ehoogerbeets and others added 2 commits October 14, 2026 12:55

Fail repository-ai-translate when the non-batch report has errors.

de66e7c

Per-unit OpenAI failures were only stored in the import report, so the CLI printed Finished and exited 0 even when translations were skipped. Co-authored-by: Cursor <cursoragent@cursor.com>

Merge branch 'master' into fix/ai-translate-cli-report-errors

a551eac

kkuliczkowski-box reviewed Oct 19, 2026

View reviewed changes

ehoogerbeets and others added 2 commits October 21, 2026 14:03

Apply Spotless formatting to AI translate CLI report changes.

1a3aba1

Co-authored-by: Cursor <cursoragent@cursor.com>

kkuliczkowski-box approved these changes Oct 22, 2026

View reviewed changes

Merge branch 'master' into fix/ai-translate-cli-report-errors

843d8e0

ehoogerbeets merged commit 1146202 into box:master Oct 22, 2026
5 checks passed

ehoogerbeets deleted the fix/ai-translate-cli-report-errors branch October 22, 2026 22:14

Navigation Menu

Repository navigation

Fail repository-ai-translate when the non-batch report has errors - #1107

Fail repository-ai-translate when the non-batch report has errors#1107
ehoogerbeets merged 5 commits into
box:masterfrom
ehoogerbeets:fix/ai-translate-cli-report-errors

ehoogerbeets commented Oct 14, 2026 •

edited

Loading

Uh oh!

kkuliczkowski-box Oct 19, 2026

Uh oh!

ehoogerbeets Oct 21, 2026

Uh oh!

ehoogerbeets Oct 21, 2026

Uh oh!

kkuliczkowski-box Oct 19, 2026

Uh oh!

ehoogerbeets Oct 21, 2026

Uh oh!

kkuliczkowski-box Oct 19, 2026

Uh oh!

ehoogerbeets Oct 21, 2026

Uh oh!

kkuliczkowski-box Oct 19, 2026

Uh oh!

ehoogerbeets Oct 21, 2026

Uh oh!

Uh oh!

Reviewers

Uh oh!

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

Conversation

ehoogerbeets commented Oct 14, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Summary

Test plan

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Reviewers

Uh oh!

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

ehoogerbeets commented Oct 14, 2026 •

edited

Loading