Add API to upload translated files as suggestions

pontoon/api/README.md

@@ -6,7 +6,7 @@ Pontoon provides a set of [RESTful](https://developer.mozilla.org/en-US/docs/Glo
 
 Most endpoints are publicly accessible and require no authentication. A few endpoints require an authenticated user.
 
-Requests can be authenticated either with a session cookie or with a Personal Access Token (PAT). Write endpoints accept only a PAT. You can create a PAT from your [user settings](https://pontoon.mozilla.org/settings/) page (see the [User Accounts & Settings](https://github.com/mozilla/pontoon/blob/main/documentation/docs/localizer/users.md#personal-access-tokens) documentation for details).
+Requests can be authenticated either with a session cookie or with a Personal Access Token (PAT). Session requests that write data are subject to Django's CSRF checks. The upload endpoints accept both; `POST /api/v2/pretranslate/`, which returns a machine pretranslation for a string, accepts only a PAT. You can create a PAT from your [user settings](https://pontoon.mozilla.org/settings/) page (see the [User Accounts & Settings](https://github.com/mozilla/pontoon/blob/main/documentation/docs/localizer/users.md#personal-access-tokens) documentation for details).
 
 Send the token in the `Authorization` header using the `Bearer` scheme:
 
@@ -75,8 +75,8 @@ $ curl --globoff "https://example.com/api/v2/locales/?page_size=50"
 
 ## Write Endpoints
 
-The following endpoints can write data and always require authentication with a Personal
-Access Token. Session cookies are not accepted.
+The following endpoints can write data and always require authentication, with a Personal
+Access Token or with a session cookie and CSRF token.
 
 ### `POST /api/v2/upload/translations/`
 
@@ -109,7 +109,8 @@ A successful request returns a summary of the import:
   "updated": 12,
   "unchanged": 3,
   "undefined_keys": [["obsolete_key"]],
-  "undefined_keys_count": 1
+  "undefined_keys_count": 1,
+  "badge_updates": [{ "name": "Translation Champion", "level": 2 }]
 }
 ```
 
@@ -122,6 +123,8 @@ A successful request returns a summary of the import:
   Only the first 100 keys are listed.
 - `undefined_keys_count`: total number of keys with no matching string in Pontoon,
   before truncation.
+- `badge_updates`: badges whose level the upload raised, each with its `name` and new
+  `level`. The user is also notified of each. Empty when no level changed.
 
 The upload is additive: strings missing from the uploaded file are left untouched, so
 partial files can be used to update a subset of translations. Re-uploading an unchanged
@@ -154,7 +157,7 @@ Status codes:
 | ----- | ---------------------------------------------------------------------------------------------------------- |
 | `200` | Upload accepted (possibly with `"updated": 0`)                                                             |
 | `400` | Missing or invalid field, unsupported format, unparseable or empty file, or file too large                 |
-| `403` | Missing token, invalid or expired token, or insufficient permission                                        |
+| `403` | Not authenticated, invalid or expired token, missing CSRF token, or insufficient permission                |
 | `404` | Unknown or disabled project, unknown locale or resource, or project or resource not enabled for the locale |
 | `409` | A concurrent upload or review changed the same translations; retry the request                             |
 | `429` | Rate limit exceeded                                                                                        |
@@ -192,7 +195,8 @@ A successful request returns a summary of the import:
   ],
   "failed_checks_count": 1,
   "undefined_keys": [["obsolete_key"]],
-  "undefined_keys_count": 1
+  "undefined_keys_count": 1,
+  "badge_updates": []
 }
 ```
 
@@ -217,6 +221,8 @@ A successful request returns a summary of the import:
   Only the first 100 keys are listed.
 - `undefined_keys_count`: total number of keys with no matching string in Pontoon,
   before truncation.
+- `badge_updates`: badges whose level the upload raised, each with its `name` and new
+  `level`. The user is also notified of each. Empty when no level changed.
 
 Unlike the built-in pretranslation, strings with unreviewed suggestions are
 pretranslated. Suggestions that don't match the uploaded translation are kept as
@@ -230,3 +236,81 @@ The requirements, limits and status codes of
 [`POST /api/v2/upload/translations/`](#post-apiv2uploadtranslations) also apply here,
 with one difference: the request is rejected with `403` unless the user is a member of
 the `pretranslators` group.
+
+### `POST /api/v2/upload/suggestions/`
+
+Store translations from an uploaded translation file as unreviewed suggestions,
+authored by the authenticated user.
+
+The request body is the same `multipart/form-data` as
+[`POST /api/v2/upload/translations/`](#post-apiv2uploadtranslations):
+
+```bash
+$ curl -X POST \
+  -H "Authorization: Bearer <YOUR-TOKEN>" \
+  -F "project=firefox" \
+  -F "locale=it" \
+  -F "resource=browser/browser.ftl" \
+  -F "uploadfile=@browser.ftl" \
+  "https://example.com/api/v2/upload/suggestions/"
+```
+
+A successful request returns a summary of the import:
+
+```json
+{
+  "created": 9,
+  "restored": 1,
+  "unchanged": 4,
+  "failed_checks": [
+    {
+      "key": ["entity_key"],
+      "errors": ["Ending newline mismatch"],
+      "warnings": []
+    }
+  ],
+  "failed_checks_count": 1,
+  "undefined_keys": [["obsolete_key"]],
+  "undefined_keys_count": 1,
+  "badge_updates": []
+}
+```
+
+- `created`: number of suggestions added.
+- `restored`: number of rejected translations matching the upload that were
+  un-rejected, becoming pending suggestions again.
+- `unchanged`: number of uploaded translations that match existing unrejected
+  translations, in any review state, ignored.
+- `failed_checks`: strings left untouched, because the uploaded translation has errors.
+  Each entry has the `key` of the string, in the same format as the `key` field of
+  entities, and the `errors` and `warnings` reported for it. Only the first 100 keys are
+  listed.
+- `failed_checks_count`: total number of strings left untouched because of errors,
+  before truncation.
+- `undefined_keys`: keys of translations with no matching string in Pontoon, ignored.
+  Each key is a list of strings, in the same format as the `key` field of entities.
+  Only the first 100 keys are listed.
+- `undefined_keys_count`: total number of keys with no matching string in Pontoon,
+  before truncation.
+- `badge_updates`: badges whose level the upload raised, each with its `name` and new
+  `level`. The user is also notified of each. Empty when no level changed.
+
+Nothing already in Pontoon is replaced or rejected: every uploaded translation is stored
+as a suggestion, unless the string already has an unrejected translation with the same
+value, whether approved, pretranslated or unreviewed. The fuzzy flag of the uploaded
+file is ignored: the translation is stored as a plain suggestion.
+
+A rejected translation matching the upload is un-rejected instead of being suggested
+again, so a translation rejected by mistake can be re-proposed. Its original author and
+date are kept, and it becomes a pending suggestion, awaiting review like any other.
+
+Uploaded translations reported with errors are left out, as the editor rejects them as
+well. Translations with warnings are stored, with their warnings, as a reviewer can
+still accept them.
+
+NOTE: unlike in the UI, where any user can submit suggestions, this endpoint
+requires translator rights. This is done to prevent abuse, since a malicious actor
+could submit thousands of suggestions.
+
+The requirements, limits and status codes of
+[`POST /api/v2/upload/translations/`](#post-apiv2uploadtranslations) also apply here.

pontoon/api/serializers.py

@@ -419,6 +419,39 @@ def get_translation(self, obj):
 UPLOAD_KEYS_ERROR_LIMIT = 100
 
 
+def undefined_keys_field() -> serializers.ListField:
+    """Upload response field listing the keys that match no entity in Pontoon."""
+    return serializers.ListField(
+        child=serializers.ListField(child=serializers.CharField()),
+        help_text=f"Keys of translations with no matching entity in Pontoon, ignored. "
+        f"Truncated to the first {UPLOAD_KEYS_ERROR_LIMIT} keys.",
+    )
+
+
+def undefined_keys_count_field() -> serializers.IntegerField:
+    """Upload response field counting the keys that match no entity in Pontoon."""
+    return serializers.IntegerField(
+        help_text="Total number of keys with no matching entity in Pontoon, "
+        "before truncation."
+    )
+
+
+class BadgeUpdateSerializer(serializers.Serializer):
+    """A badge level the user reached through the upload."""
+
+    name = serializers.CharField(help_text="Name of the badge.")
+    level = serializers.IntegerField(help_text="Level reached.")
+
+
+def badge_updates_field() -> BadgeUpdateSerializer:
+    """Upload response field listing the badge levels the user reached."""
+    return BadgeUpdateSerializer(
+        many=True,
+        help_text="Badges whose level the upload raised, with the new level. "
+        "The user is also notified of each.",
+    )
+
+
 class UploadTranslationsResponseSerializer(serializers.Serializer):
     """Result of a translation file upload."""
 
@@ -428,15 +461,9 @@ class UploadTranslationsResponseSerializer(serializers.Serializer):
     unchanged = serializers.IntegerField(
         help_text="Number of translations identical to the current ones, ignored."
     )
-    undefined_keys = serializers.ListField(
-        child=serializers.ListField(child=serializers.CharField()),
-        help_text=f"Keys of translations with no matching entity in Pontoon, ignored. "
-        f"Truncated to the first {UPLOAD_KEYS_ERROR_LIMIT} keys.",
-    )
-    undefined_keys_count = serializers.IntegerField(
-        help_text="Total number of keys with no matching entity in Pontoon, "
-        "before truncation."
-    )
+    undefined_keys = undefined_keys_field()
+    undefined_keys_count = undefined_keys_count_field()
+    badge_updates = badge_updates_field()
 
 
 class FailedCheckSerializer(serializers.Serializer):
@@ -489,12 +516,34 @@ class UploadPretranslationsResponseSerializer(serializers.Serializer):
         help_text="Total number of strings left untouched because of failing checks, "
         "before truncation."
     )
-    undefined_keys = serializers.ListField(
-        child=serializers.ListField(child=serializers.CharField()),
-        help_text=f"Keys of translations with no matching entity in Pontoon, ignored. "
-        f"Truncated to the first {UPLOAD_KEYS_ERROR_LIMIT} keys.",
+    undefined_keys = undefined_keys_field()
+    undefined_keys_count = undefined_keys_count_field()
+    badge_updates = badge_updates_field()
+
+
+class UploadSuggestionsResponseSerializer(serializers.Serializer):
+    """Result of a suggestion file upload."""
+
+    created = serializers.IntegerField(
+        help_text="Number of suggestions added by the upload."
     )
-    undefined_keys_count = serializers.IntegerField(
-        help_text="Total number of keys with no matching entity in Pontoon, "
+    restored = serializers.IntegerField(
+        help_text="Number of rejected translations matching the upload that were "
+        "un-rejected, becoming pending suggestions again."
+    )
+    unchanged = serializers.IntegerField(
+        help_text="Number of uploaded translations that the string already has as an "
+        "unrejected translation, in any review state, ignored."
+    )
+    failed_checks = FailedCheckSerializer(
+        many=True,
+        help_text="Strings left untouched, because the uploaded translation has "
+        f"errors. Truncated to the first {UPLOAD_KEYS_ERROR_LIMIT} keys.",
+    )
+    failed_checks_count = serializers.IntegerField(
+        help_text="Total number of strings left untouched because of errors, "
         "before truncation."
     )
+    undefined_keys = undefined_keys_field()
+    undefined_keys_count = undefined_keys_count_field()
+    badge_updates = badge_updates_field()