Repository navigation
Description
Summary
File and column descriptions in dataset-metadata.json are accepted without error but never saved. This happens on datasets create, datasets version and datasets metadata --update. #447 and #248 were closed by #1055, but that PR changed only how the CLI builds the request. The server still drops the descriptions. Descriptions entered in the web editor are saved as expected.
The documentation says this should work:
- docs/datasets_metadata.md lists
resources(withpath,descriptionandschema.fields[].description) as supported forkaggle datasets createandkaggle datasets version. - The same page says
kaggle datasets metadata --update"supports all fields mentioned above forkaggle datasets version".
Environment
- kaggle 2.2.4, kagglesdk 0.1.37
- Python 3.14.3, Ubuntu (GitHub Actions
ubuntu-latest) - Authenticated with
KAGGLE_API_TOKEN
Steps to reproduce
-
Make a folder with
data.csv:year,category,grants 1977-78,Refugee,9326 1978-79,Refugee,12750 -
Add
dataset-metadata.jsonbeside it, following the documented format:{ "title": "Metadata description test", "id": "<your-username>/metadata-description-test", "licenses": [{"name": "CC0-1.0"}], "resources": [ { "path": "data.csv", "description": "Rows of the table, as CSV.", "schema": { "fields": [ {"name": "year", "description": "Financial year.", "type": "string"}, {"name": "category", "description": "Visa category.", "type": "string"}, {"name": "grants", "description": "Visas granted.", "type": "number"} ] } } ] } -
Create the dataset, wait until it is ready, then upload the same files as a new version:
kaggle datasets create -p . -u kaggle datasets status <your-username>/metadata-description-test # until "ready" kaggle datasets version -p . -m "second version with the same resources" -
Send the same file again as a metadata update:
kaggle datasets metadata <your-username>/metadata-description-test --update -p . -
Open the dataset page and select
data.csvin the Data Explorer.
Expected
"About this file" shows "Rows of the table, as CSV.", and each column shows its description. The usability rating ticks File Description and Column Description.
Actual
- "About this file" says "This file does not have a description yet", and no column has a description.
- The usability rating still crosses File Description and Column Description.
- Every command exits 0.
metadata --updateprints "successfully updated dataset metadata". - The other fields in the same update are saved: keywords,
expectedUpdateFrequency,userSpecifiedSourcesand the cover image.
What else we tried
All of these were accepted with an empty errors list, and none changed the page:
resourceswith the documented types (string,number,datetime), and withinteger.- Calling
DatasetApiService.UpdateDatasetMetadatathrough kagglesdk directly, withsettings.dataset toDatasetSettingsFileentries in four forms: description only; description andtotalBytes; those pluscolumns; and all three files of the dataset. - Three dataset versions uploaded with
resourcespresent (createonce, thenversiontwice).
GetDatasetMetadata returns info.data as an empty list after each update, as #410 also notes.
Example
A public dataset that shows the problem: https://www.kaggle.com/datasets/nationaldigitalau/au-humanitarian-program-grants. Version 3 was uploaded with resources for all three files and then sent the same settings through metadata --update.
Activity
Hisernberg commentedon Nov 10, 2026
I dug into this offline to check whether the client is really at fault, and on current
main(fdc788f, kagglesdk 0.1.38) the wire payload is correct.Method: simulated the real client flow with the blob upload mocked (no network): built a dataset folder with
dataset-metadata.jsonexactly as documented, ranKaggleApi.upload_files(...)fromkaggle/api/kaggle_api_extended.py, then serialized the resultingApiDatasetNewFilethe same way the SDK does for the request body.Result — the wire JSON carries everything:
{ "token": "<token>", "description": "Rows of the table, as CSV.", "columns": [ {"order": 0, "name": "year", "type": "string", "originalType": "string", "description": "Financial year."}, {"order": 1, "name": "category", "type": "string", "originalType": "string", "description": "Visa category."}, {"order": 2, "name": "grants", "type": "numeric", "originalType": "number", "description": "Visas granted."} ] }The chain is wired end-to-end on the client:
dataset-metadata.jsonresources →_upload_file(matchesfile_name == item["path"], populatesUploadFile.descriptionandprocess_column()output) →_new_file(ApiDatasetNewFile.description+columns) → kagglesdkApiDatasetNewFile._fieldsincludesdescription, andApiDatasetColumn._fieldsincludesdescription(all verified in kagglesdk 0.1.38).So on the client side there is nothing left to fix for
datasets create/version— this isolates the drop to the server's handling of thefiles[].description/files[].columns[].descriptionfields onPOST /api/v1/datasets/create/version(and the equivalent create path), consistent with the report.kaggle datasets metadata --updategoes through a different endpoint (POST /datasets/metadata/...withDatasetSettings/DatasetSettingsFile), whose wire mapping I also verified as complete — so if descriptions are dropped there too, both drop sites are server-side.Happy to help test once the server-side fix lands.