In this repo, there's a problem with macOS CI jobs. They tend to hang and not finish, and when they do, they leave behind datasets in dask-bigquery-testing project.
This is not urgent, but we need to come up with a plan of how to either avoid this problem, or automate the cleanup.
In BigQuery, it is possible to set expiration for tables within a dataset:
https://cloud.google.com/bigquery/docs/updating-datasets#table-expiration
There doesn't seem to be a similar feature for the dataset. Deleting tables would only solve the problem partially, since each CI job also creates a new temporary dataset.
cc @ncclementi @jrbourbeau
In this repo, there's a problem with macOS CI jobs. They tend to hang and not finish, and when they do, they leave behind datasets in
dask-bigquery-testingproject.This is not urgent, but we need to come up with a plan of how to either avoid this problem, or automate the cleanup.
In BigQuery, it is possible to set expiration for tables within a dataset:
https://cloud.google.com/bigquery/docs/updating-datasets#table-expiration
There doesn't seem to be a similar feature for the dataset. Deleting tables would only solve the problem partially, since each CI job also creates a new temporary dataset.
Job is about to start running on the hosted runneractions/runner#2609.cc @ncclementi @jrbourbeau