fix: loaded dataset is not saved to the Modal cache volume

[andrewwhitecdw]

This PR addresses the following issue in fouroversix/scripts/train/prepare_dataset.py: loaded dataset is not saved to the Modal cache volume.

Changes

• fouroversix/scripts/train/prepare_dataset.py: loaded dataset is not saved to the Modal cache volume.

Details

--- a/fouroversix/scripts/train/prepare_dataset.py
+++ b/fouroversix/scripts/train/prepare_dataset.py
@@ -1,3 +1,5 @@
+import os
+
 from ..resources import FOUROVERSIX_CACHE_PATH, app, cache_volume, get_image
 
 
@@ -9,7 +11,9 @@
     volumes={FOUROVERSIX_CACHE_PATH: cache_volume},
 )
 def prepare_dataset(path: str, name: str) -> None:
-    load_dataset(path, name)
+    dataset = load_dataset(path, name)
+    dataset.save_to_disk(os.path.join(FOUROVERSIX_CACHE_PATH, name))

Tests

Let me know if you want tests added for this fix or not.