Issue 57: 7.3.3 Flows - #60

Merged

GBirkel merged 48 commits into

als-computing:mainfrom

davramov:issue_57

Jan 5, 2026

Commits on Dec 17, 2025

Init commit for 7.3.3 workflows. Created a basic move.py script that uses the transfer_controller class. Created a Config733() class and updated config.yml with a few endpoints, anticipating a glob…
davramov
committed
5b455c5
Fixed comment formatting in pytest
davramov
committed
2d7ff3a
Updating documentation
davramov
committed
de3e974
Updating documentation and added a dispatcher for bl733
davramov
committed
8eedaca
Continuing to develop the 733 flows. Updated docs, added comments indicating next steps, included a create_deployments_733.sh script.
davramov
committed
747c412
Updating documentation
davramov
committed
ec968aa
Adding globus prune_controller logic from PR#59 to move.py for now. Once that PR is merged, let's use the prune_controller
davramov
committed
3fb6361
Removing 'self' as input parameter to prune() since it is not part of a class for now.
davramov
committed
8c2e38b
Adding a mocker.patch for schedule_prefect_flow in test_globus_flow.py to avoid real calls to Prefect in the pytest
davramov
committed
c195c60
Updating prefect deployments... although I should make sure this works with Prefect3.x 🤔
davramov
committed
ddbecbf
Created a new file 'create_deployments_733_3.4.2.sh' for deploying Prefect flows following the 3.4.2 best practices. Created a new config 'orchestration/flows/bl733/prefect.yaml' that defines the f…
davramov
committed
c86522f
Making sure the correct function is registered for pruning. Making sure that the flow run names match the file names that are being processed.
davramov
committed
fbecede
Adding file name to the dispatcher flow_run_name
davramov
committed
707409c
Adding UUID for ALS Transfer 733 endpoint 💾
davramov
committed
d85c163
fixing work_pool yaml entries in orchestration/flows/bl733/prefect.yaml
davramov
committed
b60869f
Fixing flow run name formatting
davramov
committed
1899983
Adding BEAMLINE env variable to login script
davramov
committed
d5f1a13
adjusting main method file_path to test
davramov
committed
fcaae9d
Commenting out the delete block to ensure no data is accidentally deleted during testing
davramov
committed
1897e33
Adjusting the instructions in the docsctring for the correct usage of this script
davramov
committed
27a06cf
Adding the init_work_pools.sh script from the prefect 3 PR to help with deployment
davramov
committed
124354f
Commenting out Prefect JSON block stuff, since it is becoming deprecated with Prefect 3. Hard coding the max_wait_seconds=600 variable for now.
davramov
committed
9a8f641
Adjusting docstring to indicate days_from_now expects a float, not a datetime. Also adding a check to ensure days_from_now is not negative
davramov
committed
8e9ee98
Rewriting the init_work_pools script to be in python rather than shell, for improved maintainability.
davramov
committed
70c4fcb
Moving old flow registering scripts to scripts/legacy/
davramov
committed
7fc4f13
Moving old flow registering scripts to scripts/legacy/
davramov
committed
683a315
Updating init_work_pools.py to set the GLOBUS_CLIENT_ID and GLOBUS_CLIENT_SECRET in Prefect Secret blocks if set as environment variables
davramov
committed
454f34e
Adding new_file_733_flight_check flow (test Globus transfer from data733 -> NERSC with a test.txt file), and scheduling it to run every 12 hours
davramov
committed
398bfe9
Adding a todo item for updating the config type checking once PR#62 is merged
davramov
committed
09b2341
updating documentation
davramov
committed
3749ad4
pytest patches for prefect 3
davramov
committed
b6fb8d5
Updating diagram to fit vertically instead of horizontally
davramov
committed
ad84b40
Updating diagram to fit vertically instead of horizontally
davramov
committed
a98e139
fixing cut off text in diagram
davramov
committed
4b1c05c
Updating globus endpoint names in config.yaml to use beamline name (bl733) as a prefix as per Garrett's suggestion
davramov
committed
a3b8300
removing bl733 from the test_globus_flow.py pytest
davramov
committed
4a14e3c
moving pytests to a specific test folder for bl733
davramov
committed
51ed315
making the move flow call the move task, making dispatcher call the move task, and updating prefect.yaml to reflect that
davramov
committed
f361d4d
Updating dockerfile to pull prefect 3.4.2
davramov
committed
abeba68
Making sure logger = get_run_logger() within the functions for better prefect visibility. Also, adding RuntimeError to the move_733_flight_check to ensure the flow fails if the transfer fails
davramov
committed
66cad5d
Adding flow_run_name to schedule_prefect_flow call in the prune method
davramov
committed
67c033e
Adding Lamarr (ALS Computing Beamlines/Global) as a transfer endpoint for 733, and a transfer call in move.py. Updating pytest to reflect the additional transfer. Still waiting for credentials on L…
davramov
committed
44fbe4c
Removing legacy create_deployments scripts
davramov
committed
9002ba0
Fixing config.yml after rebase
davramov
committed
75b0e14
removing old move_733.py script
davramov
committed
f1b8fdb
using Variable Blocks for settings
davramov
committed
479e518
removing delete_spot733 option for bl733-settings prefect variable (not part of this workflow)
davramov
committed
84fa884
Loading max_wait_settings from Prefect Variable block
davramov
committed
5b27406