Skip to content

update etl configuration input to yml instead of environment variables#1795

Open
adamkorynta wants to merge 5 commits into
developfrom
feature/etl_follow_up
Open

update etl configuration input to yml instead of environment variables#1795
adamkorynta wants to merge 5 commits into
developfrom
feature/etl_follow_up

Conversation

@adamkorynta

Copy link
Copy Markdown
Collaborator

Summary

  • file system staging update for projects, locations, and timeseries
  • add in REGI files to VCS
  • add in docker environment for running python tests
  • move etl calls to parent docker-compose file
  • let the source URL be optional, if not set only use staged filesystem resources

Related Issue

Validation

Unit tests and docker compose in local dev environment

Checklist

  • AI tools used - GH Copilot for README.md generation, logger messages, yml parsing, and code review

- file system staging update for projects, locations, and timeseries
- add in REGI files to VCS
- add in docker environment for running python tests
- move etl calls to parent docker-compose file
- let the source URL be optional, if not set only use staged filesystem resources
@adamkorynta adamkorynta requested a review from krowvin June 22, 2026 17:56
@adamkorynta

Copy link
Copy Markdown
Collaborator Author

@dezidizon please review revisions to @RyanM-RMA's initial implementation

@adamkorynta adamkorynta changed the title update configuration input to yml instead of environment variables update etl configuration input to yml instead of environment variables Jun 22, 2026

@dezidizon dezidizon left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Performed code review and only found minor changes to capitalize several print out statements. This code is ready for merging even without implementing the requested updates.

Comment thread cda-etl/src/cda_etl/config.py Outdated
Comment thread cda-etl/src/cda_etl/config.py Outdated
Comment thread cda-etl/src/cda_etl/config.py Outdated
Comment thread cda-etl/src/cda_etl/config.py Outdated
Comment thread cda-etl/src/cda_etl/config.py Outdated
adamkorynta and others added 2 commits June 23, 2026 12:44
Co-authored-by: dezidizon <47678178+dezidizon@users.noreply.github.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants