feat(ml): physics metrics, Optuna runner, Faz 3 evaluation

[edatosun]

1. feat(ml): physics metrics, Optuna runner, Faz 3 evaluation

   • Physics metrics (flux, ring diameter, asymmetry)
   • Optuna hyperparameter search (TPE, 20 trials)
   • Faz 3 evaluation script + auto ADR generation

2. fix(ml): U-Net config compatibility + ESRGAN test params

   • U-Net now works with Hydra config
   • ESRGAN test parameter ranges updated to real values
   • Pix2Pix uses generic forward pass

[Copilot]

Pull request overview

This PR extends the ML evaluation and experimentation tooling by adding physics-informed image metrics, introducing an Optuna-based hyperparameter search runner for Pix2Pix training, and providing a Phase 3 evaluation script that enforces the SSIM go/no-go gate and writes an ADR. It also adds Kubernetes manifests to support ML inference deployment and training-related infra resources.

Changes:

• Add physics-informed evaluation metrics (flux conservation, ring diameter, asymmetry) and reformat metrics.py.
• Add an Optuna runner to search Pix2Pix hyperparameters and log results to MLflow.
• Add a Phase 3 evaluation CLI script (metrics + SSIM gate + ADR writer), plus new K8s manifests for inference/training support.

Reviewed changes

Copilot reviewed 7 out of 7 changed files in this pull request and generated 5 comments.

Show a summary per file

File	Description
services/ml/evaluation/metrics.py	Adds/organizes evaluation metrics including physics-informed metrics; minor formatting updates.
services/ml/training/optuna_runner.py	New Optuna-based hyperparameter search runner for Pix2Pix GAN training with MLflow logging.
scripts/eval_phase3.py	New Phase 3 evaluation script computing metrics, deciding GO/NO-GO, writing ADR, and setting exit code.
infra/k8s/ml/training-image-build-job.yaml	New Kaniko Job manifest for building a training image in-cluster.
infra/k8s/ml/pvcs.yaml	New PVC manifest for training outputs storage.
infra/k8s/ml/priority-class.yaml	New PriorityClass definitions to prioritize inference over training scheduling.
infra/k8s/ml/inference.yaml	New inference Deployment/Service + models PVC for serving.

Suppressed comments (4)

services/ml/training/optuna_runner.py:130

• train_gan is decorated with @hydra.main, so calling it with an explicit DictConfig (train_gan(trial_cfg)) will invoke Hydra’s wrapper and can raise a TypeError / ignore the passed config. Use the wrapped function when available so Optuna can pass the composed config directly.

    try:
        train_gan(trial_cfg)
        val_ssim = _extract_val_ssim(Path(trial_cfg.paths.output_dir))

services/ml/training/optuna_runner.py:178

• run_optuna_search() keeps an MLflow run active while study.optimize(...) runs, but train_gan starts its own MLflow run (non-nested). This will raise an exception like “Run with UUID ... is already active” when the first trial calls train_gan. The parent Optuna summary run needs to be ended before trials start (or train_gan would need to start nested runs), then resumed afterward to log the best params/metrics.

    # Parent MLflow run
    mlflow.set_tracking_uri(base_cfg.mlflow.tracking_uri)
    mlflow.set_experiment(f"{base_cfg.mlflow.experiment_name}-optuna")

    with mlflow.start_run(run_name=f"optuna-parent-{study_name}"):

services/ml/training/optuna_runner.py:140

• After train_gan, this starts a separate MLflow run per trial and logs params/metrics again. Besides duplicating what train_gan already logs, it will also interact poorly with any parent/active MLflow run management (nested vs non-nested). Prefer relying on train_gan’s MLflow logging and keep Optuna’s objective side-effect free.

    # MLflow'a trial parametrelerini log'la
    with mlflow.start_run(nested=True, run_name=f"optuna-trial-{trial.number}"):
        mlflow.log_params({k: v for k, v in params.items() if not k.startswith("_")})
        mlflow.log_metric("val_ssim", val_ssim)
        mlflow.log_metric("trial_number", trial.number)

scripts/eval_phase3.py:276

• --batch-size is parsed but currently not used anywhere. A low-impact way to make it effective is to pass it into compute_fid(..., batch_size=...) (this controls Inception feature extraction batch size).

    print("[eval_phase3] FID hesaplanıyor...")
    real_all = torch.cat(real_features_list, dim=0)
    fake_all = torch.cat(fake_features_list, dim=0)
    fid_value = compute_fid(real_all, fake_all)

---

💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.