Skip to content

compute-spec.md

Latest commit

 

History

History
104 lines (80 loc) · 1.86 KB

compute-spec.md

File metadata and controls

104 lines (80 loc) · 1.86 KB
Raw

Compute Specification Reference

compute.yml configures worker behavior for Floability runs.

Basic Example

vine_factory_config:
	min-workers: 1
	max-workers: 5
	cores: 2
	memory: 2048
	disk: 4000

Use it during run/execute:

floability run --backpack <backpack-root> --compute-spec <path-to-compute.yml>

Current Backend Support

Floability currently supports vine_factory as the worker backend. The compute spec section used by Floability is:

vine_factory_config:
	...

Supported Keys (vine_factory_config)

  • min-workers
  • max-workers
  • cores
  • disk
  • memory
  • foremen-name
  • workers-per-cycle
  • tasks-per-worker
  • timeout
  • worker-extra-options
  • condor-requirements

Example with more options:

vine_factory_config:
	min-workers: 1
	max-workers: 10
	cores: 1
	memory: 1024
	disk: 2000
	workers-per-cycle: 1
	tasks-per-worker: 1
	timeout: 0
	worker-extra-options: ""
	condor-requirements: ""

Precedence (Important)

Values are resolved in this order:

  1. Floability defaults
  2. Saved instance metadata
  3. compute.yml (vine_factory_config)
  4. CLI options

CLI options override values from compute.yml.

Examples:

# Override workers and cores from CLI
floability run --backpack <backpack-root> \
	--compute-spec <path-to-compute.yml> \
	--workers 20 \
	--cores-per-worker 2

# Override scheduler type and scheduler options
floability run --backpack <backpack-root> \
	--compute-spec <path-to-compute.yml> \
	--batch-type slurm \
	--batch-options "-p wide -t 02:00:00"

Related Options (CLI)

Common worker-related CLI flags:

  • --compute-spec
  • --batch-type local|condor|uge|slurm
  • --workers
  • --cores-per-worker
  • --batch-options
  • --debug-workers

Related Pages