Skip to content
Draft
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
26 changes: 2 additions & 24 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -57,7 +57,7 @@ These concerns are intentionally left to downstream tooling.
## Example (concept navigation)

```python
from omop_alchemy.model.vocabulary import ConceptView
from omop_alchemy.cdm.model.vocabulary.concept import ConceptView

concept = session.get(ConceptView, 320128) # Lung cancer
concept.domain.domain_id # "Condition"
Expand Down Expand Up @@ -93,26 +93,4 @@ omop-config init
omop-config configure omop_alchemy
```

See [Configuration](docs/getting-started/configuration.md) for full details.

---

## Docker Compose

The included `docker-compose.yaml` provides a PostgreSQL database and a Python
container with the `[postgres]` extra pre-installed. Default credentials work out of the box:

```bash
docker compose up
```

The `python-alchemy` service runs `omop-config configure` at startup and writes
`~/.config/omop/config.toml` on the host on first start; subsequent starts skip
configuration automatically.

To override credentials, copy `.env.example` to `.env` and edit before starting:

```bash
cp .env.example .env
docker compose up
```
See [Configuration](docs/getting-started/configuration.md) for full details.
66 changes: 0 additions & 66 deletions docker-compose.yaml

This file was deleted.

8 changes: 4 additions & 4 deletions docs/advanced/vocabulary_load_performance.md
Original file line number Diff line number Diff line change
Expand Up @@ -37,9 +37,9 @@ The next biggest bottleneck after pagination is `synchronous_commit=on` (the Pos

### Recommended settings

These settings are present in the docker-compose files for each package. If you are running PostgreSQL outside Docker, add them to `postgresql.conf` or pass them as `-c` flags.
Apply these via `postgresql.conf` or `-c` flags on whatever PostgreSQL instance you're loading into (per-package `docker-compose.yaml` files no longer exist; Docker orchestration for the OMOP stack now happens at the workspace root).

**devcontainer (omop-spires `docker-compose.override.yaml`) — 8 GB host:**
**8 GB host:**
```
synchronous_commit=off
checkpoint_timeout=30min
Expand All @@ -53,7 +53,7 @@ wal_compression=zstd
full_page_writes=off
```

**standalone docker-compose (OMOP_Alchemy / omop-graph) — ~4 GB host:**
**~4 GB host:**
```
synchronous_commit=off
checkpoint_timeout=30min
Expand Down Expand Up @@ -81,7 +81,7 @@ SELECT pg_reload_conf();
SHOW synchronous_commit; -- confirm: should show 'off'
```

Settings that ARE overridden by `-c` (e.g. `checkpoint_timeout`) require a container restart to pick up the updated docker-compose value.
Settings that ARE overridden by `-c` (e.g. `checkpoint_timeout`) require a container restart to pick up an updated value.

To monitor whether WAL-write stalls are happening during a load:

Expand Down
29 changes: 13 additions & 16 deletions docs/cli/index.md
Original file line number Diff line number Diff line change
Expand Up @@ -59,29 +59,26 @@ No connection flags are injected; all configuration comes from oa_configurator.

When a decorated command is invoked:

1. Loads `~/.config/omop/config.toml` via `load_stack_config()`.
2. Calls `OmopAlchemyConfig.from_stack(config)` to read package-specific settings and validate that the required `cdm_db` resource (or the `[tools.omop_alchemy] default_resource` override) is present. Raises `ConfigurationError` with a helpful message if it is missing.
3. Resolves the resource: `Resolver(config).resolve_resource("cdm_db")`.
4. Calls `.create_engine()` to build a SQLAlchemy engine with `schema_translate_map` applied.
5. Prints a command header showing the resource name, CDM schema, and run mode.
6. Calls the original function body with `(conn, engine, ...)`.
7. Catches `RuntimeError`, `SQLAlchemyError`, and `BackendNotSupportedError`; renders them as formatted errors and exits with code 1.
1. Calls `get_cdm_context()`, which loads `~/.config/omop/config.toml` (via `load_stack_config()`) and resolves whatever `OmopAlchemyConfig.cdm_db` currently names, returning `(pkg_config, resolved)`. Raises `RuntimeError` with a helpful message if no config file exists yet.
2. Calls `create_cdm_engine(resolved)` to build a SQLAlchemy engine (`resolved.create_engine()`, with `schema_translate_map` applied), with a clearer error if the PostgreSQL driver isn't installed.
3. Builds `conn` (`db_schema=resolved.schema_name`, `athena_source=pkg_config.athena_source_path`).
4. Prints a command header showing the connection, CDM schema, and run mode.
5. Calls the original function body with `(conn, engine, ...)`.
6. Catches `RuntimeError`, `SQLAlchemyError`, and `BackendNotSupportedError`; renders them as formatted errors and exits with code 1.

### Before and after

Without the decorator, every command would need this boilerplate:

```python
from omop_alchemy.config import TOOL_NAME
from omop_alchemy.config import create_cdm_engine, get_cdm_context

def my_command() -> None:
stack = load_stack_config()
tool = stack.tools.get(TOOL_NAME)
resource_name = (tool.default_resource if tool else None) or "cdm_db"
resolved = Resolver(stack).resolve_resource(resource_name)
engine = resolved.create_engine()
pkg_config, resolved = get_cdm_context()
engine = create_cdm_engine(resolved)
try:
# actual work here
results = do_work(engine, db_schema=resolved.cdm_schema)
results = do_work(engine, db_schema=resolved.schema_name)
console.print(render_results(results))
except Exception as exc:
handle_error(exc)
Expand All @@ -105,5 +102,5 @@ def my_command(conn, engine) -> None:

| Attribute | Description |
|---|---|
| `conn.db_schema` | CDM schema name from the resolved resource (e.g. `"omop"`) |
| `conn.athena_source` | Athena vocabulary CSV directory from `[tools.omop_alchemy.extra]`; `None` if not configured |
| `conn.db_schema` | CDM schema name from the resolved database (e.g. `"omop"`) |
| `conn.athena_source` | Athena vocabulary CSV directory from `[tools.omop_alchemy]`'s `athena_source_path` field; `None` if not configured |
84 changes: 24 additions & 60 deletions docs/getting-started/configuration.md
Original file line number Diff line number Diff line change
Expand Up @@ -13,36 +13,40 @@ Run the interactive configure command to set up the CDM database connection and
omop-config configure omop_alchemy
```

This prompts for connection details (host, dialect, credentials) and schema names, then
saves them under the canonical resource name `cdm_db` that all OMOP stack packages
This prompts for connection details (host, dialect, credentials) and schema name, then
saves them under the canonical database name `cdm_db` that all OMOP stack packages
recognise.

The resulting TOML looks like:

```toml
[databases.cdm]
dialect = "postgresql+psycopg2"
[connections.cdm]
dialect = "postgresql+psycopg"
host = "localhost"
port = 5432
user = "omop"
password = "changeme"
database_name = "omop_cdm"

[resources.cdm_db]
database = "cdm"
cdm_schema = "omop"
[databases.cdm_db]
kind = "cdm"
connection = "cdm"
schema_name = "omop"

[tools.omop_alchemy]
cdm_db = "cdm_db"
```

You can also write or edit this file manually.

## Vocabulary loading

If you plan to load OMOP vocabulary from Athena CSV files, add the path to the package
extras section:
If you plan to load OMOP vocabulary from Athena CSV files, add the path to `[tools.omop_alchemy]`:

```toml
[tools.omop_alchemy.extra]
athena_source_path = "/path/to/athena/csvs"
[tools.omop_alchemy]
cdm_db = "cdm_db"
athena_source_path = "/path/to/athena/csvs"
```

Or set it interactively:
Expand All @@ -60,63 +64,23 @@ omop-alchemy info
This prints the resolved config file path, connection details, and schema. A successful
run confirms that OMOP_Alchemy can reach your database.

## Docker Compose

The included `docker-compose.yaml` spins up a PostgreSQL database and a `python-alchemy`
container. Default credentials work out of the box — no additional setup needed:

```bash
docker compose up
```

The `python-alchemy` container runs `omop-config configure omop_alchemy` automatically at
startup. Your `~/.config/omop/config.toml` on the host is written on first start and
safe to re-run on subsequent starts: connection flags always apply, and any values already stored in `config.toml` are preserved for fields not explicitly provided.

### Overriding default values

The compose file uses built-in defaults for all database credentials. To use different
values, create a `.env` file in this directory with any of the following variables:

| Variable | Default | Description |
|---|---|---|
| `OMOP_CDM_DB_USER` | `omop` | CDM database username |
| `OMOP_CDM_DB_PASSWORD` | `omop` | CDM database password |
| `OMOP_CDM_DB_NAME` | `omop_cdm` | CDM database name |

Copy the example and edit as needed:

```bash
cp .env.example .env
# edit .env
docker compose up
```

The `.env` file is only read by Docker Compose for variable substitution — it is not
loaded by OMOP_Alchemy at runtime.

## Multiple instances

To configure a second CDM database (e.g. for production), use `--resource-name`:
To configure a second CDM database (e.g. for production), create it under its own name
and point the field's own flag at it:

```bash
omop-config configure omop_alchemy --resource-name cdm_db_prod
omop-config databases add cdm_db_prod --kind cdm --connection cdm_prod
omop-config configure omop_alchemy --cdm-db cdm_db_prod
```

This creates `cdm_db_prod` without touching the existing `cdm_db`. Because two
resources now exist, configure automatically prompts you to choose the default at
the end of the same run — no second invocation needed.

To change the default later, set `default_resource` directly in `config.toml`:

```toml
[tools.omop_alchemy]
default_resource = "cdm_db_prod"
```
This creates `cdm_db_prod` without touching the existing `cdm_db`. There is no "default"
toggle to flip afterward; each deployment's `configure` call names the entry it wants
directly.

See the [oa-configurator integration guide](https://AustralianCancerDataNetwork.github.io/oa-configurator/integration/#multiple-environments) for the full multi-environment guide.

## Further reading

- [oa_configurator quickstart](https://AustralianCancerDataNetwork.github.io/oa-configurator/) — full config reference, multiple profiles, env var export
- [oa_configurator integration guide](https://AustralianCancerDataNetwork.github.io/oa-configurator/integration/) — Docker Compose details and multi-package setups
- [oa_configurator quickstart](https://AustralianCancerDataNetwork.github.io/oa-configurator/quickstart/): full config reference, CLI walkthrough
- [oa_configurator integration guide](https://AustralianCancerDataNetwork.github.io/oa-configurator/integration/): multi-package setups
2 changes: 1 addition & 1 deletion docs/getting-started/installation.md
Original file line number Diff line number Diff line change
Expand Up @@ -17,7 +17,7 @@ engine = sa.create_engine(
echo=False,
)

with so.Session(engine) as sess:
with so.Session(engine) as session:
concepts = (
session.query(Concept)
.filter(Concept.domain_id == "Drug")
Expand Down
Loading