diff --git a/.pre-commit-config.yaml b/.pre-commit-config.yaml index f35772ce..a03d070f 100644 --- a/.pre-commit-config.yaml +++ b/.pre-commit-config.yaml @@ -2,6 +2,12 @@ exclude: '^.*\.(svg|txt|json|html)$|build|dist|benchmarks\/train_crnn_cinc2023\/utils\/artifact_pipeline\/.*' fail_fast: false +# the isort hook (10.0.0b1) requires Python >= 3.11, which matches the +# package's own floor (requires-python >= 3.11); the plain `python3` is +# >= 3.11 both locally and on the check-formatting runner +default_language_version: + python: python3 + repos: - repo: https://github.com/pre-commit/pre-commit-hooks rev: v6.0.0 diff --git a/docs/source/_static/css/custom.css b/docs/source/_static/css/custom.css index 3e7ef3fe..17bd349f 100644 --- a/docs/source/_static/css/custom.css +++ b/docs/source/_static/css/custom.css @@ -2,8 +2,25 @@ span.caption-text { color: #cc4878; } -svg { - width: 80%; +/* shrink wide content figures (e.g. tikz workflow diagrams) without + upscaling small ones; scoped to the article body so that inline SVG + icons (fontawesome replacements in the theme chrome, sidebar arrows, + etc.) keep their intrinsic 1em sizing */ +.bd-article figure svg, +.bd-article figure img { + max-width: 80%; + height: auto; +} + +/* on wide screens, drop the primary sidebar on pages whose section + navigation is empty (top-level leaf pages such as install, tutorial, + examples, changelog, and the genindex/search pages); the flex layout + then lets the article span the reclaimed width. Below 960px the + sidebar is the off-canvas drawer and is left alone. */ +@media (min-width: 960px) { + .bd-sidebar-primary:not(:has(.bd-sidenav)) { + display: none; + } } .math { @@ -12,8 +29,3 @@ svg { .eqno { float: right; } - -/* fix: ship to main content while print webpage (pydata theme) */ -.skip-link { - position: absolute; - } diff --git a/docs/source/concepts/architecture.rst b/docs/source/concepts/architecture.rst new file mode 100644 index 00000000..de37f0d0 --- /dev/null +++ b/docs/source/concepts/architecture.rst @@ -0,0 +1,146 @@ +.. _concepts/architecture: + +Model assembly +============== + +torch_ecg models are assembled bottom-up from registered building +blocks. This page explains the layering, the registries that connect +the layers, the static shape-inference contract every model honors, +and the two output conventions present in the codebase. + +The layering +------------ + +1. **Building-block configs** — :mod:`torch_ecg.model_configs` holds + ready-made :class:`~torch_ecg.cfg.CFG` objects for every backbone + family (``resnet``, ``mobilenet_v1/v2/v3``, ``vgg16``, + ``densenet``, ``multi_scopic``, ``xception``, ``regnet``, plus the + ``Transformer`` attention block and classifier/RNN heads), grouped + so a full model config such as ``ECG_CRNN_CONFIG`` is just a few + references to block configs. +2. **Backbones** — feature extractors registered in the + ``BACKBONES`` registry; any of them can serve as the ``cnn`` of a + task model without modifying the task model's code. +3. **Task models** — the top level in the ``MODELS`` registry + (:class:`~torch_ecg.models.ECG_CRNN`, ``ECG_UNET``, ...); they own + the input/output contract (leads, signal length, classes) and + combine a backbone, attention and head into one serializable unit. + +Registries +---------- + +Registries are the package's extension points — a name-to-class map +with a uniform ``build(name, **kwargs)`` entry point: + +.. list-table:: + :header-rows: 1 + :widths: 30 70 + + * - Registry + - Contents + * - ``MODELS`` / ``BACKBONES`` + - task models / feature-extraction backbones + * - ``ATTN_LAYERS`` + - attention blocks (``Transformer``, ``SE``, ...) + * - ``OPTIMIZERS`` / ``SCHEDULERS`` + - assembled by name from ``train_config`` + * - ``LOSSES`` + - criterion classes (assembled from ``train_config.loss``) + * - ``PREPROCESSORS`` / ``AUGMENTERS`` + - preprocessing / online augmentation transforms + +Registering a custom backbone requires only a decorator +(``@BACKBONES.register()``); the +:ref:`User Guide ` shows a worked example. + +The ``compute_output_shape`` contract +------------------------------------- + +Every model and backbone implements +``compute_output_shape(seq_len, batch_size=None)``, which returns the +output shape **without running a forward pass**. This makes model +composition verifiable at construction time: a task model obtains +the feature shape from its backbone in order to size its head, and a +configuration can be checked without executing it: + +.. code-block:: python + + from torch_ecg.models import ECG_CRNN + from torch_ecg.model_configs import ECG_CRNN_CONFIG + + model = ECG_CRNN(classes=["AF", "I-AVB", "LBBB", "Normal"], n_leads=12, + config=ECG_CRNN_CONFIG) + model.compute_output_shape(2000) # (None, 4) + +The classification head globally pools over time, so the logits shape +is ``(batch, n_classes)`` regardless of input length. + +Outputs: tensors and output dataclasses +------------------------------------------------- + +The classic task models, such as ``ECG_CRNN``, return plain +logit tensors. Newer code — the +`benchmarks `__ +being the reference — wraps model outputs in typed dataclasses from +:mod:`torch_ecg.components.outputs` +(:class:`~torch_ecg.components.outputs.ClassificationOutput`, +``MultiLabelClassificationOutput``, +:class:`~torch_ecg.components.outputs.WaveDelineationOutput`, +``RPeaksDetectionOutput``, ...): fields carry explicit names +(``logits``, ``bin_pred``, ``pred_classes``, ...), the objects +support ``+`` for batching and remain plain +:class:`~torch_ecg.cfg.CFG` dicts underneath, so checkpoints and +logging keep working. When writing a new task model, prefer the +dataclass contract. + +System design +------------- + +The overall workflow of the package — data files flow through readers, +preprocessing and augmentation into tensors, which feed the trainer +driving the model, while the config file drives model construction +and checkpoints enable resuming: + +.. tikz:: Main Modules and Work Flow. + :align: center + :libs: arrows.meta, positioning + + \tikzstyle{block} = [rectangle, draw, text width = 6em, text centered, rounded corners, inner sep = 3pt, minimum height = 1.0em] + \tikzstyle{arr} = [semithick, -Stealth] + + \node[block] (dr) {Data Reader}; + \node[block, right = 1.7em of dr] (pm) {Preprocessor Manager}; + \node[block, right = 2.1em of pm] (am) {Augmenter Manager}; + \node[below = 2.51em of dr] (df) {Data Files}; + \node[below = 1.8em of pm] (rd) {Raw Data}; + \node[below = 1.8em of am] (ud) {Uniformized Data}; + \node[right = 1.7em of ud] (tensor) {Tensors}; + + \node[below right = 1.8em and 0.6em of tensor] (trainer) {Trainer}; + \node[below = 5em of tensor] (model) {Model}; + \node[left = 2.7em of model] (cfg) {Config File (Dictionary)}; + \node[block, right = 5.8em of am] (lm) {Logging Manager}; + \node[left = 3.5em of trainer, text width = 3.5em, text centered] (resume) {resume from}; + \node[left = 2em of resume] (ckpt) {Checkpoint}; + + \node[above right = 0.2em and 1.7em of trainer] (tm) {Trained Model}; + \node[below right = 0.2em and 1.7em of trainer] (rm) {Recorded Metrics}; + + \draw[arr] ([yshift=-0.2em]dr.south) -- (df); + \draw[arr] ([yshift=-0.2em]pm.south) -- (rd); + \draw[arr] ([yshift=-0.2em]am.south) -- (ud); + \draw[arr] (df) -- (rd); + \draw[arr] (rd) -- (ud); + \draw[arr] (ud) -- (tensor); + \draw[arr] (tensor.south) -- ([yshift=0.6em]trainer.west); + \draw[arr] ([yshift=-0.5em]lm) -- (trainer); + \draw[arr] (cfg) -- (model); + \draw[arr] (model.north) -- ([yshift=-0.6em]trainer.west); + \draw[arr] (trainer) -- (tm.west); + \draw[arr] (trainer) -- (rm.west); + \path[] (ckpt) edge (resume); + \draw[arr] (resume) -- (trainer); + + \draw[black,rounded corners=20,thick] ([xshift=-0.7em, yshift=2.3em]dr.west) rectangle ([xshift=0.7em, yshift=-3.3em]rm.east); + + \draw[black,rounded corners=5, dashed, thick] ([xshift=-0.7em, yshift=1.1em]cfg.west) rectangle ([xshift=0.7em, yshift=-0.9em]model.east); diff --git a/docs/source/concepts/config_system.rst b/docs/source/concepts/config_system.rst new file mode 100644 index 00000000..20bc1274 --- /dev/null +++ b/docs/source/concepts/config_system.rst @@ -0,0 +1,122 @@ +.. _concepts/config_system: + +The configuration system +======================== + +All components of torch_ecg are configured through dictionary-like +objects. This page explains the three layers of configuration, the +:class:`~torch_ecg.cfg.CFG` class that carries them, the +:data:`~torch_ecg.cfg.DEFAULTS` object that stores package-wide +settings, and a checkpoint regression that shows why the distinction +between the layers matters in practice. + +Three layers of configuration +----------------------------- + +A trained model is described by three nested layers: + +1. **Training configuration** (``train_config``) — owned by the + trainer: optimizer, learning-rate scheduler, loss, monitor metric, + early stopping, batch size, and everything about *how* to train. +2. **Outer model configuration** (``model_config``) — the full config + a task model needs to be rebuilt from scratch, including fields the + architecture itself never reads, such as ``classes`` and + ``n_leads``. +3. **Inner architecture configuration** (``self.config`` inside the + model) — what the model actually consumes: which backbone, its + building-block parameters, attention and classifier settings. + +The outer layer is what :meth:`~torch_ecg.models.ECG_CRNN.from_checkpoint` +persists and reloads. Losing the distinction between the outer and +inner layers is not a cosmetic issue — see the case study below. + +``CFG``: one object, two views +------------------------------ + +:class:`~torch_ecg.cfg.CFG` subclasses ``dict`` and mirrors every +entry as an attribute, so both ``config["key"]`` and ``config.key`` +work and stay in sync: + +.. code-block:: python + + from torch_ecg.cfg import CFG + + c = CFG(hehe={"a": 1, "b": 2}) + c.update(hehe={"a": [-1], "c": -10}) + c["hehe"] # {'a': [-1], 'b': 2, 'c': -10} + c.hehe.pop("a") # disappears from the dict view too + +Note the ``update`` semantics: it merges **recursively** — existing +sub-dictionaries are updated key by key instead of being replaced, and +a plain ``dict`` value overwriting a non-dict entry is deep-copied so +the two configs do not share mutable values. Keys that would shadow +the ``dict`` API (``copy``, ``get``, ``update``, ...) are rejected +with an error rather than silently breaking the container. + +``copy()`` versus ``deepcopy()`` +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ + +``CFG`` inherits ``dict.copy``, which returns a **plain ``dict``**: +the attribute view is gone, and so is recursive isolation. Always use +:func:`copy.deepcopy` to duplicate a config: + +.. code-block:: python + + from copy import deepcopy + from torch_ecg.cfg import CFG + + tc = CFG(classes=["AF", "I-AVB"], n_leads=12) + tc.copy() # plain dict — `tc_copy.classes` raises AttributeError + deepcopy(tc) # still a CFG, fully isolated from `tc` + +``DEFAULTS``: package-wide settings +----------------------------------- + +:data:`~torch_ecg.cfg.DEFAULTS` is a module-level ``CFG`` holding the +package defaults: + +- **dtype** — a :class:`~torch_ecg.cfg.DTYPE` dataclass keeping the + numpy/torch/string/`wfdb` integer views of one dtype consistent + (``DEFAULTS.DTYPE.TORCH``, ``DEFAULTS.str_dtype``, ...); + :func:`~torch_ecg.cfg.DEFAULTS.change_dtype` switches it globally. +- **RNG** — ``DEFAULTS.RNG`` is the canonical + :class:`numpy.random.Generator` of the package, a *single* global + stream that every stochastic component draws from. + :func:`~torch_ecg.utils.seed_everything` reseeds it (plus the global + ``random`` / ``numpy.random`` states and the torch CPU/CUDA + generators), which provides run-to-run reproducibility (see + :ref:`concepts/training_internals`). +- **cache paths** — ``log_dir``, ``checkpoints``, ``model_dir``, + ``working_dir`` under ``~/.cache/torch_ecg``. + +``DEFAULTS.device`` is **lazy**: it is not probed at import time, but +detected and cached on first access. Probing the CUDA driver during +``import torch_ecg`` adds overhead in every Python 3.14+ multiprocessing +worker (the forkserver/spawn start methods re-import the package) and +would freeze the device before ``CUDA_VISIBLE_DEVICES`` could be set. + +Case study: the 0.0.32 checkpoint regression +-------------------------------------------- + +Version 0.0.32 moved checkpoint saving to safetensors, and with it +started storing only the model's *inner* ``self.config``. The outer +fields — ``classes``, ``n_leads``, and anything else the model's +``__init__`` reads before touching the architecture — were silently +dropped. ``from_checkpoint`` reconstructs the model via +``cls(config=...)``; for any model whose ``__init__`` reads +``config.classes``, loading a 0.0.32 checkpoint crashed. + +Version 0.0.34 fixes the asymmetry at the saving side: +:meth:`CkptMixin.save ` +gained a ``model_config`` parameter and the trainer passes its full +``model_config``, so trainer-written checkpoints again contain the +complete outer config, as the pre-0.0.32 ``torch.save`` path did. + +The second half of the fix generalizes beyond this case: serializing a config +requires :func:`~torch_ecg.utils.utils_nn.make_safe_globals` to strip +values that cannot round-trip (callables, path-like values, ...). +Before the fix these fields **vanished silently** (a checkpoint lost +its ``db_dir`` with no trace); now the function emits a +``RuntimeWarning`` listing every dropped field. The underlying pattern, +converting a silent failure into an explicit warning, is applied +throughout the package. diff --git a/docs/source/concepts/index.rst b/docs/source/concepts/index.rst new file mode 100644 index 00000000..eee15c01 --- /dev/null +++ b/docs/source/concepts/index.rst @@ -0,0 +1,17 @@ +.. _concepts: + +Concepts +======== + +Explanation pages for the design decisions behind the package: how +configuration flows through the system, how ECG data is represented, +how models are assembled from registered building blocks, and the +internals of the trainer. + +.. toctree:: + :maxdepth: 1 + + config_system + representations + architecture + training_internals diff --git a/docs/source/concepts/representations.rst b/docs/source/concepts/representations.rst new file mode 100644 index 00000000..de79c264 --- /dev/null +++ b/docs/source/concepts/representations.rst @@ -0,0 +1,102 @@ +.. _concepts/representations: + +Data representation conventions +=============================== + +torch_ecg standardizes a small set of representations for signals, +waveform annotations, intervals and labels. These conventions allow +readers, datasets, augmenters and models to interoperate; this page +documents them. + +Signals: lead-first by default +------------------------------ + +A multi-lead signal is a numpy array (or tensor) of shape +``(n_leads, n_samples)`` — **lead-first** — matching +``(channel, time)`` in PyTorch. The 12-lead order is fixed by +:class:`~torch_ecg.utils.ecg_arrhythmia_knowledge.Standard12Leads`: + +.. code-block:: python + + from torch_ecg.utils.ecg_arrhythmia_knowledge import Standard12Leads + + list(Standard12Leads) + # ['I', 'II', 'III', 'aVR', 'aVL', 'aVF', 'V1', ..., 'V6'] + +Use :func:`~torch_ecg.utils.utils_data.ensure_lead_fmt` when a source +uses the lead-last layout, and +:func:`~torch_ecg.utils.utils_data.ensure_siglen` to pad or crop to a +fixed length: + +.. code-block:: python + + from torch_ecg.utils.utils_data import ensure_lead_fmt + + sig # (12, 500), lead-first + ensure_lead_fmt(sig, 12, fmt="lead_last").shape # (500, 12) + +Waveform masks +-------------- + +Wave delineation (P/QRS/T boundaries) is carried as an **integer mask +of shape ``(n_leads, n_samples)``**, with a per-task ``class_map`` +assigning wave names to integers, e.g. ``{"pwave": 1, "qrs": 2}``. +Two converters move between masks and explicit waveforms/intervals: + +- :func:`~torch_ecg.utils.utils_data.mask_to_intervals` works on a + **single lead** (a 1-D mask) and returns half-open intervals + ``[a, b)`` by default (``right_inclusive=True`` switches to ``[a, b]``): + + .. code-block:: python + + mask = np.zeros(100, dtype=int) + mask[10:20] = 1; mask[80:95] = 1; mask[50:60] = 2 + mask_to_intervals(mask, vals=1) # [[10, 20], [80, 95]] + +- :func:`~torch_ecg.utils.utils_data.masks_to_waveforms` converts the + full ``(n_leads, n_samples)`` mask into per-lead lists of + :class:`~torch_ecg.utils.utils_data.ECGWaveForm` dataclasses, with + ``onset``/``offset``/``peak`` indices and a ``duration`` in + milliseconds: + + .. code-block:: python + + wfs = masks_to_waveforms(mask_2d, class_map, fs=100, leads=["I", "II"]) + wfs["I"][0].name, wfs["I"][0].onset, wfs["I"][0].duration + # ('pwave', 10, 200.0) + +Intervals and generalized intervals +----------------------------------- + +An :data:`~torch_ecg.utils.utils_interval.Interval` is a pair +``[a, b]``; a :data:`~torch_ecg.utils.utils_interval.GeneralizedInterval` +is a list of such pairs (a possibly non-contiguous span, e.g. the +union of all QRS complexes). The +:mod:`~torch_ecg.utils.utils_interval` module provides the algebra +over them — :func:`~torch_ecg.utils.utils_interval.intervals_union`, +:func:`~torch_ecg.utils.utils_interval.generalized_intervals_union`, +:func:`~torch_ecg.utils.utils_interval.intervals_intersection`, +:func:`~torch_ecg.utils.utils_interval.generalized_interval_complement`, +length helpers, and cover-finding utilities — which the dataset +samplers use to extract training segments around annotated structures. + +Labels +------ + +Classification labels are **multi-hot vectors** over the task's +``classes`` list. :func:`~torch_ecg.utils.utils_data.one_hot_encode` +builds them from class indices, for both the single-label and the +multi-label case: + +.. code-block:: python + + from torch_ecg.utils.utils_data import one_hot_encode + + one_hot_encode([[1], [0, 2]], num_classes=3) + # array([[0., 1., 0.], + # [1., 0., 1.]], dtype=float32) + +Models output raw logits over ``classes``; the sigmoid outputs are +binarized at *inference* time (challenge-specific thresholds or the +default 0.5), rather than inside the model. This keeps the training loss and +the official evaluation metrics free of hard-coded decision rules. diff --git a/docs/source/concepts/training_internals.rst b/docs/source/concepts/training_internals.rst new file mode 100644 index 00000000..182b5d78 --- /dev/null +++ b/docs/source/concepts/training_internals.rst @@ -0,0 +1,91 @@ +.. _concepts/training_internals: + +Inside the trainer +================== + +:mod:`torch_ecg.components.trainer` provides the training loop every +benchmark builds on. This page describes that loop, together with its +determinism machinery. + +Anatomy of the training loop +---------------------------- + +``BaseTrainer`` subclasses implement ``_setup_dataloaders``, +``run_one_step`` and ``evaluate``; the base class implements the loop. Per +training step:: + + out_tensors = self.run_one_step(*data) # (preds, labels), no loss + loss = self.criterion(*out_tensors) # applied by the base loop + +The criterion is **assembled by name** from ``train_config.loss`` +through the ``LOSSES`` registry (as are the optimizer and scheduler +from their config entries), so a benchmark switches losses by editing +a single config field, without code changes. An optional ``flooding_level`` wraps the +loss with the flooding regularizer before ``backward()``. + +After each epoch, the trainer validates and then applies the +following decision logic: + +- If ``train_config.monitor`` is set (an evaluation-metric name, e.g. + ``"challenge_metric"``), a new best value promotes the just-saved + checkpoint to *best model*; ``early_stopping`` (with ``patience`` + and ``min_delta``) can stop the run when no improvement accumulates + for enough epochs. +- ``keep_checkpoint_max`` bounds how many per-epoch checkpoints stay + on disk — the oldest are removed beyond the limit. +- If no monitor is set, the **last** model is kept as the best (the + trainer logs this explicitly), and resuming + (``train(..., resume=True)``) restores the model, optimizer state + and epoch counter from the checkpoint written by this same loop. + +Determinism: seeding and workers +-------------------------------- + +:func:`~torch_ecg.utils.seed_everything` seeds **all** generators the +package and its dependencies draw from: + +- ``DEFAULTS.RNG`` — the package's canonical + :class:`numpy.random.Generator` (recreated fresh), +- the global ``random`` and ``numpy.random`` states (third-party code + draws from them), +- torch's CPU and CUDA generators. + +With ``num_workers > 0`` there is a second, subtler issue. +PyTorch reseeds *its own* generators in every DataLoader worker +(``base_seed + worker_id``), but it knows nothing about user-held +:class:`numpy.random.Generator` instances. Under ``fork`` (the Linux +default), every worker inherits a **copy** of the parent's +``DEFAULTS.RNG``, so all workers would run identical augmentation +streams — the same "random" noise in every worker. +:func:`~torch_ecg.utils.worker_init_fn` derives a distinct, +reproducible seed from the worker-specific ``torch.initial_seed()`` +and reseeds ``DEFAULTS.RNG`` per worker. Together the two functions +give bit-wise reproducible runs: + +.. code-block:: python + + from torch.utils.data import DataLoader + from torch_ecg.utils import seed_everything, worker_init_fn + + seed_everything(42) + dl = DataLoader(ds, batch_size=32, num_workers=4, + worker_init_fn=worker_init_fn) + +Where augmentation happens +-------------------------- + +Online augmentation runs **inside the DataLoader workers**: the +dataset's ``__getitem__`` applies its +:class:`~torch_ecg.augmenters.AugmenterManager` to each sample before +collation, so the training loop receives only augmented data and the +CPU work parallelizes across workers. + +The manager's callable takes **two** arguments — ``(sig, label)`` — +because correct ECG augmentation must be able to keep the annotation +in sync with the signal. Today's classification labels are multi-hot +vectors that are invariant to signal-level transforms (nothing to +re-permute for transforms such as baseline wander), but transforms that *do* +reorder or reshape the signal would break if the label were not in +the signature; the two-argument contract preserves composability for +waveform-level tasks (e.g. lead-sensitive augmenters) without changes +to the calling convention. diff --git a/docs/source/index.rst b/docs/source/index.rst index 4d132410..83d628ac 100644 --- a/docs/source/index.rst +++ b/docs/source/index.rst @@ -11,61 +11,19 @@ torch-ecg documentation ECG Deep Learning Framework Implemented using PyTorch. -The system design is depicted as follows - -.. tikz:: Main Modules and Work Flow. - :align: center - :libs: arrows.meta, positioning - - \tikzstyle{block} = [rectangle, draw, text width = 6em, text centered, rounded corners, inner sep = 3pt, minimum height = 1.0em] - \tikzstyle{arr} = [semithick, -Stealth] - - \node[block] (dr) {Data Reader}; - \node[block, right = 1.7em of dr] (pm) {Preprocessor Manager}; - \node[block, right = 2.1em of pm] (am) {Augmenter Manager}; - \node[below = 2.51em of dr] (df) {Data Files}; - \node[below = 1.8em of pm] (rd) {Raw Data}; - \node[below = 1.8em of am] (ud) {Uniformized Data}; - \node[right = 1.7em of ud] (tensor) {Tensors}; - - \node[below right = 1.8em and 0.6em of tensor] (trainer) {Trainer}; - \node[below = 5em of tensor] (model) {Model}; - \node[left = 2.7em of model] (cfg) {Config File (Dictionary)}; - \node[block, right = 5.8em of am] (lm) {Logging Manager}; - \node[left = 3.5em of trainer, text width = 3.5em, text centered] (resume) {resume from}; - \node[left = 2em of resume] (ckpt) {Checkpoint}; - - \node[above right = 0.2em and 1.7em of trainer] (tm) {Trained Model}; - \node[below right = 0.2em and 1.7em of trainer] (rm) {Recorded Metrics}; - - \draw[arr] ([yshift=-0.2em]dr.south) -- (df); - \draw[arr] ([yshift=-0.2em]pm.south) -- (rd); - \draw[arr] ([yshift=-0.2em]am.south) -- (ud); - \draw[arr] (df) -- (rd); - \draw[arr] (rd) -- (ud); - \draw[arr] (ud) -- (tensor); - \draw[arr] (tensor.south) -- ([yshift=0.6em]trainer.west); - \draw[arr] ([yshift=-0.5em]lm) -- (trainer); - \draw[arr] (cfg) -- (model); - \draw[arr] (model.north) -- ([yshift=-0.6em]trainer.west); - \draw[arr] (trainer) -- (tm.west); - \draw[arr] (trainer) -- (rm.west); - \path[] (ckpt) edge (resume); - \draw[arr] (resume) -- (trainer); - - \draw[black,rounded corners=20,thick] ([xshift=-0.7em, yshift=2.3em]dr.west) rectangle ([xshift=0.7em, yshift=-3.3em]rm.east); - - \draw[black,rounded corners=5, dashed, thick] ([xshift=-0.7em, yshift=1.1em]cfg.west) rectangle ([xshift=0.7em, yshift=-0.9em]model.east); +The overall system design is depicted in +:ref:`concepts/architecture` (main modules and work flow). .. toctree:: :maxdepth: 1 :caption: Contents: Installation - User Guide Tutorial - API Reference + User Guide + Concepts Examples + API Reference Changelog Version history