From 2d10b798aae9b2660411e34879a65296bee3677c Mon Sep 17 00:00:00 2001 From: WEN Hao Date: Wed, 28 Oct 2026 07:56:32 +0800 Subject: [PATCH 1/2] docs: add the visualization and parallel-reproducibility tutorials - tutorial-visualization.rst: merge a two-seed experiment into one mean curve with error bounds and export it to PDF; the interactive Panel dashboard is introduced for notebook use; the shown code is embedded via literalinclude from test-files/tutorial_visualization.py - tutorial-parallel-reproducible.rst: one-line switch to the threads executor, then verify bitwise-identical final weights across backends with rng_mode per_client, and contrast the statistical-level global mode; embedded from test-files/tutorial_parallel_check.py - tutorial_first_experiment.py: every run now takes a fresh model from the candidate-models property; the server trains the model it is given in place, so sharing one module across runs would warm-start every run after the first - test/test_tutorial_scripts.py: two new behavioral smoke tests (curve merge and PDF export in a temporary working directory; bitwise sequential-vs-threads equality with per-client RNG) - zh_CN translations for the new and updated pages --- docs/source/index.rst | 2 + .../LC_MESSAGES/tutorial-first-experiment.po | 96 +++++----- .../tutorial-parallel-reproducible.po | 173 ++++++++++++++++++ .../LC_MESSAGES/tutorial-visualization.po | 145 +++++++++++++++ docs/source/tutorial-first-experiment.rst | 2 + .../source/tutorial-parallel-reproducible.rst | 101 ++++++++++ docs/source/tutorial-visualization.rst | 79 ++++++++ test-files/tutorial_first_experiment.py | 13 +- test-files/tutorial_parallel_check.py | 74 ++++++++ test-files/tutorial_visualization.py | 73 ++++++++ test/test_tutorial_scripts.py | 53 ++++++ 11 files changed, 757 insertions(+), 54 deletions(-) create mode 100644 docs/source/locale/zh_CN/LC_MESSAGES/tutorial-parallel-reproducible.po create mode 100644 docs/source/locale/zh_CN/LC_MESSAGES/tutorial-visualization.po create mode 100644 docs/source/tutorial-parallel-reproducible.rst create mode 100644 docs/source/tutorial-visualization.rst create mode 100644 test-files/tutorial_parallel_check.py create mode 100644 test-files/tutorial_visualization.py diff --git a/docs/source/index.rst b/docs/source/index.rst index bdb96ede..6fb62e21 100644 --- a/docs/source/index.rst +++ b/docs/source/index.rst @@ -24,6 +24,8 @@ Project Links: `GitHub `_ | `gitee \n" "Language: zh_CN\n" @@ -31,9 +31,9 @@ msgid "" "ratios, then dissect what \"communication cost\" actually means in fl-" "sim." msgstr "" -":doc:`Quickstart ` 只跑了一次训练。真正的实验要在相同条件下比较不同配置，" -"并把结果读回来。本教程将比较三个算法在两种采样比例下的表现，然后剖析 fl-sim 中" -"\"通信开销\"究竟指什么。" +":doc:`Quickstart ` " +"只跑了一次训练。真正的实验要在相同条件下比较不同配置，并把结果读回来。本教程将比较三个算法在两种采样比例下的表现，然后剖析 fl-sim " +"中\"通信开销\"究竟指什么。" #: ../../source/tutorial-first-experiment.rst:11 msgid "" @@ -42,17 +42,15 @@ msgid "" "downloads, seconds per run. The full script shown below lives at ``test-" "files/tutorial_first_experiment.py`` and is exercised in CI." msgstr "" -"全部运行基于 :class:`~fl_sim.data_processing.fed_synthetic.FedSynthetic`——" -"无需下载，每次运行只需几秒。下文展示的完整脚本位于 " -"``test-files/tutorial_first_experiment.py``，并在 CI 中被执行。" +"全部运行基于 " +":class:`~fl_sim.data_processing.fed_synthetic.FedSynthetic`——无需下载，每次运行只需几秒。下文展示的完整脚本位于" +" ``test-files/tutorial_first_experiment.py``，并在 CI 中被执行。" #: ../../source/tutorial-first-experiment.rst:15 msgid "" "Prerequisites: :doc:`quickstart` (the three-object model: dataset, model," " server) and :doc:`concepts` (what a round does)." -msgstr "" -"前置阅读： :doc:`quickstart` （三对象模型：数据集、模型、服务器）与 " -" :doc:`concepts` （一轮训练做了什么）。" +msgstr "前置阅读： :doc:`quickstart` （三对象模型：数据集、模型、服务器）与 :doc:`concepts` （一轮训练做了什么）。" #: ../../source/tutorial-first-experiment.rst:19 msgid "Set up the comparison" @@ -65,9 +63,8 @@ msgid "" "gradient, proximal and variance-reduction axes — each at **sampling " "ratios 0.5 and 1.0**." msgstr "" -"六次运行： **FedAvg** （朴素平均）、 **FedProx** （近端正则的本地目标）与 " -" **SCAFFOLD** （控制变量）——梯度、近端、方差缩减三条经典轴线——各自在 " -" **采样比例 0.5 与 1.0** 下运行。" +"六次运行： **FedAvg** （朴素平均）、 **FedProx** （近端正则的本地目标）与 **SCAFFOLD** " +"（控制变量）——梯度、近端、方差缩减三条经典轴线——各自在 **采样比例 0.5 与 1.0** 下运行。" #: ../../source/tutorial-first-experiment.rst:26 msgid "" @@ -77,9 +74,8 @@ msgid "" "sampling stream of everything after it. The order *dataset → config → " "server* keeps every run reproducible:" msgstr "" -"一个刻意的选择：数据集 **只构造一次，且先于任何配置或服务器** 。配置类的构造器在" -"构建时会重置全局 RNG，谁最后构造谁生效，其后的采样流都会被移动。按 " -"*数据集 → 配置 → 服务器* 的顺序构造，每次运行都可复现：" +"一个刻意的选择：数据集 **只构造一次，且先于任何配置或服务器** 。配置类的构造器在构建时会重置全局 " +"RNG，谁最后构造谁生效，其后的采样流都会被移动。按 *数据集 → 配置 → 服务器* 的顺序构造，每次运行都可复现：" #: ../../source/tutorial-first-experiment.rst:37 msgid "" @@ -89,9 +85,9 @@ msgid "" "frequency parameter later. Both are per-algorithm server keywords, " "collected in ``SERVER_KW``." msgstr "" -"``final_accuracy`` 读取一次运行的 JSON 日志——与 :doc:`quickstart` 中相同的文件" -"格式。SCAFFOLD 需要显式传入全局步长 ``lr``（它的配置没有默认值）；FedPD 稍后也需要" -"一个频率参数。两者都是算法各自的服务器关键字，统一收在 ``SERVER_KW`` 里。" +"``final_accuracy`` 读取一次运行的 JSON 日志——与 :doc:`quickstart` 中相同的文件格式。SCAFFOLD" +" 需要显式传入全局步长 ``lr``（它的配置没有默认值）；FedPD 稍后也需要一个频率参数。两者都是算法各自的服务器关键字，统一收在 " +"``SERVER_KW`` 里。" #: ../../source/tutorial-first-experiment.rst:44 msgid "Run and read the results" @@ -120,9 +116,9 @@ msgid "" "harder, more heterogeneous tasks (see the :doc:`algorithm theory pages " "` for when each helps)." msgstr "" -"这个合成任务 *太容易* ——全采样下 FedProx 与 SCAFFOLD 达到了和 FedAvg 相同的精度" -"天花板。它们的优势要在更难、异质性更强的任务上才显现（何时用哪个，见 " -":doc:`算法理论页 ` ）。" +"这个合成任务 *太容易* ——全采样下 FedProx 与 SCAFFOLD 达到了和 FedAvg " +"相同的精度天花板。它们的优势要在更难、异质性更强的任务上才显现（何时用哪个，见 :doc:`算法理论页 " +"` ）。" #: ../../source/tutorial-first-experiment.rst:74 msgid "" @@ -131,8 +127,8 @@ msgid "" "1.0 used exactly 180 — regardless of the algorithm. That number deserves " "a closer look, because it is easy to misread." msgstr "" -"精度的故事饱和了， **通信** 的故事没有：比例 0.5 的每次运行都恰好用了 90 次通信，" -"比例 1.0 的都恰好 180 次——与算法无关。这个数字值得细看，因为它很容易被误读。" +"精度的故事饱和了， **通信** 的故事没有：比例 0.5 的每次运行都恰好用了 90 次通信，比例 1.0 的都恰好 180 " +"次——与算法无关。这个数字值得细看，因为它很容易被误读。" #: ../../source/tutorial-first-experiment.rst:80 msgid "What ``num_communications`` counts" @@ -147,8 +143,8 @@ msgid "" " doubles to 180." msgstr "" "``num_communications`` 统计的是 **消息数，不是轮数** ：每轮对 *每个被采样客户端* " -"加一次下行广播，再加一次上行回复。30 个客户端、比例 0.5 时，每轮是 15 下行 + " -"15 上行 = 30 条消息，3 轮 = 90；比例 1.0 时翻倍为 180。" +"加一次下行广播，再加一次上行回复。30 个客户端、比例 0.5 时，每轮是 15 下行 + 15 上行 = 30 条消息，3 轮 = 90；比例 " +"1.0 时翻倍为 180。" #: ../../source/tutorial-first-experiment.rst:88 msgid "" @@ -176,8 +172,8 @@ msgid "" "round's full cost — read the final count from the server, as the scripts " "here do.)" msgstr "" -"（另注：JSON 日志里记录的 ``step`` 是 *每轮开始时* 的快照：它比最终计数少最后一轮的" -"全部开销——最终值请从服务器读取，正如本页脚本的做法。）" +"（另注：JSON 日志里记录的 ``step`` 是 *每轮开始时* " +"的快照：它比最终计数少最后一轮的全部开销——最终值请从服务器读取，正如本页脚本的做法。）" #: ../../source/tutorial-first-experiment.rst:106 msgid "Sampling is not communicating" @@ -191,9 +187,8 @@ msgid "" "every ``int(1/p)``-th round — for five rounds next to a five-round " "FedAvg:" msgstr "" -"上面三个算法每一轮都在通信。fl-sim 还内置 **可跳过通信** 的算法，它们会以很有启发性的" -"方式打破 ``2 × 采样数 × 轮数`` 的恒等式。让 FedPD（每 ``int(1/p)`` 轮才上传一次）" -"与五轮 FedAvg 并排各跑五轮：" +"上面三个算法每一轮都在通信。fl-sim 还内置 **可跳过通信** 的算法，它们会以很有启发性的方式打破 ``2 × 采样数 × 轮数`` " +"的恒等式。让 FedPD（每 ``int(1/p)`` 轮才上传一次）与五轮 FedAvg 并排各跑五轮：" #: ../../source/tutorial-first-experiment.rst:119 msgid "Output on one machine:" @@ -207,9 +202,8 @@ msgid "" "messages as three FedAvg rounds — the same communication budget buys two " "extra local iterations." msgstr "" -"分解式把话说得非常准： **下行完全相同** （75 = 5 轮 × 15 客户端——对被采样集合的广播" -"从不跳过）；FedPD 省下的 *全是上行* 。五轮 FedPD 花费的 90 条消息与三轮 FedAvg 相同" -"——同样的通信预算换来了多两轮的本地迭代。" +"分解式把话说得非常准： **下行完全相同** （75 = 5 轮 × 15 客户端——对被采样集合的广播从不跳过）；FedPD 省下的 " +"*全是上行* 。五轮 FedPD 花费的 90 条消息与三轮 FedAvg 相同——同样的通信预算换来了多两轮的本地迭代。" #: ../../source/tutorial-first-experiment.rst:132 msgid "Three practical details when you run this yourself:" @@ -223,9 +217,9 @@ msgid "" "frequency ``int(1/p)`` always wins — passing a ``comm_freq`` yourself is " "silently overwritten." msgstr "" -"``p`` 必须满足 **0 < p < 1** 。p = 1 时 FedPD 每轮都上传（退化为 FedAvg）；p > 1 时" -"上传频率变为零，运行直接崩溃。配置暴露的是 ``p`` ，派生的上传频率 ``int(1/p)`` " -"始终说了算——自己传 ``comm_freq`` 会被静默覆写。" +"``p`` 必须满足 **0 < p < 1** 。p = 1 时 FedPD 每轮都上传（退化为 FedAvg）；p > 1 " +"时上传频率变为零，运行直接崩溃。配置暴露的是 ``p`` ，派生的上传频率 ``int(1/p)`` 始终说了算——自己传 " +"``comm_freq`` 会被静默覆写。" #: ../../source/tutorial-first-experiment.rst:139 msgid "" @@ -238,11 +232,11 @@ msgid "" "appears once per (skipped client × skipped round), so it dominates the " "console. Both are how FedPD is designed to behave." msgstr "" -" **你会看到 RuntimeWarning——那是预期行为，不是错误。** 服务器对每个跳过的轮次记录 " -"``No message received from the clients...`` （\"这轮没人上传，全局模型没有动\"）；" -"每个跳过的客户端记录 ``No parameters received from server...`` （\"这轮没收到新的" -"全局模型，我继续用当前参数\"）。教程规模下，客户端侧的消息每个（跳过客户端 × 跳过轮次）" -"出现一次，会刷满控制台。两者都是 FedPD 的设计行为。" +" **你会看到 RuntimeWarning——那是预期行为，不是错误。** 服务器对每个跳过的轮次记录 ``No message " +"received from the clients...`` （\"这轮没人上传，全局模型没有动\"）；每个跳过的客户端记录 ``No " +"parameters received from server...`` " +"（\"这轮没收到新的全局模型，我继续用当前参数\"）。教程规模下，客户端侧的消息每个（跳过客户端 × 跳过轮次）出现一次，会刷满控制台。两者都是 " +"FedPD 的设计行为。" #: ../../source/tutorial-first-experiment.rst:147 msgid "" @@ -251,9 +245,8 @@ msgid "" "unlogged), for FedPD at 75 (its last round never uploads). The final " "counts come from ``server._num_communications``." msgstr "" -"JSON 的 ``step`` 快照落后多少因算法而异：五轮 FedAvg 终于 120（最后一轮的 30 条消息" -"未入日志），FedPD 终于 75（它的最后一轮从不上传）。最终计数请以 " -"``server._num_communications`` 为准。" +"JSON 的 ``step`` 快照落后多少因算法而异：五轮 FedAvg 终于 120（最后一轮的 30 条消息未入日志），FedPD 终于 " +"75（它的最后一轮从不上传）。最终计数请以 ``server._num_communications`` 为准。" #: ../../source/tutorial-first-experiment.rst:153 msgid "Next steps" @@ -263,12 +256,17 @@ msgstr "下一步" msgid "" ":doc:`tutorial-cli-matrix` — run the same kind of comparison from the " "command line, as a YAML-driven experiment matrix." -msgstr "" -":doc:`tutorial-cli-matrix`——用 YAML 实验矩阵从命令行跑同样的对照。" +msgstr ":doc:`tutorial-cli-matrix`——用 YAML 实验矩阵从命令行跑同样的对照。" #: ../../source/tutorial-first-experiment.rst:157 msgid "" +":doc:`tutorial-visualization` — merge multi-seed runs of this comparison " +"into mean curves with error bounds." +msgstr "" +":doc:`tutorial-visualization`——把本次对照的多 seed 运行合并成带误差带的均值曲线。" + +#: ../../source/tutorial-first-experiment.rst:159 +msgid "" ":doc:`parallel_execution` — speed these runs up with a parallel executor " "without changing the results." -msgstr "" -":doc:`parallel_execution`——用并行执行后端加速这些运行，且不改变结果。" +msgstr ":doc:`parallel_execution`——用并行执行后端加速这些运行，且不改变结果。" diff --git a/docs/source/locale/zh_CN/LC_MESSAGES/tutorial-parallel-reproducible.po b/docs/source/locale/zh_CN/LC_MESSAGES/tutorial-parallel-reproducible.po new file mode 100644 index 00000000..16a49b77 --- /dev/null +++ b/docs/source/locale/zh_CN/LC_MESSAGES/tutorial-parallel-reproducible.po @@ -0,0 +1,173 @@ +# SOME DESCRIPTIVE TITLE. +# Copyright (C) 2023, WEN Hao +# This file is distributed under the same license as the fl-sim package. +# FIRST AUTHOR , 2026. +# +#, fuzzy +msgid "" +msgstr "" +"Project-Id-Version: fl-sim \n" +"Report-Msgid-Bugs-To: \n" +"POT-Creation-Date: 2026-10-28 07:53+0800\n" +"PO-Revision-Date: YEAR-MO-DA HO:MI+ZONE\n" +"Last-Translator: FULL NAME \n" +"Language: zh_CN\n" +"Language-Team: zh_CN \n" +"Plural-Forms: nplurals=1; plural=0;\n" +"MIME-Version: 1.0\n" +"Content-Type: text/plain; charset=utf-8\n" +"Content-Transfer-Encoding: 8bit\n" +"Generated-By: Babel 2.16.0\n" + +#: ../../source/tutorial-parallel-reproducible.rst:4 +msgid "Tutorial: parallel and reproducible runs" +msgstr "教程：并行与可复现的运行" + +#: ../../source/tutorial-parallel-reproducible.rst:6 +msgid "" +"Experiments grow; wall-clock time should not. fl-sim can execute the " +"clients' local steps concurrently on a thread pool — with a one-line " +"config change — and this tutorial shows how to keep the run " +"*reproducible* while doing so: at the end, two different backends produce" +" **bitwise identical final model weights**." +msgstr "" +"实验规模会变大，墙钟时间不该跟着变。fl-sim 能在线程池上并发执行客户端的本地步——只需" +"一行配置——本教程展示这样做的同时如何保持运行 *可复现* ：最后，两个不同的后端产出" +" **逐位一致的最终模型权重** 。" + +#: ../../source/tutorial-parallel-reproducible.rst:12 +msgid "" +"The full script lives at ``test-files/tutorial_parallel_check.py`` and is" +" exercised in CI. Prerequisites: :doc:`tutorial-first-experiment`." +msgstr "" +"完整脚本位于 ``test-files/tutorial_parallel_check.py``，并在 CI 中被执行。" +"前置阅读： :doc:`tutorial-first-experiment` 。" + +#: ../../source/tutorial-parallel-reproducible.rst:15 +msgid "" +"The mechanics behind this are documented in :doc:`parallel_execution` " +"(backends, guarantees, and how custom algorithms stay compatible); this " +"page is the hands-on version." +msgstr "" +"背后的机制见 :doc:`parallel_execution` （后端、保证、自定义算法如何保持兼容）；" +"本页是动手版。" + +#: ../../source/tutorial-parallel-reproducible.rst:20 +msgid "One line of parallelism" +msgstr "一行开启并行" + +#: ../../source/tutorial-parallel-reproducible.rst:22 +msgid "" +"Add ``executor=\"threads\"`` to the server config. Nothing else changes —" +" algorithms, datasets, logging and the participation schedule all behave " +"identically; only the per-client local steps (evaluate → local update → " +"build upload) run on a thread pool:" +msgstr "" +"在服务器配置里加 ``executor=\"threads\"`` 。其余一切不变——算法、数据集、日志、" +"参与调度全都原样；只有每个客户端的本地步（评估 → 本地更新 → 构建上传）" +"跑在线程池上：" + +#: ../../source/tutorial-parallel-reproducible.rst:32 +msgid "" +"The helpers capture what we will compare: the final model weights " +"(``weights_of``) and an exact equality check over them (``bitwise_equal``" +" — ``torch.equal``, not ``allclose``)." +msgstr "" +"两个辅助函数负责我们要比较的东西：最终模型权重（ ``weights_of`` ）及其精确相等检查" +"（ ``bitwise_equal`` ——用 ``torch.equal`` ，不是 ``allclose`` ）。" + +#: ../../source/tutorial-parallel-reproducible.rst:37 +msgid "The two backends agree exactly" +msgstr "两种后端完全一致" + +#: ../../source/tutorial-parallel-reproducible.rst:39 +msgid "" +"Run the same experiment on both backends with the client config's " +"``rng_mode=\"per_client\"`` — each client's dataloaders get their own " +"generators seeded from ``(seed, client_id, round)`` — and compare:" +msgstr "" +"把客户端配置设为 ``rng_mode=\"per_client\"`` （每个客户端的 dataloader 拿到由 " +"``(seed, client_id, round)`` 派生种子的专属生成器），在两种后端上跑同一实验并比较：" + +#: ../../source/tutorial-parallel-reproducible.rst:48 +msgid "Output:" +msgstr "输出：" + +#: ../../source/tutorial-parallel-reproducible.rst:54 +msgid "" +"Bitwise identical — not \"close\", not \"same accuracy\": the same " +"tensors. This holds regardless of how many workers ran or how the " +"operating system scheduled them." +msgstr "" +"逐位一致——不是\"接近\"，也不是\"精度相同\"：就是同一批张量。无论跑了多少个 worker、" +"操作系统如何调度，这个结论都成立。" + +#: ../../source/tutorial-parallel-reproducible.rst:59 +msgid "What the global RNG mode buys (and costs)" +msgstr "全局 RNG 模式得到了什么（又付出了什么）" + +#: ../../source/tutorial-parallel-reproducible.rst:61 +msgid "" +"``rng_mode`` defaults to ``\"global\"``: clients share the process-global" +" torch RNG. The script runs the same comparison in that mode too:" +msgstr "" +"``rng_mode`` 默认为 ``\"global\"`` ：客户端共享进程级全局 torch RNG。脚本在那种模式下" +"也跑了同样的比较：" + +#: ../../source/tutorial-parallel-reproducible.rst:69 +msgid "" +"Under the global mode the *participation schedule* and the *aggregation " +"arithmetic* are still backend-independent (sampling decisions and reply " +"consumption happen on the main thread in fixed order), so the " +"communication counts always match. The clients' local trajectories, " +"however, consume the shared RNG concurrently — so their interleaving " +"depends on thread scheduling, and the final weights may differ between " +"backends and between repeats:" +msgstr "" +"全局模式下， *参与调度* 与 *聚合算术* 仍与后端无关（采样决策与回复消费都在主线程按固定" +"顺序进行），因此通信计数永远一致。但客户端的本地轨迹会并发消耗共享 RNG——交叠情况取决于" +"线程调度，最终权重可能因后端不同、重复运行不同而不同：" + +#: ../../source/tutorial-parallel-reproducible.rst:81 +msgid "" +"(On a small run like this one the tasks often serialize and even the " +"weights match — treat that as luck, not a guarantee.)" +msgstr "（像这样的小规模运行，任务常常自动串行，连权重都恰好一致——那是运气，不是保证。）" + +#: ../../source/tutorial-parallel-reproducible.rst:84 +msgid "The rule of thumb:" +msgstr "经验法则：" + +#: ../../source/tutorial-parallel-reproducible.rst:86 +msgid "" +"**exploring**: global mode is fine; run-to-run variation is part of the " +"measurement (average over seeds, as in :doc:`tutorial-visualization`)." +msgstr "" +" **探索阶段** ：全局模式没问题；运行间的波动本来就是测量的一部分（对多个 seed 取平均，" +"见 :doc:`tutorial-visualization` ）。" + +#: ../../source/tutorial-parallel-reproducible.rst:89 +msgid "" +"**paper-grade seed alignment**: either stay on ``\"sequential\"``, or set" +" ``rng_mode=\"per_client\"`` — then every backend, every parallelism " +"degree and every repeat produce the same trajectory from the same seed." +msgstr "" +" **论文级种子对齐** ：要么保持 ``\"sequential\"``，要么设 ``rng_mode=\"per_client\"``" +"——之后同一种子在每个后端、每种并行度、每次重复下都走出同一条轨迹。" + +#: ../../source/tutorial-parallel-reproducible.rst:95 +msgid "Next steps" +msgstr "下一步" + +#: ../../source/tutorial-parallel-reproducible.rst:97 +msgid "" +":doc:`parallel_execution` — the full contract: guarantees per backend, " +"the ``FL_SIM_EXECUTOR``/``FL_SIM_NUM_WORKERS`` environment overrides, " +"measured speedups, and what custom algorithms must (not) touch." +msgstr "" +":doc:`parallel_execution`——完整契约：每个后端的保证、 ``FL_SIM_EXECUTOR`` / " +"``FL_SIM_NUM_WORKERS`` 环境变量覆盖、实测加速比，以及自定义算法哪些能碰哪些不能碰。" + +#: ../../source/tutorial-parallel-reproducible.rst:100 +msgid ":doc:`tutorial-visualization` — plot the runs you just made reproducible." +msgstr ":doc:`tutorial-visualization`——把刚变得可复现的这些运行画出来。" diff --git a/docs/source/locale/zh_CN/LC_MESSAGES/tutorial-visualization.po b/docs/source/locale/zh_CN/LC_MESSAGES/tutorial-visualization.po new file mode 100644 index 00000000..98cc8da1 --- /dev/null +++ b/docs/source/locale/zh_CN/LC_MESSAGES/tutorial-visualization.po @@ -0,0 +1,145 @@ +# SOME DESCRIPTIVE TITLE. +# Copyright (C) 2023, WEN Hao +# This file is distributed under the same license as the fl-sim package. +# FIRST AUTHOR , 2026. +# +#, fuzzy +msgid "" +msgstr "" +"Project-Id-Version: fl-sim \n" +"Report-Msgid-Bugs-To: \n" +"POT-Creation-Date: 2026-10-28 07:53+0800\n" +"PO-Revision-Date: YEAR-MO-DA HO:MI+ZONE\n" +"Last-Translator: FULL NAME \n" +"Language: zh_CN\n" +"Language-Team: zh_CN \n" +"Plural-Forms: nplurals=1; plural=0;\n" +"MIME-Version: 1.0\n" +"Content-Type: text/plain; charset=utf-8\n" +"Content-Transfer-Encoding: 8bit\n" +"Generated-By: Babel 2.16.0\n" + +#: ../../source/tutorial-visualization.rst:4 +msgid "Tutorial: visualize and export results" +msgstr "教程：可视化与导出结果" + +#: ../../source/tutorial-visualization.rst:6 +msgid "" +"Every run writes ``.json`` logs (see :doc:`quickstart`); fl-sim ships a " +"visualization layer that turns them into publication-ready curves — " +"interactively in Jupyter, or programmatically from plain Python. This " +"tutorial does both: merge a multi-seed experiment into one mean curve " +"with error bounds, export it to PDF, and open the interactive dashboard." +msgstr "" +"每次运行都会写 ``.json`` 日志（见 :doc:`quickstart` ）；fl-sim 自带的可视化层能把它们变成" +"可直接发表的曲线——在 Jupyter 里交互式操作，或在纯 Python 里以编程方式处理。本教程两者" +"都做：把多 seed 实验合并成一条带误差带的均值曲线，导出 PDF，再打开交互式仪表盘。" + +#: ../../source/tutorial-visualization.rst:12 +msgid "" +"The full script lives at ``test-files/tutorial_visualization.py`` and is " +"exercised in CI (everything except the dashboard rendering, which needs " +"Jupyter). Prerequisites: :doc:`tutorial-first-experiment`." +msgstr "" +"完整脚本位于 ``test-files/tutorial_visualization.py``，并在 CI 中被执行（仪表盘渲染除外，" +"它需要 Jupyter）。前置阅读： :doc:`tutorial-first-experiment` 。" + +#: ../../source/tutorial-visualization.rst:16 +msgid "" +"The reference for all knobs (fonts, smoothing, widgets) is :doc:`viz`; " +"this page is the workflow." +msgstr "所有可调项（字体、平滑、控件）的参考见 :doc:`viz` ；本页讲的是工作流程。" + +#: ../../source/tutorial-visualization.rst:20 +msgid "Get some logs" +msgstr "准备日志" + +#: ../../source/tutorial-visualization.rst:22 +msgid "" +"The script trains the same FedAvg experiment under two seeds — the " +"minimal \"multi-seed\" setup. Note the per-run ``tag``: log file names " +"carry it, so the two runs cannot overwrite each other (the same reasoning" +" as in :doc:`tutorial-cli-matrix`):" +msgstr "" +"脚本用两个种子训练同一个 FedAvg 实验——最小的\"多 seed\"设置。注意每次运行的 " +"``tag`` ：日志文件名携带它，两次运行不会互相覆盖（道理同 :doc:`tutorial-cli-matrix` ）：" + +#: ../../source/tutorial-visualization.rst:33 +msgid "Merge seeds into one curve" +msgstr "把多个 seed 合并成一条曲线" + +#: ../../source/tutorial-visualization.rst:35 +msgid "" +"``find_log_files`` collects the JSON logs; " +"``aggregate_results_from_json_log_with_custom_axis`` extracts a curve " +"(``step`` on x, ``acc`` on y here — any recorded metric works); " +"``plot_mean_curve_with_error_bounds`` merges the curves into a mean with " +"an uncertainty band (``\"sem\"``, ``\"std\"``, ``\"quartile\"`` or " +"``\"iqr\"``). Saving the figure is plain matplotlib:" +msgstr "" +"``find_log_files`` 收集 JSON 日志； ``aggregate_results_from_json_log_with_custom_axis`` " +"抽取曲线（这里 x 轴 ``step``、y 轴 ``acc``——任何记录过的指标都行）； " +"``plot_mean_curve_with_error_bounds`` 把曲线合并成带不确定度带的均值（ ``\"sem\"`` 、 " +"``\"std\"`` 、 ``\"quartile\"`` 或 ``\"iqr\"`` ）。存图就是普通的 matplotlib：" + +#: ../../source/tutorial-visualization.rst:46 +msgid "" +"This writes ``fedavg-multi-seed.pdf`` next to your working directory. Two" +" things worth noticing:" +msgstr "这会在工作目录旁写出 ``fedavg-multi-seed.pdf`` 。两点值得注意：" + +#: ../../source/tutorial-visualization.rst:49 +msgid "" +"The x axis is the **cumulative communication count** — the metric :doc" +":`tutorial-first-experiment` dissected. Plotting against communications " +"(rather than rounds) is what makes different communication-efficiency " +"algorithms comparable in one figure." +msgstr "" +"x 轴是 **累计通信次数** ——即 :doc:`tutorial-first-experiment` 剖析过的那个量。" +"以通信（而不是轮数）为横轴，才能把通信效率不同的算法放进同一张图里比较。" + +#: ../../source/tutorial-visualization.rst:53 +msgid "" +"With only two seeds the band is a range, not a statistic — use more seeds" +" (just add more ``seed=`` runs; the merging code does not change)." +msgstr "" +"只有两个 seed 时，带子只是一个范围，还不是统计量——多加几个 ``seed=`` 运行即可" +"（合并代码一行都不用改）。" + +#: ../../source/tutorial-visualization.rst:58 +msgid "Explore interactively" +msgstr "交互式探索" + +#: ../../source/tutorial-visualization.rst:60 +msgid "" +"In a Jupyter notebook, the ``Panel`` dashboard does all of the above " +"interactively: it lists the log files it finds, plots selectable metrics," +" and gives you sliders for smoothing, fonts and figure size, plus one-" +"click export to PDF/SVG/PNG:" +msgstr "" +"在 Jupyter notebook 里， ``Panel`` 仪表盘把上面的所有事都变成交互式的：列出找到的日志" +"文件、绘制可选的指标，提供平滑、字体与图幅的滑块，以及一键导出 PDF/SVG/PNG：" + +#: ../../source/tutorial-visualization.rst:70 +msgid "" +"Run this cell in a notebook (JupyterLab, Colab, SageMaker, …) and the " +"dashboard renders inline; see :doc:`viz` for a demo animation and the " +"full feature list." +msgstr "" +"在 notebook（JupyterLab、Colab、SageMaker 等）里运行这个单元格，仪表盘就会内嵌渲染；" +"演示动画与完整功能列表见 :doc:`viz` 。" + +#: ../../source/tutorial-visualization.rst:75 +msgid "Next steps" +msgstr "下一步" + +#: ../../source/tutorial-visualization.rst:77 +msgid "" +":doc:`tutorial-parallel-reproducible` — make these runs faster (parallel " +"executor) without giving up reproducibility." +msgstr "" +":doc:`tutorial-parallel-reproducible`——让这些运行更快（并行执行器），且不放弃可复现性。" + +#: ../../source/tutorial-visualization.rst:79 +msgid ":doc:`viz` — the reference for every visualization option." +msgstr ":doc:`viz`——所有可视化选项的参考。" diff --git a/docs/source/tutorial-first-experiment.rst b/docs/source/tutorial-first-experiment.rst index 98dfc219..736aa0a2 100644 --- a/docs/source/tutorial-first-experiment.rst +++ b/docs/source/tutorial-first-experiment.rst @@ -154,5 +154,7 @@ Next steps - :doc:`tutorial-cli-matrix` — run the same kind of comparison from the command line, as a YAML-driven experiment matrix. +- :doc:`tutorial-visualization` — merge multi-seed runs of this + comparison into mean curves with error bounds. - :doc:`parallel_execution` — speed these runs up with a parallel executor without changing the results. diff --git a/docs/source/tutorial-parallel-reproducible.rst b/docs/source/tutorial-parallel-reproducible.rst new file mode 100644 index 00000000..fc9521ee --- /dev/null +++ b/docs/source/tutorial-parallel-reproducible.rst @@ -0,0 +1,101 @@ +.. _tutorial-parallel-reproducible: + +Tutorial: parallel and reproducible runs +======================================== + +Experiments grow; wall-clock time should not. fl-sim can execute the +clients' local steps concurrently on a thread pool — with a one-line +config change — and this tutorial shows how to keep the run +*reproducible* while doing so: at the end, two different backends +produce **bitwise identical final model weights**. + +The full script lives at ``test-files/tutorial_parallel_check.py`` and +is exercised in CI. Prerequisites: :doc:`tutorial-first-experiment`. + +The mechanics behind this are documented in +:doc:`parallel_execution` (backends, guarantees, and how custom +algorithms stay compatible); this page is the hands-on version. + +One line of parallelism +----------------------- + +Add ``executor="threads"`` to the server config. Nothing else changes — +algorithms, datasets, logging and the participation schedule all behave +identically; only the per-client local steps (evaluate → local update → +build upload) run on a thread pool: + +.. literalinclude:: ../../test-files/tutorial_parallel_check.py + :language: python + :start-after: # --- begin: setup --- + :end-before: # --- end: setup --- + +The helpers capture what we will compare: the final model weights +(``weights_of``) and an exact equality check over them +(``bitwise_equal`` — ``torch.equal``, not ``allclose``). + +The two backends agree exactly +------------------------------ + +Run the same experiment on both backends with the client config's +``rng_mode="per_client"`` — each client's dataloaders get their own +generators seeded from ``(seed, client_id, round)`` — and compare: + +.. literalinclude:: ../../test-files/tutorial_parallel_check.py + :language: python + :start-after: # --- begin: per-client-check --- + :end-before: # --- end: per-client-check --- + +Output: + +.. code-block:: text + + per-client RNG: communications equal: True, final weights bitwise equal: True + +Bitwise identical — not "close", not "same accuracy": the same tensors. +This holds regardless of how many workers ran or how the operating +system scheduled them. + +What the global RNG mode buys (and costs) +----------------------------------------- + +``rng_mode`` defaults to ``"global"``: clients share the process-global +torch RNG. The script runs the same comparison in that mode too: + +.. literalinclude:: ../../test-files/tutorial_parallel_check.py + :language: python + :start-after: # --- begin: global-check --- + :end-before: # --- end: global-check --- + +Under the global mode the *participation schedule* and the *aggregation +arithmetic* are still backend-independent (sampling decisions and reply +consumption happen on the main thread in fixed order), so the +communication counts always match. The clients' local trajectories, +however, consume the shared RNG concurrently — so their interleaving +depends on thread scheduling, and the final weights may differ between +backends and between repeats: + +.. code-block:: text + + global RNG: communications equal: True, final weights bitwise equal: True (may be False by design) + +(On a small run like this one the tasks often serialize and even the +weights match — treat that as luck, not a guarantee.) + +The rule of thumb: + +- **exploring**: global mode is fine; run-to-run variation is part of + the measurement (average over seeds, as in + :doc:`tutorial-visualization`). +- **paper-grade seed alignment**: either stay on ``"sequential"``, or + set ``rng_mode="per_client"`` — then every backend, every parallelism + degree and every repeat produce the same trajectory from the same + seed. + +Next steps +---------- + +- :doc:`parallel_execution` — the full contract: guarantees per backend, + the ``FL_SIM_EXECUTOR``/``FL_SIM_NUM_WORKERS`` environment overrides, + measured speedups, and what custom algorithms must (not) touch. +- :doc:`tutorial-visualization` — plot the runs you just made + reproducible. diff --git a/docs/source/tutorial-visualization.rst b/docs/source/tutorial-visualization.rst new file mode 100644 index 00000000..8094abdc --- /dev/null +++ b/docs/source/tutorial-visualization.rst @@ -0,0 +1,79 @@ +.. _tutorial-visualization: + +Tutorial: visualize and export results +====================================== + +Every run writes ``.json`` logs (see :doc:`quickstart`); fl-sim ships a +visualization layer that turns them into publication-ready curves — +interactively in Jupyter, or programmatically from plain Python. This +tutorial does both: merge a multi-seed experiment into one mean curve +with error bounds, export it to PDF, and open the interactive dashboard. + +The full script lives at ``test-files/tutorial_visualization.py`` and is +exercised in CI (everything except the dashboard rendering, which needs +Jupyter). Prerequisites: :doc:`tutorial-first-experiment`. + +The reference for all knobs (fonts, smoothing, widgets) is +:doc:`viz`; this page is the workflow. + +Get some logs +------------- + +The script trains the same FedAvg experiment under two seeds — the +minimal "multi-seed" setup. Note the per-run ``tag``: log file names +carry it, so the two runs cannot overwrite each other (the same +reasoning as in :doc:`tutorial-cli-matrix`): + +.. literalinclude:: ../../test-files/tutorial_visualization.py + :language: python + :start-after: # --- begin: setup --- + :end-before: # --- end: setup --- + +Merge seeds into one curve +-------------------------- + +``find_log_files`` collects the JSON logs; ``aggregate_results_from_json_log_with_custom_axis`` +extracts a curve (``step`` on x, ``acc`` on y here — any recorded metric +works); ``plot_mean_curve_with_error_bounds`` merges the curves into a +mean with an uncertainty band (``"sem"``, ``"std"``, ``"quartile"`` or +``"iqr"``). Saving the figure is plain matplotlib: + +.. literalinclude:: ../../test-files/tutorial_visualization.py + :language: python + :start-after: # --- begin: plot-mean --- + :end-before: # --- end: plot-mean --- + +This writes ``fedavg-multi-seed.pdf`` next to your working directory. +Two things worth noticing: + +- The x axis is the **cumulative communication count** — the metric + :doc:`tutorial-first-experiment` dissected. Plotting against + communications (rather than rounds) is what makes different + communication-efficiency algorithms comparable in one figure. +- With only two seeds the band is a range, not a statistic — use more + seeds (just add more ``seed=`` runs; the merging code does not + change). + +Explore interactively +--------------------- + +In a Jupyter notebook, the ``Panel`` dashboard does all of the above +interactively: it lists the log files it finds, plots selectable +metrics, and gives you sliders for smoothing, fonts and figure size, +plus one-click export to PDF/SVG/PNG: + +.. literalinclude:: ../../test-files/tutorial_visualization.py + :language: python + :start-after: # --- begin: panel --- + :end-before: # --- end: panel --- + +Run this cell in a notebook (JupyterLab, Colab, SageMaker, …) and the +dashboard renders inline; see :doc:`viz` for a demo animation and the +full feature list. + +Next steps +---------- + +- :doc:`tutorial-parallel-reproducible` — make these runs faster + (parallel executor) without giving up reproducibility. +- :doc:`viz` — the reference for every visualization option. diff --git a/test-files/tutorial_first_experiment.py b/test-files/tutorial_first_experiment.py index 18ecc91a..a614ac0f 100644 --- a/test-files/tutorial_first_experiment.py +++ b/test-files/tutorial_first_experiment.py @@ -41,20 +41,23 @@ # one synthetic dataset, split across 30 clients, shared by every run below dataset = FedSynthetic(1.0, 1.0, True, 30) -model = dataset.candidate_models["mlp_d1"] def run(algorithm, sample_ratio, num_iters, tag, **extra_kw): """Train one algorithm and return the server. - Algorithm-specific server keywords (SCAFFOLD's global step size, - FedPD's upload frequency ``p``) come from ``SERVER_KW``; anything - passed as ``extra_kw`` is merged on top. + Every run takes a FRESH model from ``dataset.candidate_models`` + (a property building a new module on each access): the server trains + the model it is given in place, so sharing one model across runs + would warm-start every run after the first and contaminate the + comparison. Algorithm-specific server keywords (SCAFFOLD's global + step size, FedPD's upload frequency ``p``) come from ``SERVER_KW``; + anything passed as ``extra_kw`` is merged on top. """ server_cls, server_config_cls, client_config_cls = ALGORITHMS[algorithm] server_kw = {**SERVER_KW[algorithm], **extra_kw} server = server_cls( - model, + dataset.candidate_models["mlp_d1"], dataset, server_config_cls( num_iters=num_iters, diff --git a/test-files/tutorial_parallel_check.py b/test-files/tutorial_parallel_check.py new file mode 100644 index 00000000..29e455aa --- /dev/null +++ b/test-files/tutorial_parallel_check.py @@ -0,0 +1,74 @@ +""" +Parallelize a run and verify it stays reproducible. + +This script is the source of the code blocks shown in the +``tutorial-parallel-reproducible`` page of the documentation +(``docs/source/tutorial-parallel-reproducible.rst``): the marked sections +are included verbatim via Sphinx ``literalinclude`` directives, so the +docs and this script can never drift apart. The runs are pinned to the +tutorial budget (``num_iters <= 5``, ``num_clients <= 30``, one local +epoch), enforced by ``test/test_tutorial_scripts.py``. +""" + +# --- begin: setup --- +import torch + +from fl_sim.algorithms.fedopt import FedAvgClientConfig, FedAvgServer, FedAvgServerConfig +from fl_sim.data_processing.fed_synthetic import FedSynthetic + +dataset = FedSynthetic(1.0, 1.0, True, 30) + + +def run(executor, tag, rng_mode): + """One FedAvg run on the given executor backend.""" + return FedAvgServer( + # a fresh model per run (see the first-experiment tutorial) + dataset.candidate_models["mlp_d1"], + dataset, + FedAvgServerConfig( + num_iters=3, + num_clients=30, + clients_sample_ratio=0.5, + executor=executor, # "sequential" (default) or "threads" + num_workers=2, + log_dir=f"t4_{tag}", + ), + FedAvgClientConfig(batch_size=32, num_epochs=1, rng_mode=rng_mode), + ) + + +def weights_of(server): + return list(server.model.state_dict().values()) + + +def bitwise_equal(a, b): + return all(torch.equal(x, y) for x, y in zip(weights_of(a), weights_of(b))) + + +# --- end: setup --- + +# --- begin: per-client-check --- +sequential = run("sequential", "seq", rng_mode="per_client") +threads = run("threads", "threads", rng_mode="per_client") + +print( + f"per-client RNG: communications equal: " + f"{sequential._num_communications == threads._num_communications}, " + f"final weights bitwise equal: {bitwise_equal(sequential, threads)}" +) +# --- end: per-client-check --- + +# --- begin: global-check --- +sequential_g = run("sequential", "seq_global", rng_mode="global") +threads_g = run("threads", "threads_global", rng_mode="global") + +# with the process-global RNG, the participation sets and the aggregation +# arithmetic stay identical, but the clients' local trajectories consume +# the shared RNG concurrently, so the final weights may differ — this is +# the documented (and expected) statistical-level behaviour +print( + f"global RNG: communications equal: " + f"{sequential_g._num_communications == threads_g._num_communications}, " + f"final weights bitwise equal: {bitwise_equal(sequential_g, threads_g)} (may be False by design)" +) +# --- end: global-check --- diff --git a/test-files/tutorial_visualization.py b/test-files/tutorial_visualization.py new file mode 100644 index 00000000..1243961f --- /dev/null +++ b/test-files/tutorial_visualization.py @@ -0,0 +1,73 @@ +""" +Visualize and export experiment results. + +This script is the source of the code blocks shown in the +``tutorial-visualization`` page of the documentation +(``docs/source/tutorial-visualization.rst``): the marked sections are +included verbatim via Sphinx ``literalinclude`` directives, so the docs +and this script can never drift apart. The training runs below are +pinned to the tutorial budget (``num_iters <= 5``, ``num_clients <= 30``, +one local epoch), enforced by ``test/test_tutorial_scripts.py``. + +The two runs at the top stand in for the experiment logs produced by +the earlier tutorials (or your own experiments): everything after them +works the same on any log directory. + +The interactive part of the tutorial (the ``Panel`` dashboard) needs a +Jupyter notebook and is therefore only constructed, not rendered, here. +""" + +# --- begin: setup --- +from fl_sim.algorithms.fedopt import FedAvgClientConfig, FedAvgServer, FedAvgServerConfig +from fl_sim.data_processing.fed_synthetic import FedSynthetic + +dataset = FedSynthetic(1.0, 1.0, True, 30) + +# two seeds of the same experiment: exactly what "multi-seed" means +for seed in (0, 1): + server = FedAvgServer( + # a fresh model per run: the server trains the model it is given + # in place, so sharing one module would warm-start the second run + dataset.candidate_models["mlp_d1"], + dataset, + FedAvgServerConfig( + num_iters=3, + num_clients=30, + clients_sample_ratio=0.5, + seed=seed, + log_dir="t3_seeds", + tag=f"seed_{seed}", # distinct tag per seed: no log overwrites + ), + FedAvgClientConfig(batch_size=32, num_epochs=1), + ) + server.train() +# --- end: setup --- + +from pathlib import Path + +# --- begin: plot-mean --- +import matplotlib.pyplot as plt + +from fl_sim.utils.viz import aggregate_results_from_json_log_with_custom_axis, find_log_files + +# the runs above logged into the t3_seeds directory under fl-sim's log root +log_dir = Path(server._logger_manager.log_dir) +files = find_log_files(directory=log_dir, filters="seed_") +curves = [aggregate_results_from_json_log_with_custom_axis(f, part="val", x_metric="step", y_metric="acc") for f in files] + +fig, ax = plt.subplots(figsize=(6, 4)) +from fl_sim.utils.viz import plot_mean_curve_with_error_bounds + +fig, ax = plot_mean_curve_with_error_bounds(curves, error_type="sem", label="FedAvg", fig_ax=(fig, ax)) +ax.set_xlabel("cumulative communications") +ax.set_ylabel("val acc") +fig.savefig("fedavg-multi-seed.pdf", bbox_inches="tight") +# --- end: plot-mean --- + +# --- begin: panel --- +from fl_sim.utils.viz import Panel + +# in a Jupyter notebook this renders an interactive dashboard: +# pick log files, tune curves, export to PDF/SVG/PNG +panel = Panel(logdir="t3_seeds") +# --- end: panel --- diff --git a/test/test_tutorial_scripts.py b/test/test_tutorial_scripts.py index 59e2ac84..7e129777 100644 --- a/test/test_tutorial_scripts.py +++ b/test/test_tutorial_scripts.py @@ -33,6 +33,8 @@ TUTORIAL_SCRIPTS = { "quickstart": REPO_ROOT / "test-files" / "quickstart.py", "tutorial_first_experiment": REPO_ROOT / "test-files" / "tutorial_first_experiment.py", + "tutorial_visualization": REPO_ROOT / "test-files" / "tutorial_visualization.py", + "tutorial_parallel_check": REPO_ROOT / "test-files" / "tutorial_parallel_check.py", } CLI_MATRIX_DEMO = REPO_ROOT / "example-configs" / "fedsynthetic-demo.yml" @@ -44,6 +46,14 @@ def isolated_log_dir(tmp_path, monkeypatch): yield tmp_path +@pytest.fixture() +def in_tmp_cwd(tmp_path, monkeypatch): + """Run scripts from a temporary working directory, so any file they + write relative to the CWD (e.g. an exported PDF) stays contained.""" + monkeypatch.chdir(tmp_path) + yield tmp_path + + def _run_script(script: Path): """Execute a tutorial script in-process, restoring ``sys.path`` afterwards. @@ -148,6 +158,49 @@ def test_cli_matrix_demo_expands_and_runs(isolated_log_dir): _check_tutorial_budget(run_dir) +def test_visualization_script_merges_seeds_and_exports(isolated_log_dir, in_tmp_cwd): + namespace = _run_script(TUTORIAL_SCRIPTS["tutorial_visualization"]) + + # two tagged runs, both within the tutorial budget + run_dir = isolated_log_dir / "t3_seeds" + jsons = [p for p in run_dir.glob("*.json") if p.name != "config.json"] + assert len(jsons) == 2, "the two seeds must not overwrite each other's logs" + _check_tutorial_budget(run_dir) + + # the merge produced one non-empty curve per seed, and the PDF export + # landed in the (temporary) working directory + curves = namespace["curves"] + assert len(curves) == 2 + for curve in curves: + assert len(curve["x"]) > 0 and len(curve["y"]) > 0 + assert (in_tmp_cwd / "fedavg-multi-seed.pdf").exists(), "the mean curve should be exported to PDF" + + # the dashboard object is constructible outside Jupyter (it prints a + # note and stays inert there; rendering is covered by manual use) + assert namespace["panel"] is not None + + +def test_parallel_check_script_verifies_bitwise_reproducibility(isolated_log_dir): + namespace = _run_script(TUTORIAL_SCRIPTS["tutorial_parallel_check"]) + + # the L1 guarantee, demonstrated end to end: same seed, per-client RNG, + # two different executor backends -> bitwise identical final weights + sequential, threads = namespace["sequential"], namespace["threads"] + assert sequential._num_communications == threads._num_communications + assert namespace["bitwise_equal"]( + sequential, threads + ), "sequential and threads with rng_mode='per_client' must agree bitwise" + + # global mode: participation stays backend-independent; local + # trajectories may interleave by design, so only the counts are asserted + sequential_g, threads_g = namespace["sequential_g"], namespace["threads_g"] + assert sequential_g._num_communications == threads_g._num_communications + + for sub in isolated_log_dir.iterdir(): + if sub.name.startswith("t4_") and sub.is_dir(): + _check_tutorial_budget(sub) + + def test_tutorial_scripts_keep_literalinclude_markers(): """The markers delimit the fragments the docs include; dropping one silently empties a documentation code block.""" From d1551220dbec8b998e6a30377e9ea6fc846b9bd6 Mon Sep 17 00:00:00 2001 From: WEN Hao Date: Thu, 29 Oct 2026 00:08:59 +0800 Subject: [PATCH 2/2] docs: tighten the tone of the two new tutorial pages Align the phrasing with the rest of the documentation: "Experiments grow; wall-clock time should not" becomes a plain declarative opening, question/imperative headings become noun phrases ("One line of parallelism" -> "Enabling parallel execution", "Explore interactively" -> "Interactive exploration", "Get some logs" -> "Preparing the logs"), and metaphorical wording is replaced ("what the global RNG mode buys (and costs)" -> "Behavior of the global RNG mode", "treat that as luck, not a guarantee" -> "incidental rather than guaranteed", "the rule of thumb" -> "Practical guidance"). The zh_CN translations are reworded to match. Script comments and docstrings shown on the pages are adjusted in the same pass. No technical content changes. --- .../tutorial-parallel-reproducible.po | 155 ++++++------------ .../LC_MESSAGES/tutorial-visualization.po | 149 +++++++---------- .../source/tutorial-parallel-reproducible.rst | 38 ++--- docs/source/tutorial-visualization.rst | 35 ++-- test-files/tutorial_first_experiment.py | 8 +- test-files/tutorial_parallel_check.py | 2 +- test-files/tutorial_visualization.py | 8 +- 7 files changed, 156 insertions(+), 239 deletions(-) diff --git a/docs/source/locale/zh_CN/LC_MESSAGES/tutorial-parallel-reproducible.po b/docs/source/locale/zh_CN/LC_MESSAGES/tutorial-parallel-reproducible.po index 16a49b77..ea133b23 100644 --- a/docs/source/locale/zh_CN/LC_MESSAGES/tutorial-parallel-reproducible.po +++ b/docs/source/locale/zh_CN/LC_MESSAGES/tutorial-parallel-reproducible.po @@ -6,17 +6,17 @@ #, fuzzy msgid "" msgstr "" -"Project-Id-Version: fl-sim \n" +"Project-Id-Version: fl-sim\n" "Report-Msgid-Bugs-To: \n" -"POT-Creation-Date: 2026-10-28 07:53+0800\n" +"POT-Creation-Date: 2026-10-29 00:02+0800\n" "PO-Revision-Date: YEAR-MO-DA HO:MI+ZONE\n" "Last-Translator: FULL NAME \n" -"Language: zh_CN\n" "Language-Team: zh_CN \n" -"Plural-Forms: nplurals=1; plural=0;\n" +"Language: zh_CN\n" "MIME-Version: 1.0\n" "Content-Type: text/plain; charset=utf-8\n" "Content-Transfer-Encoding: 8bit\n" +"Plural-Forms: nplurals=1; plural=0;\n" "Generated-By: Babel 2.16.0\n" #: ../../source/tutorial-parallel-reproducible.rst:4 @@ -25,135 +25,82 @@ msgstr "教程：并行与可复现的运行" #: ../../source/tutorial-parallel-reproducible.rst:6 msgid "" -"Experiments grow; wall-clock time should not. fl-sim can execute the " -"clients' local steps concurrently on a thread pool — with a one-line " -"config change — and this tutorial shows how to keep the run " -"*reproducible* while doing so: at the end, two different backends produce" -" **bitwise identical final model weights**." -msgstr "" -"实验规模会变大，墙钟时间不该跟着变。fl-sim 能在线程池上并发执行客户端的本地步——只需" -"一行配置——本教程展示这样做的同时如何保持运行 *可复现* ：最后，两个不同的后端产出" -" **逐位一致的最终模型权重** 。" +"Larger experiments take longer to train. fl-sim can execute the clients' local steps concurrently on a thread pool — with a one-line config change — and this tutorial shows how to keep the run " +"*reproducible* while doing so: at the end, two different backends produce **bitwise identical final model weights**." +msgstr "实验规模越大，训练耗时越长。fl-sim 可以在线程池上并发执行客户端的本地步——只需一行配置——本教程展示这样做的同时如何保持运行 *可复现* ：最后，两个不同的后端产出 **逐位一致的最终模型权重** 。" #: ../../source/tutorial-parallel-reproducible.rst:12 -msgid "" -"The full script lives at ``test-files/tutorial_parallel_check.py`` and is" -" exercised in CI. Prerequisites: :doc:`tutorial-first-experiment`." -msgstr "" -"完整脚本位于 ``test-files/tutorial_parallel_check.py``，并在 CI 中被执行。" -"前置阅读： :doc:`tutorial-first-experiment` 。" +msgid "The full script lives at ``test-files/tutorial_parallel_check.py`` and is exercised in CI. Prerequisites: :doc:`tutorial-first-experiment`." +msgstr "完整脚本位于 ``test-files/tutorial_parallel_check.py``，并在 CI 中被执行。前置阅读： :doc:`tutorial-first-experiment` 。" #: ../../source/tutorial-parallel-reproducible.rst:15 -msgid "" -"The mechanics behind this are documented in :doc:`parallel_execution` " -"(backends, guarantees, and how custom algorithms stay compatible); this " -"page is the hands-on version." -msgstr "" -"背后的机制见 :doc:`parallel_execution` （后端、保证、自定义算法如何保持兼容）；" -"本页是动手版。" +msgid "The mechanics behind this are documented in :doc:`parallel_execution` (backends, guarantees, and how custom algorithms stay compatible); this page is the hands-on version." +msgstr "背后的机制见 :doc:`parallel_execution` （后端、保证、自定义算法如何保持兼容）；本页是动手版。" #: ../../source/tutorial-parallel-reproducible.rst:20 -msgid "One line of parallelism" -msgstr "一行开启并行" +msgid "Enabling parallel execution" +msgstr "开启并行执行" #: ../../source/tutorial-parallel-reproducible.rst:22 msgid "" -"Add ``executor=\"threads\"`` to the server config. Nothing else changes —" -" algorithms, datasets, logging and the participation schedule all behave " -"identically; only the per-client local steps (evaluate → local update → " -"build upload) run on a thread pool:" -msgstr "" -"在服务器配置里加 ``executor=\"threads\"`` 。其余一切不变——算法、数据集、日志、" -"参与调度全都原样；只有每个客户端的本地步（评估 → 本地更新 → 构建上传）" -"跑在线程池上：" +"Add ``executor=\"threads\"`` to the server config. Nothing else changes — algorithms, datasets, logging and the participation schedule all behave identically; only the per-client local steps " +"(evaluate → local update → build upload) run on a thread pool:" +msgstr "在服务器配置里加 ``executor=\"threads\"`` 。其余一切不变——算法、数据集、日志、参与调度全都原样；只有每个客户端的本地步（评估 → 本地更新 → 构建上传）跑在线程池上：" #: ../../source/tutorial-parallel-reproducible.rst:32 -msgid "" -"The helpers capture what we will compare: the final model weights " -"(``weights_of``) and an exact equality check over them (``bitwise_equal``" -" — ``torch.equal``, not ``allclose``)." -msgstr "" -"两个辅助函数负责我们要比较的东西：最终模型权重（ ``weights_of`` ）及其精确相等检查" -"（ ``bitwise_equal`` ——用 ``torch.equal`` ，不是 ``allclose`` ）。" +msgid "The helpers capture what we will compare: the final model weights (``weights_of``) and an exact equality check over them (``bitwise_equal`` — ``torch.equal``, not ``allclose``)." +msgstr "两个辅助函数负责我们要比较的东西：最终模型权重（ ``weights_of`` ）及其精确相等检查（ ``bitwise_equal`` ——用 ``torch.equal`` ，不是 ``allclose`` ）。" #: ../../source/tutorial-parallel-reproducible.rst:37 -msgid "The two backends agree exactly" -msgstr "两种后端完全一致" +msgid "The two backends produce identical results" +msgstr "两种后端产出完全一致的结果" #: ../../source/tutorial-parallel-reproducible.rst:39 msgid "" -"Run the same experiment on both backends with the client config's " -"``rng_mode=\"per_client\"`` — each client's dataloaders get their own " -"generators seeded from ``(seed, client_id, round)`` — and compare:" -msgstr "" -"把客户端配置设为 ``rng_mode=\"per_client\"`` （每个客户端的 dataloader 拿到由 " -"``(seed, client_id, round)`` 派生种子的专属生成器），在两种后端上跑同一实验并比较：" +"Run the same experiment on both backends with the client config's ``rng_mode=\"per_client\"`` — each client's dataloaders get their own generators seeded from ``(seed, client_id, round)`` — and " +"compare:" +msgstr "把客户端配置设为 ``rng_mode=\"per_client\"`` （每个客户端的 dataloader 拿到由 ``(seed, client_id, round)`` 派生种子的专属生成器），在两种后端上跑同一实验并比较：" #: ../../source/tutorial-parallel-reproducible.rst:48 msgid "Output:" msgstr "输出：" #: ../../source/tutorial-parallel-reproducible.rst:54 -msgid "" -"Bitwise identical — not \"close\", not \"same accuracy\": the same " -"tensors. This holds regardless of how many workers ran or how the " -"operating system scheduled them." -msgstr "" -"逐位一致——不是\"接近\"，也不是\"精度相同\"：就是同一批张量。无论跑了多少个 worker、" -"操作系统如何调度，这个结论都成立。" +msgid "The results are bitwise identical — not merely close, but the same tensors. This holds regardless of how many workers ran or how the operating system scheduled them." +msgstr "结果逐位一致——不只是\"接近\"，也不只是\"精度相同\"：就是同一批张量。无论运行了多少 worker、操作系统如何调度，这一结论都成立。" #: ../../source/tutorial-parallel-reproducible.rst:59 -msgid "What the global RNG mode buys (and costs)" -msgstr "全局 RNG 模式得到了什么（又付出了什么）" +msgid "Behavior of the global RNG mode" +msgstr "全局 RNG 模式的行为" #: ../../source/tutorial-parallel-reproducible.rst:61 -msgid "" -"``rng_mode`` defaults to ``\"global\"``: clients share the process-global" -" torch RNG. The script runs the same comparison in that mode too:" -msgstr "" -"``rng_mode`` 默认为 ``\"global\"`` ：客户端共享进程级全局 torch RNG。脚本在那种模式下" -"也跑了同样的比较：" +msgid "``rng_mode`` defaults to ``\"global\"``: clients share the process-global torch RNG. The script runs the same comparison in that mode too:" +msgstr "``rng_mode`` 默认为 ``\"global\"`` ：客户端共享进程级全局 torch RNG。脚本在那种模式下也跑了同样的比较：" #: ../../source/tutorial-parallel-reproducible.rst:69 msgid "" -"Under the global mode the *participation schedule* and the *aggregation " -"arithmetic* are still backend-independent (sampling decisions and reply " -"consumption happen on the main thread in fixed order), so the " -"communication counts always match. The clients' local trajectories, " -"however, consume the shared RNG concurrently — so their interleaving " -"depends on thread scheduling, and the final weights may differ between " -"backends and between repeats:" -msgstr "" -"全局模式下， *参与调度* 与 *聚合算术* 仍与后端无关（采样决策与回复消费都在主线程按固定" -"顺序进行），因此通信计数永远一致。但客户端的本地轨迹会并发消耗共享 RNG——交叠情况取决于" -"线程调度，最终权重可能因后端不同、重复运行不同而不同：" +"Under the global mode the *participation schedule* and the *aggregation arithmetic* are still backend-independent (sampling decisions and reply consumption happen on the main thread in fixed order)," +" so the communication counts always match. The clients' local trajectories, however, consume the shared RNG concurrently — so their interleaving depends on thread scheduling, and the final weights " +"may differ between backends and between repeats:" +msgstr "全局模式下， *参与调度* 与 *聚合算术* 仍与后端无关（采样决策与回复消费都在主线程按固定顺序进行），因此通信计数永远一致。但客户端的本地轨迹会并发消耗共享 RNG——交叠情况取决于线程调度，最终权重可能因后端不同、重复运行不同而不同：" #: ../../source/tutorial-parallel-reproducible.rst:81 -msgid "" -"(On a small run like this one the tasks often serialize and even the " -"weights match — treat that as luck, not a guarantee.)" -msgstr "（像这样的小规模运行，任务常常自动串行，连权重都恰好一致——那是运气，不是保证。）" +msgid "(On a small run like this one, the tasks often serialize and even the weights may match; this is incidental rather than guaranteed.)" +msgstr "（在这样的小规模运行中，任务常常自动串行，连权重都可能恰好一致；这是偶然现象，并非保证。）" #: ../../source/tutorial-parallel-reproducible.rst:84 -msgid "The rule of thumb:" -msgstr "经验法则：" +msgid "Practical guidance:" +msgstr "实践建议：" #: ../../source/tutorial-parallel-reproducible.rst:86 -msgid "" -"**exploring**: global mode is fine; run-to-run variation is part of the " -"measurement (average over seeds, as in :doc:`tutorial-visualization`)." -msgstr "" -" **探索阶段** ：全局模式没问题；运行间的波动本来就是测量的一部分（对多个 seed 取平均，" -"见 :doc:`tutorial-visualization` ）。" +msgid "**exploring**: global mode is acceptable; run-to-run variation is part of the measurement (average over seeds, as in :doc:`tutorial-visualization`)." +msgstr " **探索阶段** ：全局模式即可；运行间波动本来就是测量的一部分（对多个 seed 取平均，见 :doc:`tutorial-visualization` ）。" #: ../../source/tutorial-parallel-reproducible.rst:89 msgid "" -"**paper-grade seed alignment**: either stay on ``\"sequential\"``, or set" -" ``rng_mode=\"per_client\"`` — then every backend, every parallelism " -"degree and every repeat produce the same trajectory from the same seed." -msgstr "" -" **论文级种子对齐** ：要么保持 ``\"sequential\"``，要么设 ``rng_mode=\"per_client\"``" -"——之后同一种子在每个后端、每种并行度、每次重复下都走出同一条轨迹。" +"**paper-grade seed alignment**: either stay on ``\"sequential\"``, or set ``rng_mode=\"per_client\"`` — then every backend, every parallelism degree and every repeat produce the same trajectory from" +" the same seed." +msgstr " **论文级种子对齐** ：要么保持 ``\"sequential\"``，要么设 ``rng_mode=\"per_client\"``——之后同一种子在每个后端、每种并行度、每次重复下都走出同一条轨迹。" #: ../../source/tutorial-parallel-reproducible.rst:95 msgid "Next steps" @@ -161,13 +108,19 @@ msgstr "下一步" #: ../../source/tutorial-parallel-reproducible.rst:97 msgid "" -":doc:`parallel_execution` — the full contract: guarantees per backend, " -"the ``FL_SIM_EXECUTOR``/``FL_SIM_NUM_WORKERS`` environment overrides, " -"measured speedups, and what custom algorithms must (not) touch." -msgstr "" -":doc:`parallel_execution`——完整契约：每个后端的保证、 ``FL_SIM_EXECUTOR`` / " -"``FL_SIM_NUM_WORKERS`` 环境变量覆盖、实测加速比，以及自定义算法哪些能碰哪些不能碰。" +":doc:`parallel_execution` — the full contract: guarantees per backend, the ``FL_SIM_EXECUTOR``/``FL_SIM_NUM_WORKERS`` environment overrides, measured speedups, and what custom algorithms must (not) " +"touch." +msgstr ":doc:`parallel_execution`——完整契约：每个后端的保证、 ``FL_SIM_EXECUTOR`` / ``FL_SIM_NUM_WORKERS`` 环境变量覆盖、实测加速比，以及自定义算法哪些能碰哪些不能碰。" #: ../../source/tutorial-parallel-reproducible.rst:100 msgid ":doc:`tutorial-visualization` — plot the runs you just made reproducible." msgstr ":doc:`tutorial-visualization`——把刚变得可复现的这些运行画出来。" + +#~ msgid "One line of parallelism" +#~ msgstr "一行开启并行" + +#~ msgid "The two backends agree exactly" +#~ msgstr "两种后端完全一致" + +#~ msgid "The rule of thumb:" +#~ msgstr "经验法则：" diff --git a/docs/source/locale/zh_CN/LC_MESSAGES/tutorial-visualization.po b/docs/source/locale/zh_CN/LC_MESSAGES/tutorial-visualization.po index 98cc8da1..8f6ed96c 100644 --- a/docs/source/locale/zh_CN/LC_MESSAGES/tutorial-visualization.po +++ b/docs/source/locale/zh_CN/LC_MESSAGES/tutorial-visualization.po @@ -6,17 +6,17 @@ #, fuzzy msgid "" msgstr "" -"Project-Id-Version: fl-sim \n" +"Project-Id-Version: fl-sim\n" "Report-Msgid-Bugs-To: \n" -"POT-Creation-Date: 2026-10-28 07:53+0800\n" +"POT-Creation-Date: 2026-10-29 00:02+0800\n" "PO-Revision-Date: YEAR-MO-DA HO:MI+ZONE\n" "Last-Translator: FULL NAME \n" -"Language: zh_CN\n" "Language-Team: zh_CN \n" -"Plural-Forms: nplurals=1; plural=0;\n" +"Language: zh_CN\n" "MIME-Version: 1.0\n" "Content-Type: text/plain; charset=utf-8\n" "Content-Transfer-Encoding: 8bit\n" +"Plural-Forms: nplurals=1; plural=0;\n" "Generated-By: Babel 2.16.0\n" #: ../../source/tutorial-visualization.rst:4 @@ -25,121 +25,84 @@ msgstr "教程：可视化与导出结果" #: ../../source/tutorial-visualization.rst:6 msgid "" -"Every run writes ``.json`` logs (see :doc:`quickstart`); fl-sim ships a " -"visualization layer that turns them into publication-ready curves — " -"interactively in Jupyter, or programmatically from plain Python. This " -"tutorial does both: merge a multi-seed experiment into one mean curve " -"with error bounds, export it to PDF, and open the interactive dashboard." -msgstr "" -"每次运行都会写 ``.json`` 日志（见 :doc:`quickstart` ）；fl-sim 自带的可视化层能把它们变成" -"可直接发表的曲线——在 Jupyter 里交互式操作，或在纯 Python 里以编程方式处理。本教程两者" -"都做：把多 seed 实验合并成一条带误差带的均值曲线，导出 PDF，再打开交互式仪表盘。" +"Every run writes ``.json`` logs (see :doc:`quickstart`); fl-sim provides a visualization layer that turns them into publication-ready curves, both interactively in Jupyter and programmatically from " +"plain Python. This tutorial covers both: merging a multi-seed experiment into one mean curve with error bounds, exporting it to PDF, and opening the interactive dashboard." +msgstr "每次运行都会写 ``.json`` 日志（见 :doc:`quickstart` ）；fl-sim 提供一个可视化层，把它们变成可直接发表的曲线——既可在 Jupyter 中交互操作，也可在纯 Python 中以编程方式处理。本教程两者都讲：把多 seed 实验合并成一条带误差带的均值曲线，导出 PDF，再打开交互式仪表盘。" -#: ../../source/tutorial-visualization.rst:12 +#: ../../source/tutorial-visualization.rst:13 msgid "" -"The full script lives at ``test-files/tutorial_visualization.py`` and is " -"exercised in CI (everything except the dashboard rendering, which needs " -"Jupyter). Prerequisites: :doc:`tutorial-first-experiment`." -msgstr "" -"完整脚本位于 ``test-files/tutorial_visualization.py``，并在 CI 中被执行（仪表盘渲染除外，" -"它需要 Jupyter）。前置阅读： :doc:`tutorial-first-experiment` 。" +"The full script lives at ``test-files/tutorial_visualization.py`` and is exercised in CI (everything except the dashboard rendering, which needs Jupyter). Prerequisites: :doc:`tutorial-first-" +"experiment`." +msgstr "完整脚本位于 ``test-files/tutorial_visualization.py``，并在 CI 中被执行（仪表盘渲染除外，它需要 Jupyter）。前置阅读： :doc:`tutorial-first-experiment` 。" -#: ../../source/tutorial-visualization.rst:16 -msgid "" -"The reference for all knobs (fonts, smoothing, widgets) is :doc:`viz`; " -"this page is the workflow." +#: ../../source/tutorial-visualization.rst:17 +msgid "The reference for all knobs (fonts, smoothing, widgets) is :doc:`viz`; this page is the workflow." msgstr "所有可调项（字体、平滑、控件）的参考见 :doc:`viz` ；本页讲的是工作流程。" -#: ../../source/tutorial-visualization.rst:20 -msgid "Get some logs" +#: ../../source/tutorial-visualization.rst:21 +msgid "Preparing the logs" msgstr "准备日志" -#: ../../source/tutorial-visualization.rst:22 +#: ../../source/tutorial-visualization.rst:23 msgid "" -"The script trains the same FedAvg experiment under two seeds — the " -"minimal \"multi-seed\" setup. Note the per-run ``tag``: log file names " -"carry it, so the two runs cannot overwrite each other (the same reasoning" -" as in :doc:`tutorial-cli-matrix`):" -msgstr "" -"脚本用两个种子训练同一个 FedAvg 实验——最小的\"多 seed\"设置。注意每次运行的 " -"``tag`` ：日志文件名携带它，两次运行不会互相覆盖（道理同 :doc:`tutorial-cli-matrix` ）：" +"The script trains the same FedAvg experiment under two seeds — the minimal \"multi-seed\" setup. Note the per-run ``tag``: log file names carry it, so the two runs cannot overwrite each other (the " +"same reasoning as in :doc:`tutorial-cli-matrix`):" +msgstr "脚本用两个种子训练同一个 FedAvg 实验——最小的\"多 seed\"设置。注意每次运行的 ``tag`` ：日志文件名携带它，两次运行不会互相覆盖（道理同 :doc:`tutorial-cli-matrix` ）：" -#: ../../source/tutorial-visualization.rst:33 -msgid "Merge seeds into one curve" +#: ../../source/tutorial-visualization.rst:34 +msgid "Merging seeds into one curve" msgstr "把多个 seed 合并成一条曲线" -#: ../../source/tutorial-visualization.rst:35 +#: ../../source/tutorial-visualization.rst:36 msgid "" -"``find_log_files`` collects the JSON logs; " -"``aggregate_results_from_json_log_with_custom_axis`` extracts a curve " -"(``step`` on x, ``acc`` on y here — any recorded metric works); " -"``plot_mean_curve_with_error_bounds`` merges the curves into a mean with " -"an uncertainty band (``\"sem\"``, ``\"std\"``, ``\"quartile\"`` or " -"``\"iqr\"``). Saving the figure is plain matplotlib:" +"``find_log_files`` collects the JSON logs; ``aggregate_results_from_json_log_with_custom_axis`` extracts a curve (``step`` on x, ``acc`` on y here — any recorded metric works); " +"``plot_mean_curve_with_error_bounds`` merges the curves into a mean with an uncertainty band (``\"sem\"``, ``\"std\"``, ``\"quartile\"`` or ``\"iqr\"``). Saving the figure is plain matplotlib:" msgstr "" -"``find_log_files`` 收集 JSON 日志； ``aggregate_results_from_json_log_with_custom_axis`` " -"抽取曲线（这里 x 轴 ``step``、y 轴 ``acc``——任何记录过的指标都行）； " -"``plot_mean_curve_with_error_bounds`` 把曲线合并成带不确定度带的均值（ ``\"sem\"`` 、 " -"``\"std\"`` 、 ``\"quartile\"`` 或 ``\"iqr\"`` ）。存图就是普通的 matplotlib：" +"``find_log_files`` 收集 JSON 日志； ``aggregate_results_from_json_log_with_custom_axis`` 抽取曲线（这里 x 轴 ``step``、y 轴 ``acc``——任何记录过的指标都行）； ``plot_mean_curve_with_error_bounds`` 把曲线合并成带不确定度带的均值（ ``\"sem\"`` " +"、 ``\"std\"`` 、 ``\"quartile\"`` 或 ``\"iqr\"`` ）。存图就是普通的 matplotlib：" -#: ../../source/tutorial-visualization.rst:46 -msgid "" -"This writes ``fedavg-multi-seed.pdf`` next to your working directory. Two" -" things worth noticing:" -msgstr "这会在工作目录旁写出 ``fedavg-multi-seed.pdf`` 。两点值得注意：" +#: ../../source/tutorial-visualization.rst:47 +msgid "This writes ``fedavg-multi-seed.pdf`` next to the working directory. Two remarks:" +msgstr "这会在工作目录旁写出 ``fedavg-multi-seed.pdf`` 。两点说明：" -#: ../../source/tutorial-visualization.rst:49 +#: ../../source/tutorial-visualization.rst:50 msgid "" -"The x axis is the **cumulative communication count** — the metric :doc" -":`tutorial-first-experiment` dissected. Plotting against communications " -"(rather than rounds) is what makes different communication-efficiency " -"algorithms comparable in one figure." -msgstr "" -"x 轴是 **累计通信次数** ——即 :doc:`tutorial-first-experiment` 剖析过的那个量。" -"以通信（而不是轮数）为横轴，才能把通信效率不同的算法放进同一张图里比较。" +"The x axis is the **cumulative communication count** — the metric :doc:`tutorial-first-experiment` dissected. Plotting against communications (rather than rounds) is what makes different " +"communication-efficiency algorithms comparable in one figure." +msgstr "x 轴是 **累计通信次数** ——即 :doc:`tutorial-first-experiment` 剖析过的那个量。以通信（而不是轮数）为横轴，才能把通信效率不同的算法放进同一张图里比较。" -#: ../../source/tutorial-visualization.rst:53 -msgid "" -"With only two seeds the band is a range, not a statistic — use more seeds" -" (just add more ``seed=`` runs; the merging code does not change)." -msgstr "" -"只有两个 seed 时，带子只是一个范围，还不是统计量——多加几个 ``seed=`` 运行即可" -"（合并代码一行都不用改）。" +#: ../../source/tutorial-visualization.rst:54 +msgid "With only two seeds the band is a range, not a statistic — use more seeds (just add more ``seed=`` runs; the merging code does not change)." +msgstr "只有两个 seed 时，带子只是一个范围，还不是统计量——多加几个 ``seed=`` 运行即可（合并代码一行都不用改）。" -#: ../../source/tutorial-visualization.rst:58 -msgid "Explore interactively" +#: ../../source/tutorial-visualization.rst:59 +msgid "Interactive exploration" msgstr "交互式探索" -#: ../../source/tutorial-visualization.rst:60 +#: ../../source/tutorial-visualization.rst:61 msgid "" -"In a Jupyter notebook, the ``Panel`` dashboard does all of the above " -"interactively: it lists the log files it finds, plots selectable metrics," -" and gives you sliders for smoothing, fonts and figure size, plus one-" -"click export to PDF/SVG/PNG:" -msgstr "" -"在 Jupyter notebook 里， ``Panel`` 仪表盘把上面的所有事都变成交互式的：列出找到的日志" -"文件、绘制可选的指标，提供平滑、字体与图幅的滑块，以及一键导出 PDF/SVG/PNG：" +"In a Jupyter notebook, the ``Panel`` dashboard performs all of the above interactively: it lists the log files it finds, plots selectable metrics, and provides sliders for smoothing, fonts and " +"figure size, plus export to PDF/SVG/PNG:" +msgstr "在 Jupyter notebook 里， ``Panel`` 仪表盘以上全部可交互完成：列出找到的日志文件、绘制可选指标，提供平滑、字体与图幅的滑块，并可导出 PDF/SVG/PNG：" -#: ../../source/tutorial-visualization.rst:70 -msgid "" -"Run this cell in a notebook (JupyterLab, Colab, SageMaker, …) and the " -"dashboard renders inline; see :doc:`viz` for a demo animation and the " -"full feature list." -msgstr "" -"在 notebook（JupyterLab、Colab、SageMaker 等）里运行这个单元格，仪表盘就会内嵌渲染；" -"演示动画与完整功能列表见 :doc:`viz` 。" +#: ../../source/tutorial-visualization.rst:71 +msgid "Run this cell in a notebook (JupyterLab, Colab, SageMaker, …) and the dashboard renders inline; see :doc:`viz` for a demo animation and the full feature list." +msgstr "在 notebook（JupyterLab、Colab、SageMaker 等）里运行这个单元格，仪表盘就会内嵌渲染；演示动画与完整功能列表见 :doc:`viz` 。" -#: ../../source/tutorial-visualization.rst:75 +#: ../../source/tutorial-visualization.rst:76 msgid "Next steps" msgstr "下一步" -#: ../../source/tutorial-visualization.rst:77 -msgid "" -":doc:`tutorial-parallel-reproducible` — make these runs faster (parallel " -"executor) without giving up reproducibility." -msgstr "" -":doc:`tutorial-parallel-reproducible`——让这些运行更快（并行执行器），且不放弃可复现性。" +#: ../../source/tutorial-visualization.rst:78 +msgid ":doc:`tutorial-parallel-reproducible` — make these runs faster (parallel executor) without giving up reproducibility." +msgstr ":doc:`tutorial-parallel-reproducible`——让这些运行更快（并行执行器），且不放弃可复现性。" -#: ../../source/tutorial-visualization.rst:79 +#: ../../source/tutorial-visualization.rst:80 msgid ":doc:`viz` — the reference for every visualization option." msgstr ":doc:`viz`——所有可视化选项的参考。" + +#~ msgid "Get some logs" +#~ msgstr "准备日志" + +#~ msgid "Explore interactively" +#~ msgstr "交互式探索" diff --git a/docs/source/tutorial-parallel-reproducible.rst b/docs/source/tutorial-parallel-reproducible.rst index fc9521ee..0ffcb4b6 100644 --- a/docs/source/tutorial-parallel-reproducible.rst +++ b/docs/source/tutorial-parallel-reproducible.rst @@ -3,11 +3,11 @@ Tutorial: parallel and reproducible runs ======================================== -Experiments grow; wall-clock time should not. fl-sim can execute the -clients' local steps concurrently on a thread pool — with a one-line -config change — and this tutorial shows how to keep the run -*reproducible* while doing so: at the end, two different backends -produce **bitwise identical final model weights**. +Larger experiments take longer to train. fl-sim can execute the clients' +local steps concurrently on a thread pool — with a one-line config change +— and this tutorial shows how to keep the run *reproducible* while doing +so: at the end, two different backends produce **bitwise identical final +model weights**. The full script lives at ``test-files/tutorial_parallel_check.py`` and is exercised in CI. Prerequisites: :doc:`tutorial-first-experiment`. @@ -16,8 +16,8 @@ The mechanics behind this are documented in :doc:`parallel_execution` (backends, guarantees, and how custom algorithms stay compatible); this page is the hands-on version. -One line of parallelism ------------------------ +Enabling parallel execution +--------------------------- Add ``executor="threads"`` to the server config. Nothing else changes — algorithms, datasets, logging and the participation schedule all behave @@ -33,8 +33,8 @@ The helpers capture what we will compare: the final model weights (``weights_of``) and an exact equality check over them (``bitwise_equal`` — ``torch.equal``, not ``allclose``). -The two backends agree exactly ------------------------------- +The two backends produce identical results +------------------------------------------ Run the same experiment on both backends with the client config's ``rng_mode="per_client"`` — each client's dataloaders get their own @@ -51,12 +51,12 @@ Output: per-client RNG: communications equal: True, final weights bitwise equal: True -Bitwise identical — not "close", not "same accuracy": the same tensors. -This holds regardless of how many workers ran or how the operating -system scheduled them. +The results are bitwise identical — not merely close, but the same +tensors. This holds regardless of how many workers ran or how the +operating system scheduled them. -What the global RNG mode buys (and costs) ------------------------------------------ +Behavior of the global RNG mode +------------------------------- ``rng_mode`` defaults to ``"global"``: clients share the process-global torch RNG. The script runs the same comparison in that mode too: @@ -78,13 +78,13 @@ backends and between repeats: global RNG: communications equal: True, final weights bitwise equal: True (may be False by design) -(On a small run like this one the tasks often serialize and even the -weights match — treat that as luck, not a guarantee.) +(On a small run like this one, the tasks often serialize and even the +weights may match; this is incidental rather than guaranteed.) -The rule of thumb: +Practical guidance: -- **exploring**: global mode is fine; run-to-run variation is part of - the measurement (average over seeds, as in +- **exploring**: global mode is acceptable; run-to-run variation is part + of the measurement (average over seeds, as in :doc:`tutorial-visualization`). - **paper-grade seed alignment**: either stay on ``"sequential"``, or set ``rng_mode="per_client"`` — then every backend, every parallelism diff --git a/docs/source/tutorial-visualization.rst b/docs/source/tutorial-visualization.rst index 8094abdc..171604d1 100644 --- a/docs/source/tutorial-visualization.rst +++ b/docs/source/tutorial-visualization.rst @@ -3,11 +3,12 @@ Tutorial: visualize and export results ====================================== -Every run writes ``.json`` logs (see :doc:`quickstart`); fl-sim ships a -visualization layer that turns them into publication-ready curves — -interactively in Jupyter, or programmatically from plain Python. This -tutorial does both: merge a multi-seed experiment into one mean curve -with error bounds, export it to PDF, and open the interactive dashboard. +Every run writes ``.json`` logs (see :doc:`quickstart`); fl-sim provides a +visualization layer that turns them into publication-ready curves, both +interactively in Jupyter and programmatically from plain Python. This +tutorial covers both: merging a multi-seed experiment into one mean curve +with error bounds, exporting it to PDF, and opening the interactive +dashboard. The full script lives at ``test-files/tutorial_visualization.py`` and is exercised in CI (everything except the dashboard rendering, which needs @@ -16,8 +17,8 @@ Jupyter). Prerequisites: :doc:`tutorial-first-experiment`. The reference for all knobs (fonts, smoothing, widgets) is :doc:`viz`; this page is the workflow. -Get some logs -------------- +Preparing the logs +------------------ The script trains the same FedAvg experiment under two seeds — the minimal "multi-seed" setup. Note the per-run ``tag``: log file names @@ -29,8 +30,8 @@ reasoning as in :doc:`tutorial-cli-matrix`): :start-after: # --- begin: setup --- :end-before: # --- end: setup --- -Merge seeds into one curve --------------------------- +Merging seeds into one curve +---------------------------- ``find_log_files`` collects the JSON logs; ``aggregate_results_from_json_log_with_custom_axis`` extracts a curve (``step`` on x, ``acc`` on y here — any recorded metric @@ -43,8 +44,8 @@ mean with an uncertainty band (``"sem"``, ``"std"``, ``"quartile"`` or :start-after: # --- begin: plot-mean --- :end-before: # --- end: plot-mean --- -This writes ``fedavg-multi-seed.pdf`` next to your working directory. -Two things worth noticing: +This writes ``fedavg-multi-seed.pdf`` next to the working directory. +Two remarks: - The x axis is the **cumulative communication count** — the metric :doc:`tutorial-first-experiment` dissected. Plotting against @@ -54,13 +55,13 @@ Two things worth noticing: seeds (just add more ``seed=`` runs; the merging code does not change). -Explore interactively ---------------------- +Interactive exploration +----------------------- -In a Jupyter notebook, the ``Panel`` dashboard does all of the above -interactively: it lists the log files it finds, plots selectable -metrics, and gives you sliders for smoothing, fonts and figure size, -plus one-click export to PDF/SVG/PNG: +In a Jupyter notebook, the ``Panel`` dashboard performs all of the above +interactively: it lists the log files it finds, plots selectable metrics, +and provides sliders for smoothing, fonts and figure size, plus export to +PDF/SVG/PNG: .. literalinclude:: ../../test-files/tutorial_visualization.py :language: python diff --git a/test-files/tutorial_first_experiment.py b/test-files/tutorial_first_experiment.py index a614ac0f..f68e624c 100644 --- a/test-files/tutorial_first_experiment.py +++ b/test-files/tutorial_first_experiment.py @@ -5,14 +5,14 @@ ``tutorial-first-experiment`` page of the documentation (``docs/source/tutorial-first-experiment.rst``): the marked sections are included verbatim via Sphinx ``literalinclude`` directives, so the docs -and this script can never drift apart. The size is pinned to the tutorial +and this script cannot drift apart. The size is pinned to the tutorial budget (``num_iters <= 5``, ``num_clients <= 30``, one local epoch), enforced by ``test/test_tutorial_scripts.py``. -The dataset is built ONCE, before any config/server, on purpose: config +The dataset is built once, before any config or server: config constructors re-seed the global RNG when they run, so whatever was built -last would otherwise shift the results. Construct in the order -dataset -> config -> server and the numbers are stable. +last would otherwise shift the results. Constructing in the order +dataset -> config -> server keeps the results reproducible. """ # --- begin: setup --- diff --git a/test-files/tutorial_parallel_check.py b/test-files/tutorial_parallel_check.py index 29e455aa..6f26c0a9 100644 --- a/test-files/tutorial_parallel_check.py +++ b/test-files/tutorial_parallel_check.py @@ -5,7 +5,7 @@ ``tutorial-parallel-reproducible`` page of the documentation (``docs/source/tutorial-parallel-reproducible.rst``): the marked sections are included verbatim via Sphinx ``literalinclude`` directives, so the -docs and this script can never drift apart. The runs are pinned to the +docs and this script cannot drift apart. The runs are pinned to the tutorial budget (``num_iters <= 5``, ``num_clients <= 30``, one local epoch), enforced by ``test/test_tutorial_scripts.py``. """ diff --git a/test-files/tutorial_visualization.py b/test-files/tutorial_visualization.py index 1243961f..e222780e 100644 --- a/test-files/tutorial_visualization.py +++ b/test-files/tutorial_visualization.py @@ -9,9 +9,9 @@ pinned to the tutorial budget (``num_iters <= 5``, ``num_clients <= 30``, one local epoch), enforced by ``test/test_tutorial_scripts.py``. -The two runs at the top stand in for the experiment logs produced by -the earlier tutorials (or your own experiments): everything after them -works the same on any log directory. +The two runs at the top produce the experiment logs used below; the same +code works on any log directory, including those from the earlier +tutorials. The interactive part of the tutorial (the ``Panel`` dashboard) needs a Jupyter notebook and is therefore only constructed, not rendered, here. @@ -23,7 +23,7 @@ dataset = FedSynthetic(1.0, 1.0, True, 30) -# two seeds of the same experiment: exactly what "multi-seed" means +# two seeds of the same experiment: the minimal multi-seed setup for seed in (0, 1): server = FedAvgServer( # a fresh model per run: the server trains the model it is given