diff --git a/docs/source/index.rst b/docs/source/index.rst index bdb96ede..6fb62e21 100644 --- a/docs/source/index.rst +++ b/docs/source/index.rst @@ -24,6 +24,8 @@ Project Links: `GitHub `_ | `gitee \n" "Language: zh_CN\n" @@ -31,9 +31,9 @@ msgid "" "ratios, then dissect what \"communication cost\" actually means in fl-" "sim." msgstr "" -":doc:`Quickstart ` 只跑了一次训练。真正的实验要在相同条件下比较不同配置，" -"并把结果读回来。本教程将比较三个算法在两种采样比例下的表现，然后剖析 fl-sim 中" -"\"通信开销\"究竟指什么。" +":doc:`Quickstart ` " +"只跑了一次训练。真正的实验要在相同条件下比较不同配置，并把结果读回来。本教程将比较三个算法在两种采样比例下的表现，然后剖析 fl-sim " +"中\"通信开销\"究竟指什么。" #: ../../source/tutorial-first-experiment.rst:11 msgid "" @@ -42,17 +42,15 @@ msgid "" "downloads, seconds per run. The full script shown below lives at ``test-" "files/tutorial_first_experiment.py`` and is exercised in CI." msgstr "" -"全部运行基于 :class:`~fl_sim.data_processing.fed_synthetic.FedSynthetic`——" -"无需下载，每次运行只需几秒。下文展示的完整脚本位于 " -"``test-files/tutorial_first_experiment.py``，并在 CI 中被执行。" +"全部运行基于 " +":class:`~fl_sim.data_processing.fed_synthetic.FedSynthetic`——无需下载，每次运行只需几秒。下文展示的完整脚本位于" +" ``test-files/tutorial_first_experiment.py``，并在 CI 中被执行。" #: ../../source/tutorial-first-experiment.rst:15 msgid "" "Prerequisites: :doc:`quickstart` (the three-object model: dataset, model," " server) and :doc:`concepts` (what a round does)." -msgstr "" -"前置阅读： :doc:`quickstart` （三对象模型：数据集、模型、服务器）与 " -" :doc:`concepts` （一轮训练做了什么）。" +msgstr "前置阅读： :doc:`quickstart` （三对象模型：数据集、模型、服务器）与 :doc:`concepts` （一轮训练做了什么）。" #: ../../source/tutorial-first-experiment.rst:19 msgid "Set up the comparison" @@ -65,9 +63,8 @@ msgid "" "gradient, proximal and variance-reduction axes — each at **sampling " "ratios 0.5 and 1.0**." msgstr "" -"六次运行： **FedAvg** （朴素平均）、 **FedProx** （近端正则的本地目标）与 " -" **SCAFFOLD** （控制变量）——梯度、近端、方差缩减三条经典轴线——各自在 " -" **采样比例 0.5 与 1.0** 下运行。" +"六次运行： **FedAvg** （朴素平均）、 **FedProx** （近端正则的本地目标）与 **SCAFFOLD** " +"（控制变量）——梯度、近端、方差缩减三条经典轴线——各自在 **采样比例 0.5 与 1.0** 下运行。" #: ../../source/tutorial-first-experiment.rst:26 msgid "" @@ -77,9 +74,8 @@ msgid "" "sampling stream of everything after it. The order *dataset → config → " "server* keeps every run reproducible:" msgstr "" -"一个刻意的选择：数据集 **只构造一次，且先于任何配置或服务器** 。配置类的构造器在" -"构建时会重置全局 RNG，谁最后构造谁生效，其后的采样流都会被移动。按 " -"*数据集 → 配置 → 服务器* 的顺序构造，每次运行都可复现：" +"一个刻意的选择：数据集 **只构造一次，且先于任何配置或服务器** 。配置类的构造器在构建时会重置全局 " +"RNG，谁最后构造谁生效，其后的采样流都会被移动。按 *数据集 → 配置 → 服务器* 的顺序构造，每次运行都可复现：" #: ../../source/tutorial-first-experiment.rst:37 msgid "" @@ -89,9 +85,9 @@ msgid "" "frequency parameter later. Both are per-algorithm server keywords, " "collected in ``SERVER_KW``." msgstr "" -"``final_accuracy`` 读取一次运行的 JSON 日志——与 :doc:`quickstart` 中相同的文件" -"格式。SCAFFOLD 需要显式传入全局步长 ``lr``（它的配置没有默认值）；FedPD 稍后也需要" -"一个频率参数。两者都是算法各自的服务器关键字，统一收在 ``SERVER_KW`` 里。" +"``final_accuracy`` 读取一次运行的 JSON 日志——与 :doc:`quickstart` 中相同的文件格式。SCAFFOLD" +" 需要显式传入全局步长 ``lr``（它的配置没有默认值）；FedPD 稍后也需要一个频率参数。两者都是算法各自的服务器关键字，统一收在 " +"``SERVER_KW`` 里。" #: ../../source/tutorial-first-experiment.rst:44 msgid "Run and read the results" @@ -120,9 +116,9 @@ msgid "" "harder, more heterogeneous tasks (see the :doc:`algorithm theory pages " "` for when each helps)." msgstr "" -"这个合成任务 *太容易* ——全采样下 FedProx 与 SCAFFOLD 达到了和 FedAvg 相同的精度" -"天花板。它们的优势要在更难、异质性更强的任务上才显现（何时用哪个，见 " -":doc:`算法理论页 ` ）。" +"这个合成任务 *太容易* ——全采样下 FedProx 与 SCAFFOLD 达到了和 FedAvg " +"相同的精度天花板。它们的优势要在更难、异质性更强的任务上才显现（何时用哪个，见 :doc:`算法理论页 " +"` ）。" #: ../../source/tutorial-first-experiment.rst:74 msgid "" @@ -131,8 +127,8 @@ msgid "" "1.0 used exactly 180 — regardless of the algorithm. That number deserves " "a closer look, because it is easy to misread." msgstr "" -"精度的故事饱和了， **通信** 的故事没有：比例 0.5 的每次运行都恰好用了 90 次通信，" -"比例 1.0 的都恰好 180 次——与算法无关。这个数字值得细看，因为它很容易被误读。" +"精度的故事饱和了， **通信** 的故事没有：比例 0.5 的每次运行都恰好用了 90 次通信，比例 1.0 的都恰好 180 " +"次——与算法无关。这个数字值得细看，因为它很容易被误读。" #: ../../source/tutorial-first-experiment.rst:80 msgid "What ``num_communications`` counts" @@ -147,8 +143,8 @@ msgid "" " doubles to 180." msgstr "" "``num_communications`` 统计的是 **消息数，不是轮数** ：每轮对 *每个被采样客户端* " -"加一次下行广播，再加一次上行回复。30 个客户端、比例 0.5 时，每轮是 15 下行 + " -"15 上行 = 30 条消息，3 轮 = 90；比例 1.0 时翻倍为 180。" +"加一次下行广播，再加一次上行回复。30 个客户端、比例 0.5 时，每轮是 15 下行 + 15 上行 = 30 条消息，3 轮 = 90；比例 " +"1.0 时翻倍为 180。" #: ../../source/tutorial-first-experiment.rst:88 msgid "" @@ -176,8 +172,8 @@ msgid "" "round's full cost — read the final count from the server, as the scripts " "here do.)" msgstr "" -"（另注：JSON 日志里记录的 ``step`` 是 *每轮开始时* 的快照：它比最终计数少最后一轮的" -"全部开销——最终值请从服务器读取，正如本页脚本的做法。）" +"（另注：JSON 日志里记录的 ``step`` 是 *每轮开始时* " +"的快照：它比最终计数少最后一轮的全部开销——最终值请从服务器读取，正如本页脚本的做法。）" #: ../../source/tutorial-first-experiment.rst:106 msgid "Sampling is not communicating" @@ -191,9 +187,8 @@ msgid "" "every ``int(1/p)``-th round — for five rounds next to a five-round " "FedAvg:" msgstr "" -"上面三个算法每一轮都在通信。fl-sim 还内置 **可跳过通信** 的算法，它们会以很有启发性的" -"方式打破 ``2 × 采样数 × 轮数`` 的恒等式。让 FedPD（每 ``int(1/p)`` 轮才上传一次）" -"与五轮 FedAvg 并排各跑五轮：" +"上面三个算法每一轮都在通信。fl-sim 还内置 **可跳过通信** 的算法，它们会以很有启发性的方式打破 ``2 × 采样数 × 轮数`` " +"的恒等式。让 FedPD（每 ``int(1/p)`` 轮才上传一次）与五轮 FedAvg 并排各跑五轮：" #: ../../source/tutorial-first-experiment.rst:119 msgid "Output on one machine:" @@ -207,9 +202,8 @@ msgid "" "messages as three FedAvg rounds — the same communication budget buys two " "extra local iterations." msgstr "" -"分解式把话说得非常准： **下行完全相同** （75 = 5 轮 × 15 客户端——对被采样集合的广播" -"从不跳过）；FedPD 省下的 *全是上行* 。五轮 FedPD 花费的 90 条消息与三轮 FedAvg 相同" -"——同样的通信预算换来了多两轮的本地迭代。" +"分解式把话说得非常准： **下行完全相同** （75 = 5 轮 × 15 客户端——对被采样集合的广播从不跳过）；FedPD 省下的 " +"*全是上行* 。五轮 FedPD 花费的 90 条消息与三轮 FedAvg 相同——同样的通信预算换来了多两轮的本地迭代。" #: ../../source/tutorial-first-experiment.rst:132 msgid "Three practical details when you run this yourself:" @@ -223,9 +217,9 @@ msgid "" "frequency ``int(1/p)`` always wins — passing a ``comm_freq`` yourself is " "silently overwritten." msgstr "" -"``p`` 必须满足 **0 < p < 1** 。p = 1 时 FedPD 每轮都上传（退化为 FedAvg）；p > 1 时" -"上传频率变为零，运行直接崩溃。配置暴露的是 ``p`` ，派生的上传频率 ``int(1/p)`` " -"始终说了算——自己传 ``comm_freq`` 会被静默覆写。" +"``p`` 必须满足 **0 < p < 1** 。p = 1 时 FedPD 每轮都上传（退化为 FedAvg）；p > 1 " +"时上传频率变为零，运行直接崩溃。配置暴露的是 ``p`` ，派生的上传频率 ``int(1/p)`` 始终说了算——自己传 " +"``comm_freq`` 会被静默覆写。" #: ../../source/tutorial-first-experiment.rst:139 msgid "" @@ -238,11 +232,11 @@ msgid "" "appears once per (skipped client × skipped round), so it dominates the " "console. Both are how FedPD is designed to behave." msgstr "" -" **你会看到 RuntimeWarning——那是预期行为，不是错误。** 服务器对每个跳过的轮次记录 " -"``No message received from the clients...`` （\"这轮没人上传，全局模型没有动\"）；" -"每个跳过的客户端记录 ``No parameters received from server...`` （\"这轮没收到新的" -"全局模型，我继续用当前参数\"）。教程规模下，客户端侧的消息每个（跳过客户端 × 跳过轮次）" -"出现一次，会刷满控制台。两者都是 FedPD 的设计行为。" +" **你会看到 RuntimeWarning——那是预期行为，不是错误。** 服务器对每个跳过的轮次记录 ``No message " +"received from the clients...`` （\"这轮没人上传，全局模型没有动\"）；每个跳过的客户端记录 ``No " +"parameters received from server...`` " +"（\"这轮没收到新的全局模型，我继续用当前参数\"）。教程规模下，客户端侧的消息每个（跳过客户端 × 跳过轮次）出现一次，会刷满控制台。两者都是 " +"FedPD 的设计行为。" #: ../../source/tutorial-first-experiment.rst:147 msgid "" @@ -251,9 +245,8 @@ msgid "" "unlogged), for FedPD at 75 (its last round never uploads). The final " "counts come from ``server._num_communications``." msgstr "" -"JSON 的 ``step`` 快照落后多少因算法而异：五轮 FedAvg 终于 120（最后一轮的 30 条消息" -"未入日志），FedPD 终于 75（它的最后一轮从不上传）。最终计数请以 " -"``server._num_communications`` 为准。" +"JSON 的 ``step`` 快照落后多少因算法而异：五轮 FedAvg 终于 120（最后一轮的 30 条消息未入日志），FedPD 终于 " +"75（它的最后一轮从不上传）。最终计数请以 ``server._num_communications`` 为准。" #: ../../source/tutorial-first-experiment.rst:153 msgid "Next steps" @@ -263,12 +256,17 @@ msgstr "下一步" msgid "" ":doc:`tutorial-cli-matrix` — run the same kind of comparison from the " "command line, as a YAML-driven experiment matrix." -msgstr "" -":doc:`tutorial-cli-matrix`——用 YAML 实验矩阵从命令行跑同样的对照。" +msgstr ":doc:`tutorial-cli-matrix`——用 YAML 实验矩阵从命令行跑同样的对照。" #: ../../source/tutorial-first-experiment.rst:157 msgid "" +":doc:`tutorial-visualization` — merge multi-seed runs of this comparison " +"into mean curves with error bounds." +msgstr "" +":doc:`tutorial-visualization`——把本次对照的多 seed 运行合并成带误差带的均值曲线。" + +#: ../../source/tutorial-first-experiment.rst:159 +msgid "" ":doc:`parallel_execution` — speed these runs up with a parallel executor " "without changing the results." -msgstr "" -":doc:`parallel_execution`——用并行执行后端加速这些运行，且不改变结果。" +msgstr ":doc:`parallel_execution`——用并行执行后端加速这些运行，且不改变结果。" diff --git a/docs/source/locale/zh_CN/LC_MESSAGES/tutorial-parallel-reproducible.po b/docs/source/locale/zh_CN/LC_MESSAGES/tutorial-parallel-reproducible.po new file mode 100644 index 00000000..ea133b23 --- /dev/null +++ b/docs/source/locale/zh_CN/LC_MESSAGES/tutorial-parallel-reproducible.po @@ -0,0 +1,126 @@ +# SOME DESCRIPTIVE TITLE. +# Copyright (C) 2023, WEN Hao +# This file is distributed under the same license as the fl-sim package. +# FIRST AUTHOR , 2026. +# +#, fuzzy +msgid "" +msgstr "" +"Project-Id-Version: fl-sim\n" +"Report-Msgid-Bugs-To: \n" +"POT-Creation-Date: 2026-10-29 00:02+0800\n" +"PO-Revision-Date: YEAR-MO-DA HO:MI+ZONE\n" +"Last-Translator: FULL NAME \n" +"Language-Team: zh_CN \n" +"Language: zh_CN\n" +"MIME-Version: 1.0\n" +"Content-Type: text/plain; charset=utf-8\n" +"Content-Transfer-Encoding: 8bit\n" +"Plural-Forms: nplurals=1; plural=0;\n" +"Generated-By: Babel 2.16.0\n" + +#: ../../source/tutorial-parallel-reproducible.rst:4 +msgid "Tutorial: parallel and reproducible runs" +msgstr "教程：并行与可复现的运行" + +#: ../../source/tutorial-parallel-reproducible.rst:6 +msgid "" +"Larger experiments take longer to train. fl-sim can execute the clients' local steps concurrently on a thread pool — with a one-line config change — and this tutorial shows how to keep the run " +"*reproducible* while doing so: at the end, two different backends produce **bitwise identical final model weights**." +msgstr "实验规模越大，训练耗时越长。fl-sim 可以在线程池上并发执行客户端的本地步——只需一行配置——本教程展示这样做的同时如何保持运行 *可复现* ：最后，两个不同的后端产出 **逐位一致的最终模型权重** 。" + +#: ../../source/tutorial-parallel-reproducible.rst:12 +msgid "The full script lives at ``test-files/tutorial_parallel_check.py`` and is exercised in CI. Prerequisites: :doc:`tutorial-first-experiment`." +msgstr "完整脚本位于 ``test-files/tutorial_parallel_check.py``，并在 CI 中被执行。前置阅读： :doc:`tutorial-first-experiment` 。" + +#: ../../source/tutorial-parallel-reproducible.rst:15 +msgid "The mechanics behind this are documented in :doc:`parallel_execution` (backends, guarantees, and how custom algorithms stay compatible); this page is the hands-on version." +msgstr "背后的机制见 :doc:`parallel_execution` （后端、保证、自定义算法如何保持兼容）；本页是动手版。" + +#: ../../source/tutorial-parallel-reproducible.rst:20 +msgid "Enabling parallel execution" +msgstr "开启并行执行" + +#: ../../source/tutorial-parallel-reproducible.rst:22 +msgid "" +"Add ``executor=\"threads\"`` to the server config. Nothing else changes — algorithms, datasets, logging and the participation schedule all behave identically; only the per-client local steps " +"(evaluate → local update → build upload) run on a thread pool:" +msgstr "在服务器配置里加 ``executor=\"threads\"`` 。其余一切不变——算法、数据集、日志、参与调度全都原样；只有每个客户端的本地步（评估 → 本地更新 → 构建上传）跑在线程池上：" + +#: ../../source/tutorial-parallel-reproducible.rst:32 +msgid "The helpers capture what we will compare: the final model weights (``weights_of``) and an exact equality check over them (``bitwise_equal`` — ``torch.equal``, not ``allclose``)." +msgstr "两个辅助函数负责我们要比较的东西：最终模型权重（ ``weights_of`` ）及其精确相等检查（ ``bitwise_equal`` ——用 ``torch.equal`` ，不是 ``allclose`` ）。" + +#: ../../source/tutorial-parallel-reproducible.rst:37 +msgid "The two backends produce identical results" +msgstr "两种后端产出完全一致的结果" + +#: ../../source/tutorial-parallel-reproducible.rst:39 +msgid "" +"Run the same experiment on both backends with the client config's ``rng_mode=\"per_client\"`` — each client's dataloaders get their own generators seeded from ``(seed, client_id, round)`` — and " +"compare:" +msgstr "把客户端配置设为 ``rng_mode=\"per_client\"`` （每个客户端的 dataloader 拿到由 ``(seed, client_id, round)`` 派生种子的专属生成器），在两种后端上跑同一实验并比较：" + +#: ../../source/tutorial-parallel-reproducible.rst:48 +msgid "Output:" +msgstr "输出：" + +#: ../../source/tutorial-parallel-reproducible.rst:54 +msgid "The results are bitwise identical — not merely close, but the same tensors. This holds regardless of how many workers ran or how the operating system scheduled them." +msgstr "结果逐位一致——不只是\"接近\"，也不只是\"精度相同\"：就是同一批张量。无论运行了多少 worker、操作系统如何调度，这一结论都成立。" + +#: ../../source/tutorial-parallel-reproducible.rst:59 +msgid "Behavior of the global RNG mode" +msgstr "全局 RNG 模式的行为" + +#: ../../source/tutorial-parallel-reproducible.rst:61 +msgid "``rng_mode`` defaults to ``\"global\"``: clients share the process-global torch RNG. The script runs the same comparison in that mode too:" +msgstr "``rng_mode`` 默认为 ``\"global\"`` ：客户端共享进程级全局 torch RNG。脚本在那种模式下也跑了同样的比较：" + +#: ../../source/tutorial-parallel-reproducible.rst:69 +msgid "" +"Under the global mode the *participation schedule* and the *aggregation arithmetic* are still backend-independent (sampling decisions and reply consumption happen on the main thread in fixed order)," +" so the communication counts always match. The clients' local trajectories, however, consume the shared RNG concurrently — so their interleaving depends on thread scheduling, and the final weights " +"may differ between backends and between repeats:" +msgstr "全局模式下， *参与调度* 与 *聚合算术* 仍与后端无关（采样决策与回复消费都在主线程按固定顺序进行），因此通信计数永远一致。但客户端的本地轨迹会并发消耗共享 RNG——交叠情况取决于线程调度，最终权重可能因后端不同、重复运行不同而不同：" + +#: ../../source/tutorial-parallel-reproducible.rst:81 +msgid "(On a small run like this one, the tasks often serialize and even the weights may match; this is incidental rather than guaranteed.)" +msgstr "（在这样的小规模运行中，任务常常自动串行，连权重都可能恰好一致；这是偶然现象，并非保证。）" + +#: ../../source/tutorial-parallel-reproducible.rst:84 +msgid "Practical guidance:" +msgstr "实践建议：" + +#: ../../source/tutorial-parallel-reproducible.rst:86 +msgid "**exploring**: global mode is acceptable; run-to-run variation is part of the measurement (average over seeds, as in :doc:`tutorial-visualization`)." +msgstr " **探索阶段** ：全局模式即可；运行间波动本来就是测量的一部分（对多个 seed 取平均，见 :doc:`tutorial-visualization` ）。" + +#: ../../source/tutorial-parallel-reproducible.rst:89 +msgid "" +"**paper-grade seed alignment**: either stay on ``\"sequential\"``, or set ``rng_mode=\"per_client\"`` — then every backend, every parallelism degree and every repeat produce the same trajectory from" +" the same seed." +msgstr " **论文级种子对齐** ：要么保持 ``\"sequential\"``，要么设 ``rng_mode=\"per_client\"``——之后同一种子在每个后端、每种并行度、每次重复下都走出同一条轨迹。" + +#: ../../source/tutorial-parallel-reproducible.rst:95 +msgid "Next steps" +msgstr "下一步" + +#: ../../source/tutorial-parallel-reproducible.rst:97 +msgid "" +":doc:`parallel_execution` — the full contract: guarantees per backend, the ``FL_SIM_EXECUTOR``/``FL_SIM_NUM_WORKERS`` environment overrides, measured speedups, and what custom algorithms must (not) " +"touch." +msgstr ":doc:`parallel_execution`——完整契约：每个后端的保证、 ``FL_SIM_EXECUTOR`` / ``FL_SIM_NUM_WORKERS`` 环境变量覆盖、实测加速比，以及自定义算法哪些能碰哪些不能碰。" + +#: ../../source/tutorial-parallel-reproducible.rst:100 +msgid ":doc:`tutorial-visualization` — plot the runs you just made reproducible." +msgstr ":doc:`tutorial-visualization`——把刚变得可复现的这些运行画出来。" + +#~ msgid "One line of parallelism" +#~ msgstr "一行开启并行" + +#~ msgid "The two backends agree exactly" +#~ msgstr "两种后端完全一致" + +#~ msgid "The rule of thumb:" +#~ msgstr "经验法则：" diff --git a/docs/source/locale/zh_CN/LC_MESSAGES/tutorial-visualization.po b/docs/source/locale/zh_CN/LC_MESSAGES/tutorial-visualization.po new file mode 100644 index 00000000..8f6ed96c --- /dev/null +++ b/docs/source/locale/zh_CN/LC_MESSAGES/tutorial-visualization.po @@ -0,0 +1,108 @@ +# SOME DESCRIPTIVE TITLE. +# Copyright (C) 2023, WEN Hao +# This file is distributed under the same license as the fl-sim package. +# FIRST AUTHOR , 2026. +# +#, fuzzy +msgid "" +msgstr "" +"Project-Id-Version: fl-sim\n" +"Report-Msgid-Bugs-To: \n" +"POT-Creation-Date: 2026-10-29 00:02+0800\n" +"PO-Revision-Date: YEAR-MO-DA HO:MI+ZONE\n" +"Last-Translator: FULL NAME \n" +"Language-Team: zh_CN \n" +"Language: zh_CN\n" +"MIME-Version: 1.0\n" +"Content-Type: text/plain; charset=utf-8\n" +"Content-Transfer-Encoding: 8bit\n" +"Plural-Forms: nplurals=1; plural=0;\n" +"Generated-By: Babel 2.16.0\n" + +#: ../../source/tutorial-visualization.rst:4 +msgid "Tutorial: visualize and export results" +msgstr "教程：可视化与导出结果" + +#: ../../source/tutorial-visualization.rst:6 +msgid "" +"Every run writes ``.json`` logs (see :doc:`quickstart`); fl-sim provides a visualization layer that turns them into publication-ready curves, both interactively in Jupyter and programmatically from " +"plain Python. This tutorial covers both: merging a multi-seed experiment into one mean curve with error bounds, exporting it to PDF, and opening the interactive dashboard." +msgstr "每次运行都会写 ``.json`` 日志（见 :doc:`quickstart` ）；fl-sim 提供一个可视化层，把它们变成可直接发表的曲线——既可在 Jupyter 中交互操作，也可在纯 Python 中以编程方式处理。本教程两者都讲：把多 seed 实验合并成一条带误差带的均值曲线，导出 PDF，再打开交互式仪表盘。" + +#: ../../source/tutorial-visualization.rst:13 +msgid "" +"The full script lives at ``test-files/tutorial_visualization.py`` and is exercised in CI (everything except the dashboard rendering, which needs Jupyter). Prerequisites: :doc:`tutorial-first-" +"experiment`." +msgstr "完整脚本位于 ``test-files/tutorial_visualization.py``，并在 CI 中被执行（仪表盘渲染除外，它需要 Jupyter）。前置阅读： :doc:`tutorial-first-experiment` 。" + +#: ../../source/tutorial-visualization.rst:17 +msgid "The reference for all knobs (fonts, smoothing, widgets) is :doc:`viz`; this page is the workflow." +msgstr "所有可调项（字体、平滑、控件）的参考见 :doc:`viz` ；本页讲的是工作流程。" + +#: ../../source/tutorial-visualization.rst:21 +msgid "Preparing the logs" +msgstr "准备日志" + +#: ../../source/tutorial-visualization.rst:23 +msgid "" +"The script trains the same FedAvg experiment under two seeds — the minimal \"multi-seed\" setup. Note the per-run ``tag``: log file names carry it, so the two runs cannot overwrite each other (the " +"same reasoning as in :doc:`tutorial-cli-matrix`):" +msgstr "脚本用两个种子训练同一个 FedAvg 实验——最小的\"多 seed\"设置。注意每次运行的 ``tag`` ：日志文件名携带它，两次运行不会互相覆盖（道理同 :doc:`tutorial-cli-matrix` ）：" + +#: ../../source/tutorial-visualization.rst:34 +msgid "Merging seeds into one curve" +msgstr "把多个 seed 合并成一条曲线" + +#: ../../source/tutorial-visualization.rst:36 +msgid "" +"``find_log_files`` collects the JSON logs; ``aggregate_results_from_json_log_with_custom_axis`` extracts a curve (``step`` on x, ``acc`` on y here — any recorded metric works); " +"``plot_mean_curve_with_error_bounds`` merges the curves into a mean with an uncertainty band (``\"sem\"``, ``\"std\"``, ``\"quartile\"`` or ``\"iqr\"``). Saving the figure is plain matplotlib:" +msgstr "" +"``find_log_files`` 收集 JSON 日志； ``aggregate_results_from_json_log_with_custom_axis`` 抽取曲线（这里 x 轴 ``step``、y 轴 ``acc``——任何记录过的指标都行）； ``plot_mean_curve_with_error_bounds`` 把曲线合并成带不确定度带的均值（ ``\"sem\"`` " +"、 ``\"std\"`` 、 ``\"quartile\"`` 或 ``\"iqr\"`` ）。存图就是普通的 matplotlib：" + +#: ../../source/tutorial-visualization.rst:47 +msgid "This writes ``fedavg-multi-seed.pdf`` next to the working directory. Two remarks:" +msgstr "这会在工作目录旁写出 ``fedavg-multi-seed.pdf`` 。两点说明：" + +#: ../../source/tutorial-visualization.rst:50 +msgid "" +"The x axis is the **cumulative communication count** — the metric :doc:`tutorial-first-experiment` dissected. Plotting against communications (rather than rounds) is what makes different " +"communication-efficiency algorithms comparable in one figure." +msgstr "x 轴是 **累计通信次数** ——即 :doc:`tutorial-first-experiment` 剖析过的那个量。以通信（而不是轮数）为横轴，才能把通信效率不同的算法放进同一张图里比较。" + +#: ../../source/tutorial-visualization.rst:54 +msgid "With only two seeds the band is a range, not a statistic — use more seeds (just add more ``seed=`` runs; the merging code does not change)." +msgstr "只有两个 seed 时，带子只是一个范围，还不是统计量——多加几个 ``seed=`` 运行即可（合并代码一行都不用改）。" + +#: ../../source/tutorial-visualization.rst:59 +msgid "Interactive exploration" +msgstr "交互式探索" + +#: ../../source/tutorial-visualization.rst:61 +msgid "" +"In a Jupyter notebook, the ``Panel`` dashboard performs all of the above interactively: it lists the log files it finds, plots selectable metrics, and provides sliders for smoothing, fonts and " +"figure size, plus export to PDF/SVG/PNG:" +msgstr "在 Jupyter notebook 里， ``Panel`` 仪表盘以上全部可交互完成：列出找到的日志文件、绘制可选指标，提供平滑、字体与图幅的滑块，并可导出 PDF/SVG/PNG：" + +#: ../../source/tutorial-visualization.rst:71 +msgid "Run this cell in a notebook (JupyterLab, Colab, SageMaker, …) and the dashboard renders inline; see :doc:`viz` for a demo animation and the full feature list." +msgstr "在 notebook（JupyterLab、Colab、SageMaker 等）里运行这个单元格，仪表盘就会内嵌渲染；演示动画与完整功能列表见 :doc:`viz` 。" + +#: ../../source/tutorial-visualization.rst:76 +msgid "Next steps" +msgstr "下一步" + +#: ../../source/tutorial-visualization.rst:78 +msgid ":doc:`tutorial-parallel-reproducible` — make these runs faster (parallel executor) without giving up reproducibility." +msgstr ":doc:`tutorial-parallel-reproducible`——让这些运行更快（并行执行器），且不放弃可复现性。" + +#: ../../source/tutorial-visualization.rst:80 +msgid ":doc:`viz` — the reference for every visualization option." +msgstr ":doc:`viz`——所有可视化选项的参考。" + +#~ msgid "Get some logs" +#~ msgstr "准备日志" + +#~ msgid "Explore interactively" +#~ msgstr "交互式探索" diff --git a/docs/source/tutorial-first-experiment.rst b/docs/source/tutorial-first-experiment.rst index 98dfc219..736aa0a2 100644 --- a/docs/source/tutorial-first-experiment.rst +++ b/docs/source/tutorial-first-experiment.rst @@ -154,5 +154,7 @@ Next steps - :doc:`tutorial-cli-matrix` — run the same kind of comparison from the command line, as a YAML-driven experiment matrix. +- :doc:`tutorial-visualization` — merge multi-seed runs of this + comparison into mean curves with error bounds. - :doc:`parallel_execution` — speed these runs up with a parallel executor without changing the results. diff --git a/docs/source/tutorial-parallel-reproducible.rst b/docs/source/tutorial-parallel-reproducible.rst new file mode 100644 index 00000000..0ffcb4b6 --- /dev/null +++ b/docs/source/tutorial-parallel-reproducible.rst @@ -0,0 +1,101 @@ +.. _tutorial-parallel-reproducible: + +Tutorial: parallel and reproducible runs +======================================== + +Larger experiments take longer to train. fl-sim can execute the clients' +local steps concurrently on a thread pool — with a one-line config change +— and this tutorial shows how to keep the run *reproducible* while doing +so: at the end, two different backends produce **bitwise identical final +model weights**. + +The full script lives at ``test-files/tutorial_parallel_check.py`` and +is exercised in CI. Prerequisites: :doc:`tutorial-first-experiment`. + +The mechanics behind this are documented in +:doc:`parallel_execution` (backends, guarantees, and how custom +algorithms stay compatible); this page is the hands-on version. + +Enabling parallel execution +--------------------------- + +Add ``executor="threads"`` to the server config. Nothing else changes — +algorithms, datasets, logging and the participation schedule all behave +identically; only the per-client local steps (evaluate → local update → +build upload) run on a thread pool: + +.. literalinclude:: ../../test-files/tutorial_parallel_check.py + :language: python + :start-after: # --- begin: setup --- + :end-before: # --- end: setup --- + +The helpers capture what we will compare: the final model weights +(``weights_of``) and an exact equality check over them +(``bitwise_equal`` — ``torch.equal``, not ``allclose``). + +The two backends produce identical results +------------------------------------------ + +Run the same experiment on both backends with the client config's +``rng_mode="per_client"`` — each client's dataloaders get their own +generators seeded from ``(seed, client_id, round)`` — and compare: + +.. literalinclude:: ../../test-files/tutorial_parallel_check.py + :language: python + :start-after: # --- begin: per-client-check --- + :end-before: # --- end: per-client-check --- + +Output: + +.. code-block:: text + + per-client RNG: communications equal: True, final weights bitwise equal: True + +The results are bitwise identical — not merely close, but the same +tensors. This holds regardless of how many workers ran or how the +operating system scheduled them. + +Behavior of the global RNG mode +------------------------------- + +``rng_mode`` defaults to ``"global"``: clients share the process-global +torch RNG. The script runs the same comparison in that mode too: + +.. literalinclude:: ../../test-files/tutorial_parallel_check.py + :language: python + :start-after: # --- begin: global-check --- + :end-before: # --- end: global-check --- + +Under the global mode the *participation schedule* and the *aggregation +arithmetic* are still backend-independent (sampling decisions and reply +consumption happen on the main thread in fixed order), so the +communication counts always match. The clients' local trajectories, +however, consume the shared RNG concurrently — so their interleaving +depends on thread scheduling, and the final weights may differ between +backends and between repeats: + +.. code-block:: text + + global RNG: communications equal: True, final weights bitwise equal: True (may be False by design) + +(On a small run like this one, the tasks often serialize and even the +weights may match; this is incidental rather than guaranteed.) + +Practical guidance: + +- **exploring**: global mode is acceptable; run-to-run variation is part + of the measurement (average over seeds, as in + :doc:`tutorial-visualization`). +- **paper-grade seed alignment**: either stay on ``"sequential"``, or + set ``rng_mode="per_client"`` — then every backend, every parallelism + degree and every repeat produce the same trajectory from the same + seed. + +Next steps +---------- + +- :doc:`parallel_execution` — the full contract: guarantees per backend, + the ``FL_SIM_EXECUTOR``/``FL_SIM_NUM_WORKERS`` environment overrides, + measured speedups, and what custom algorithms must (not) touch. +- :doc:`tutorial-visualization` — plot the runs you just made + reproducible. diff --git a/docs/source/tutorial-visualization.rst b/docs/source/tutorial-visualization.rst new file mode 100644 index 00000000..171604d1 --- /dev/null +++ b/docs/source/tutorial-visualization.rst @@ -0,0 +1,80 @@ +.. _tutorial-visualization: + +Tutorial: visualize and export results +====================================== + +Every run writes ``.json`` logs (see :doc:`quickstart`); fl-sim provides a +visualization layer that turns them into publication-ready curves, both +interactively in Jupyter and programmatically from plain Python. This +tutorial covers both: merging a multi-seed experiment into one mean curve +with error bounds, exporting it to PDF, and opening the interactive +dashboard. + +The full script lives at ``test-files/tutorial_visualization.py`` and is +exercised in CI (everything except the dashboard rendering, which needs +Jupyter). Prerequisites: :doc:`tutorial-first-experiment`. + +The reference for all knobs (fonts, smoothing, widgets) is +:doc:`viz`; this page is the workflow. + +Preparing the logs +------------------ + +The script trains the same FedAvg experiment under two seeds — the +minimal "multi-seed" setup. Note the per-run ``tag``: log file names +carry it, so the two runs cannot overwrite each other (the same +reasoning as in :doc:`tutorial-cli-matrix`): + +.. literalinclude:: ../../test-files/tutorial_visualization.py + :language: python + :start-after: # --- begin: setup --- + :end-before: # --- end: setup --- + +Merging seeds into one curve +---------------------------- + +``find_log_files`` collects the JSON logs; ``aggregate_results_from_json_log_with_custom_axis`` +extracts a curve (``step`` on x, ``acc`` on y here — any recorded metric +works); ``plot_mean_curve_with_error_bounds`` merges the curves into a +mean with an uncertainty band (``"sem"``, ``"std"``, ``"quartile"`` or +``"iqr"``). Saving the figure is plain matplotlib: + +.. literalinclude:: ../../test-files/tutorial_visualization.py + :language: python + :start-after: # --- begin: plot-mean --- + :end-before: # --- end: plot-mean --- + +This writes ``fedavg-multi-seed.pdf`` next to the working directory. +Two remarks: + +- The x axis is the **cumulative communication count** — the metric + :doc:`tutorial-first-experiment` dissected. Plotting against + communications (rather than rounds) is what makes different + communication-efficiency algorithms comparable in one figure. +- With only two seeds the band is a range, not a statistic — use more + seeds (just add more ``seed=`` runs; the merging code does not + change). + +Interactive exploration +----------------------- + +In a Jupyter notebook, the ``Panel`` dashboard performs all of the above +interactively: it lists the log files it finds, plots selectable metrics, +and provides sliders for smoothing, fonts and figure size, plus export to +PDF/SVG/PNG: + +.. literalinclude:: ../../test-files/tutorial_visualization.py + :language: python + :start-after: # --- begin: panel --- + :end-before: # --- end: panel --- + +Run this cell in a notebook (JupyterLab, Colab, SageMaker, …) and the +dashboard renders inline; see :doc:`viz` for a demo animation and the +full feature list. + +Next steps +---------- + +- :doc:`tutorial-parallel-reproducible` — make these runs faster + (parallel executor) without giving up reproducibility. +- :doc:`viz` — the reference for every visualization option. diff --git a/test-files/tutorial_first_experiment.py b/test-files/tutorial_first_experiment.py index 18ecc91a..f68e624c 100644 --- a/test-files/tutorial_first_experiment.py +++ b/test-files/tutorial_first_experiment.py @@ -5,14 +5,14 @@ ``tutorial-first-experiment`` page of the documentation (``docs/source/tutorial-first-experiment.rst``): the marked sections are included verbatim via Sphinx ``literalinclude`` directives, so the docs -and this script can never drift apart. The size is pinned to the tutorial +and this script cannot drift apart. The size is pinned to the tutorial budget (``num_iters <= 5``, ``num_clients <= 30``, one local epoch), enforced by ``test/test_tutorial_scripts.py``. -The dataset is built ONCE, before any config/server, on purpose: config +The dataset is built once, before any config or server: config constructors re-seed the global RNG when they run, so whatever was built -last would otherwise shift the results. Construct in the order -dataset -> config -> server and the numbers are stable. +last would otherwise shift the results. Constructing in the order +dataset -> config -> server keeps the results reproducible. """ # --- begin: setup --- @@ -41,20 +41,23 @@ # one synthetic dataset, split across 30 clients, shared by every run below dataset = FedSynthetic(1.0, 1.0, True, 30) -model = dataset.candidate_models["mlp_d1"] def run(algorithm, sample_ratio, num_iters, tag, **extra_kw): """Train one algorithm and return the server. - Algorithm-specific server keywords (SCAFFOLD's global step size, - FedPD's upload frequency ``p``) come from ``SERVER_KW``; anything - passed as ``extra_kw`` is merged on top. + Every run takes a FRESH model from ``dataset.candidate_models`` + (a property building a new module on each access): the server trains + the model it is given in place, so sharing one model across runs + would warm-start every run after the first and contaminate the + comparison. Algorithm-specific server keywords (SCAFFOLD's global + step size, FedPD's upload frequency ``p``) come from ``SERVER_KW``; + anything passed as ``extra_kw`` is merged on top. """ server_cls, server_config_cls, client_config_cls = ALGORITHMS[algorithm] server_kw = {**SERVER_KW[algorithm], **extra_kw} server = server_cls( - model, + dataset.candidate_models["mlp_d1"], dataset, server_config_cls( num_iters=num_iters, diff --git a/test-files/tutorial_parallel_check.py b/test-files/tutorial_parallel_check.py new file mode 100644 index 00000000..6f26c0a9 --- /dev/null +++ b/test-files/tutorial_parallel_check.py @@ -0,0 +1,74 @@ +""" +Parallelize a run and verify it stays reproducible. + +This script is the source of the code blocks shown in the +``tutorial-parallel-reproducible`` page of the documentation +(``docs/source/tutorial-parallel-reproducible.rst``): the marked sections +are included verbatim via Sphinx ``literalinclude`` directives, so the +docs and this script cannot drift apart. The runs are pinned to the +tutorial budget (``num_iters <= 5``, ``num_clients <= 30``, one local +epoch), enforced by ``test/test_tutorial_scripts.py``. +""" + +# --- begin: setup --- +import torch + +from fl_sim.algorithms.fedopt import FedAvgClientConfig, FedAvgServer, FedAvgServerConfig +from fl_sim.data_processing.fed_synthetic import FedSynthetic + +dataset = FedSynthetic(1.0, 1.0, True, 30) + + +def run(executor, tag, rng_mode): + """One FedAvg run on the given executor backend.""" + return FedAvgServer( + # a fresh model per run (see the first-experiment tutorial) + dataset.candidate_models["mlp_d1"], + dataset, + FedAvgServerConfig( + num_iters=3, + num_clients=30, + clients_sample_ratio=0.5, + executor=executor, # "sequential" (default) or "threads" + num_workers=2, + log_dir=f"t4_{tag}", + ), + FedAvgClientConfig(batch_size=32, num_epochs=1, rng_mode=rng_mode), + ) + + +def weights_of(server): + return list(server.model.state_dict().values()) + + +def bitwise_equal(a, b): + return all(torch.equal(x, y) for x, y in zip(weights_of(a), weights_of(b))) + + +# --- end: setup --- + +# --- begin: per-client-check --- +sequential = run("sequential", "seq", rng_mode="per_client") +threads = run("threads", "threads", rng_mode="per_client") + +print( + f"per-client RNG: communications equal: " + f"{sequential._num_communications == threads._num_communications}, " + f"final weights bitwise equal: {bitwise_equal(sequential, threads)}" +) +# --- end: per-client-check --- + +# --- begin: global-check --- +sequential_g = run("sequential", "seq_global", rng_mode="global") +threads_g = run("threads", "threads_global", rng_mode="global") + +# with the process-global RNG, the participation sets and the aggregation +# arithmetic stay identical, but the clients' local trajectories consume +# the shared RNG concurrently, so the final weights may differ — this is +# the documented (and expected) statistical-level behaviour +print( + f"global RNG: communications equal: " + f"{sequential_g._num_communications == threads_g._num_communications}, " + f"final weights bitwise equal: {bitwise_equal(sequential_g, threads_g)} (may be False by design)" +) +# --- end: global-check --- diff --git a/test-files/tutorial_visualization.py b/test-files/tutorial_visualization.py new file mode 100644 index 00000000..e222780e --- /dev/null +++ b/test-files/tutorial_visualization.py @@ -0,0 +1,73 @@ +""" +Visualize and export experiment results. + +This script is the source of the code blocks shown in the +``tutorial-visualization`` page of the documentation +(``docs/source/tutorial-visualization.rst``): the marked sections are +included verbatim via Sphinx ``literalinclude`` directives, so the docs +and this script can never drift apart. The training runs below are +pinned to the tutorial budget (``num_iters <= 5``, ``num_clients <= 30``, +one local epoch), enforced by ``test/test_tutorial_scripts.py``. + +The two runs at the top produce the experiment logs used below; the same +code works on any log directory, including those from the earlier +tutorials. + +The interactive part of the tutorial (the ``Panel`` dashboard) needs a +Jupyter notebook and is therefore only constructed, not rendered, here. +""" + +# --- begin: setup --- +from fl_sim.algorithms.fedopt import FedAvgClientConfig, FedAvgServer, FedAvgServerConfig +from fl_sim.data_processing.fed_synthetic import FedSynthetic + +dataset = FedSynthetic(1.0, 1.0, True, 30) + +# two seeds of the same experiment: the minimal multi-seed setup +for seed in (0, 1): + server = FedAvgServer( + # a fresh model per run: the server trains the model it is given + # in place, so sharing one module would warm-start the second run + dataset.candidate_models["mlp_d1"], + dataset, + FedAvgServerConfig( + num_iters=3, + num_clients=30, + clients_sample_ratio=0.5, + seed=seed, + log_dir="t3_seeds", + tag=f"seed_{seed}", # distinct tag per seed: no log overwrites + ), + FedAvgClientConfig(batch_size=32, num_epochs=1), + ) + server.train() +# --- end: setup --- + +from pathlib import Path + +# --- begin: plot-mean --- +import matplotlib.pyplot as plt + +from fl_sim.utils.viz import aggregate_results_from_json_log_with_custom_axis, find_log_files + +# the runs above logged into the t3_seeds directory under fl-sim's log root +log_dir = Path(server._logger_manager.log_dir) +files = find_log_files(directory=log_dir, filters="seed_") +curves = [aggregate_results_from_json_log_with_custom_axis(f, part="val", x_metric="step", y_metric="acc") for f in files] + +fig, ax = plt.subplots(figsize=(6, 4)) +from fl_sim.utils.viz import plot_mean_curve_with_error_bounds + +fig, ax = plot_mean_curve_with_error_bounds(curves, error_type="sem", label="FedAvg", fig_ax=(fig, ax)) +ax.set_xlabel("cumulative communications") +ax.set_ylabel("val acc") +fig.savefig("fedavg-multi-seed.pdf", bbox_inches="tight") +# --- end: plot-mean --- + +# --- begin: panel --- +from fl_sim.utils.viz import Panel + +# in a Jupyter notebook this renders an interactive dashboard: +# pick log files, tune curves, export to PDF/SVG/PNG +panel = Panel(logdir="t3_seeds") +# --- end: panel --- diff --git a/test/test_tutorial_scripts.py b/test/test_tutorial_scripts.py index 59e2ac84..7e129777 100644 --- a/test/test_tutorial_scripts.py +++ b/test/test_tutorial_scripts.py @@ -33,6 +33,8 @@ TUTORIAL_SCRIPTS = { "quickstart": REPO_ROOT / "test-files" / "quickstart.py", "tutorial_first_experiment": REPO_ROOT / "test-files" / "tutorial_first_experiment.py", + "tutorial_visualization": REPO_ROOT / "test-files" / "tutorial_visualization.py", + "tutorial_parallel_check": REPO_ROOT / "test-files" / "tutorial_parallel_check.py", } CLI_MATRIX_DEMO = REPO_ROOT / "example-configs" / "fedsynthetic-demo.yml" @@ -44,6 +46,14 @@ def isolated_log_dir(tmp_path, monkeypatch): yield tmp_path +@pytest.fixture() +def in_tmp_cwd(tmp_path, monkeypatch): + """Run scripts from a temporary working directory, so any file they + write relative to the CWD (e.g. an exported PDF) stays contained.""" + monkeypatch.chdir(tmp_path) + yield tmp_path + + def _run_script(script: Path): """Execute a tutorial script in-process, restoring ``sys.path`` afterwards. @@ -148,6 +158,49 @@ def test_cli_matrix_demo_expands_and_runs(isolated_log_dir): _check_tutorial_budget(run_dir) +def test_visualization_script_merges_seeds_and_exports(isolated_log_dir, in_tmp_cwd): + namespace = _run_script(TUTORIAL_SCRIPTS["tutorial_visualization"]) + + # two tagged runs, both within the tutorial budget + run_dir = isolated_log_dir / "t3_seeds" + jsons = [p for p in run_dir.glob("*.json") if p.name != "config.json"] + assert len(jsons) == 2, "the two seeds must not overwrite each other's logs" + _check_tutorial_budget(run_dir) + + # the merge produced one non-empty curve per seed, and the PDF export + # landed in the (temporary) working directory + curves = namespace["curves"] + assert len(curves) == 2 + for curve in curves: + assert len(curve["x"]) > 0 and len(curve["y"]) > 0 + assert (in_tmp_cwd / "fedavg-multi-seed.pdf").exists(), "the mean curve should be exported to PDF" + + # the dashboard object is constructible outside Jupyter (it prints a + # note and stays inert there; rendering is covered by manual use) + assert namespace["panel"] is not None + + +def test_parallel_check_script_verifies_bitwise_reproducibility(isolated_log_dir): + namespace = _run_script(TUTORIAL_SCRIPTS["tutorial_parallel_check"]) + + # the L1 guarantee, demonstrated end to end: same seed, per-client RNG, + # two different executor backends -> bitwise identical final weights + sequential, threads = namespace["sequential"], namespace["threads"] + assert sequential._num_communications == threads._num_communications + assert namespace["bitwise_equal"]( + sequential, threads + ), "sequential and threads with rng_mode='per_client' must agree bitwise" + + # global mode: participation stays backend-independent; local + # trajectories may interleave by design, so only the counts are asserted + sequential_g, threads_g = namespace["sequential_g"], namespace["threads_g"] + assert sequential_g._num_communications == threads_g._num_communications + + for sub in isolated_log_dir.iterdir(): + if sub.name.startswith("t4_") and sub.is_dir(): + _check_tutorial_budget(sub) + + def test_tutorial_scripts_keep_literalinclude_markers(): """The markers delimit the fragments the docs include; dropping one silently empties a documentation code block."""