Skip to content

[Raiden Weight Sync 7/7] Clean up rollout path and remove redundant weight sync logic - #5172

Open
YixuanWang-99 wants to merge 1 commit into
mainfrom
yixuann-m6-rollout-cleanup
Open

[Raiden Weight Sync 7/7] Clean up rollout path and remove redundant weight sync logic#5172
YixuanWang-99 wants to merge 1 commit into
mainfrom
yixuann-m6-rollout-cleanup

Conversation

@YixuanWang-99

@YixuanWang-99 YixuanWang-99 commented Sep 9, 2026

Copy link
Copy Markdown
Collaborator

Overview

Part of the stacked Raiden weight-sync enablement PR chain replacing #5089.

Stack:

Details

  • Net −354 lines.
  • Removes deprecated and redundant rollout and weight sync duplication in maxtext_vllm_rollout.py and train_rl.py, now that the logic is cleanly owned by M3/M4.

Checklist

Before submitting this PR, please make sure (put X in square brackets):

  • I have performed a self-review of my code. For an optional AI review, add the gemini-review label.
  • I have necessary comments in my code, particularly in hard-to-understand areas.
  • I have run end-to-end tests tests and provided workload links above if applicable.
  • I have made or will make corresponding changes to the doc if needed, including adding new documentation pages to the relevant Table of Contents (toctree directive) as explained in our documentation.

@gemini-code-assist gemini-code-assist Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Code Review

This pull request simplifies the weight synchronization and rollout logic for vLLM integration by removing unused standalone converters, Qwen scanned weight unrolling, and local compatibility patches. Feedback on the changes highlights two critical issues: first, replacing the standard logging module with absl.logging will cause an AttributeError when calling logging.getLogger() in _log_and_flush_traceback; second, the fallback logic to determine if a model is Gemma relies on config.model_config, which is not present on VllmConfig and should instead use config.engine_kwargs.

Comment thread src/maxtext/integration/vllm/maxtext_vllm_rollout.py
Comment on lines +311 to +324
if not self._model_name:
model_config = getattr(config, "model_config", None)
self._model_name = getattr(model_config, "model", "") or ""
architectures = getattr(model_config, "architectures", []) or []
hf_config = getattr(model_config, "hf_config", None)
model_type = getattr(hf_config, "model_type", "") or ""
arch_str = " ".join(str(a) for a in architectures)
self._is_gemma = (
"gemma" in str(self._model_name).lower()
or "gemma" in str(model_type).lower()
or "gemma" in str(arch_str).lower()
)
else:
self._is_gemma = "gemma" in str(self._model_name).lower()

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

high

If model_name is not provided or is empty, the fallback logic tries to retrieve model_config from config. However, VllmConfig does not have a model_config attribute, meaning self._model_name will remain empty and self._is_gemma will incorrectly evaluate to False for Gemma models. Falling back to config.engine_kwargs.get("model") provides a robust way to retrieve the model name/path.

Suggested change
if not self._model_name:
model_config = getattr(config, "model_config", None)
self._model_name = getattr(model_config, "model", "") or ""
architectures = getattr(model_config, "architectures", []) or []
hf_config = getattr(model_config, "hf_config", None)
model_type = getattr(hf_config, "model_type", "") or ""
arch_str = " ".join(str(a) for a in architectures)
self._is_gemma = (
"gemma" in str(self._model_name).lower()
or "gemma" in str(model_type).lower()
or "gemma" in str(arch_str).lower()
)
else:
self._is_gemma = "gemma" in str(self._model_name).lower()
if not self._model_name:
engine_kwargs = getattr(config, "engine_kwargs", {}) or {}
self._model_name = engine_kwargs.get("model", "") or ""
model_config = getattr(config, "model_config", None)
architectures = getattr(model_config, "architectures", []) or []
hf_config = getattr(model_config, "hf_config", None)
model_type = getattr(hf_config, "model_type", "") or ""
arch_str = " ".join(str(a) for a in architectures)
self._is_gemma = (
"gemma" in str(self._model_name).lower()
or "gemma" in str(model_type).lower()
or "gemma" in str(arch_str).lower()
)
else:
self._is_gemma = "gemma" in str(self._model_name).lower()

@YixuanWang-99
YixuanWang-99 force-pushed the yixuann-m4-engine-raiden branch from 9844f2e to 6786aae Compare September 9, 2026 02:49
@YixuanWang-99
YixuanWang-99 force-pushed the yixuann-m6-rollout-cleanup branch 2 times, most recently from 94952a0 to cf5947c Compare September 9, 2026 16:37
Comment thread src/maxtext/integration/vllm/maxtext_vllm_rollout.py Outdated
self._layer_pattern_length = layer_pattern_length
self._model_name = model_name or ""
if not self._model_name:
model_config = getattr(config, "model_config", None)

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can this logic be simplified?

@YixuanWang-99
YixuanWang-99 force-pushed the yixuann-m4-engine-raiden branch from 6786aae to a6ade7c Compare September 9, 2026 17:56
@YixuanWang-99
YixuanWang-99 force-pushed the yixuann-m6-rollout-cleanup branch from cf5947c to f6ffea7 Compare September 9, 2026 17:56
@YixuanWang-99
YixuanWang-99 force-pushed the yixuann-m4-engine-raiden branch from a6ade7c to ff6a0eb Compare September 9, 2026 19:05
@YixuanWang-99
YixuanWang-99 force-pushed the yixuann-m6-rollout-cleanup branch 2 times, most recently from b191f20 to 89a0bd5 Compare September 9, 2026 19:13
@YixuanWang-99
YixuanWang-99 force-pushed the yixuann-m4-engine-raiden branch 2 times, most recently from 75ae687 to 7b3bf86 Compare September 9, 2026 20:52
@YixuanWang-99
YixuanWang-99 force-pushed the yixuann-m6-rollout-cleanup branch from 89a0bd5 to 353739a Compare September 9, 2026 20:52
@YixuanWang-99
YixuanWang-99 force-pushed the yixuann-m4-engine-raiden branch from 7b3bf86 to 870c66b Compare September 9, 2026 20:53
@YixuanWang-99
YixuanWang-99 force-pushed the yixuann-m6-rollout-cleanup branch from 353739a to c2be3f7 Compare September 9, 2026 20:53
@SurbhiJainUSC
SurbhiJainUSC changed the base branch from yixuann-m4-engine-raiden to main September 9, 2026 21:07
@SurbhiJainUSC
SurbhiJainUSC force-pushed the yixuann-m6-rollout-cleanup branch from c2be3f7 to 07b780e Compare September 9, 2026 21:28
@igorts-git
igorts-git force-pushed the yixuann-m6-rollout-cleanup branch from 07b780e to 0c4bf4f Compare September 9, 2026 22:19
- Remap vLLM hybrid KV cache groups using layer_name_to_kvcache_index before passing to MaxText decoder layers, resolving shard_map rank mismatches on attention vs GDN layers.
- Strip auxiliary runner kwargs before forwarding to self.model.
- Write updated KV caches back to original physical cache slots.
- Add rollout_tensor_parallelism field to VLLM config.
- Remove redundant rollout weight sync logic in favor of Raiden FFI path.
@igorts-git
igorts-git force-pushed the yixuann-m6-rollout-cleanup branch from 0c4bf4f to 46722c8 Compare September 9, 2026 22:51
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants