[Raiden Weight Sync 7/7] Clean up rollout path and remove redundant weight sync logic - #5172
[Raiden Weight Sync 7/7] Clean up rollout path and remove redundant weight sync logic#5172YixuanWang-99 wants to merge 1 commit into
Conversation
There was a problem hiding this comment.
Code Review
This pull request simplifies the weight synchronization and rollout logic for vLLM integration by removing unused standalone converters, Qwen scanned weight unrolling, and local compatibility patches. Feedback on the changes highlights two critical issues: first, replacing the standard logging module with absl.logging will cause an AttributeError when calling logging.getLogger() in _log_and_flush_traceback; second, the fallback logic to determine if a model is Gemma relies on config.model_config, which is not present on VllmConfig and should instead use config.engine_kwargs.
| if not self._model_name: | ||
| model_config = getattr(config, "model_config", None) | ||
| self._model_name = getattr(model_config, "model", "") or "" | ||
| architectures = getattr(model_config, "architectures", []) or [] | ||
| hf_config = getattr(model_config, "hf_config", None) | ||
| model_type = getattr(hf_config, "model_type", "") or "" | ||
| arch_str = " ".join(str(a) for a in architectures) | ||
| self._is_gemma = ( | ||
| "gemma" in str(self._model_name).lower() | ||
| or "gemma" in str(model_type).lower() | ||
| or "gemma" in str(arch_str).lower() | ||
| ) | ||
| else: | ||
| self._is_gemma = "gemma" in str(self._model_name).lower() |
There was a problem hiding this comment.
If model_name is not provided or is empty, the fallback logic tries to retrieve model_config from config. However, VllmConfig does not have a model_config attribute, meaning self._model_name will remain empty and self._is_gemma will incorrectly evaluate to False for Gemma models. Falling back to config.engine_kwargs.get("model") provides a robust way to retrieve the model name/path.
| if not self._model_name: | |
| model_config = getattr(config, "model_config", None) | |
| self._model_name = getattr(model_config, "model", "") or "" | |
| architectures = getattr(model_config, "architectures", []) or [] | |
| hf_config = getattr(model_config, "hf_config", None) | |
| model_type = getattr(hf_config, "model_type", "") or "" | |
| arch_str = " ".join(str(a) for a in architectures) | |
| self._is_gemma = ( | |
| "gemma" in str(self._model_name).lower() | |
| or "gemma" in str(model_type).lower() | |
| or "gemma" in str(arch_str).lower() | |
| ) | |
| else: | |
| self._is_gemma = "gemma" in str(self._model_name).lower() | |
| if not self._model_name: | |
| engine_kwargs = getattr(config, "engine_kwargs", {}) or {} | |
| self._model_name = engine_kwargs.get("model", "") or "" | |
| model_config = getattr(config, "model_config", None) | |
| architectures = getattr(model_config, "architectures", []) or [] | |
| hf_config = getattr(model_config, "hf_config", None) | |
| model_type = getattr(hf_config, "model_type", "") or "" | |
| arch_str = " ".join(str(a) for a in architectures) | |
| self._is_gemma = ( | |
| "gemma" in str(self._model_name).lower() | |
| or "gemma" in str(model_type).lower() | |
| or "gemma" in str(arch_str).lower() | |
| ) | |
| else: | |
| self._is_gemma = "gemma" in str(self._model_name).lower() |
9844f2e to
6786aae
Compare
94952a0 to
cf5947c
Compare
| self._layer_pattern_length = layer_pattern_length | ||
| self._model_name = model_name or "" | ||
| if not self._model_name: | ||
| model_config = getattr(config, "model_config", None) |
There was a problem hiding this comment.
Can this logic be simplified?
6786aae to
a6ade7c
Compare
cf5947c to
f6ffea7
Compare
a6ade7c to
ff6a0eb
Compare
b191f20 to
89a0bd5
Compare
75ae687 to
7b3bf86
Compare
89a0bd5 to
353739a
Compare
7b3bf86 to
870c66b
Compare
353739a to
c2be3f7
Compare
c2be3f7 to
07b780e
Compare
07b780e to
0c4bf4f
Compare
- Remap vLLM hybrid KV cache groups using layer_name_to_kvcache_index before passing to MaxText decoder layers, resolving shard_map rank mismatches on attention vs GDN layers. - Strip auxiliary runner kwargs before forwarding to self.model. - Write updated KV caches back to original physical cache slots. - Add rollout_tensor_parallelism field to VLLM config. - Remove redundant rollout weight sync logic in favor of Raiden FFI path.
0c4bf4f to
46722c8
Compare
Overview
Part of the stacked Raiden weight-sync enablement PR chain replacing #5089.
Stack:
Details
maxtext_vllm_rollout.pyandtrain_rl.py, now that the logic is cleanly owned by M3/M4.Checklist
Before submitting this PR, please make sure (put X in square brackets):
gemini-reviewlabel.