Skip to content

Add challenge 120: Fused AdamW (Medium) - #316

Open
adityasharmaaaaa wants to merge 2 commits into
AlphaGPU:mainfrom
adityasharmaaaaa:add-fused-adamw-challenge
Open

adityasharmaaaaa wants to merge 2 commits into
AlphaGPU:mainfrom
adityasharmaaaaa:add-fused-adamw-challenge

Conversation

@adityasharmaaaaa

Copy link
Copy Markdown
Contributor

Adds a new challenge implementing a fused AdamW optimizer step (decoupled weight decay + bias-corrected first/second moment updates), covering params/m/v as in-place stateful tensors, a gap in the existing challenge set, which has no optimizer-related content despite covering PPO/DPO/GRPO/GAE losses. Includes starters for all 6 supported frameworks (CUDA, Mojo, CuTe, PyTorch, Triton, JAX), an example/functional/performance test suite (11 functional cases covering zero-grad, warm/cold optimizer state, extreme step counts, non-default betas, etc.), and a challenge.html writeup with two hand-verified worked examples. Verified pre-commit run --all-files passes (mojo-format excluded locally, no mojo binary installed) and ran reference_impl against every generated test case to confirm finite, valid output.

Copy link
Copy Markdown
Contributor

Review notes — not merging (challenge ID collision)

114 is already taken on main as challenges/hard/114_multi_head_latent_attention.

Challenge quality looks solid (decoupled AdamW reference, good edge cases, all 6 starters, CI green), but please renumber to a free ID (120+ after today's merges of 115/117/119) and update challenges/medium/114_fused_adamw → new path before we can squash-merge.

Thanks for the contribution — happy to merge once the ID is unique.

@adityasharmaaaaa
adityasharmaaaaa force-pushed the add-fused-adamw-challenge branch from ede346c to 0bbfe96 Compare September 30, 2026 14:57
@adityasharmaaaaa adityasharmaaaaa changed the title Add challenge 114: Fused AdamW (Medium) Add challenge 120: Fused AdamW (Medium) Sep 30, 2026
@adityasharmaaaaa

Copy link
Copy Markdown
Contributor Author

@kunal-mansukhani Renumbered to 120 (folder is now challenges/medium/120_fused_adamw) and rebased on latest main. Ready for another look.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants