Skip to content

Polish Code City Apocalypse Dashboard#25

Open
BleakNarratives wants to merge 1 commit into
mainfrom
feat/code-city-apocalypse-polish-5595286712531347712
Open

Polish Code City Apocalypse Dashboard#25
BleakNarratives wants to merge 1 commit into
mainfrom
feat/code-city-apocalypse-polish-5595286712531347712

Conversation

@BleakNarratives
Copy link
Copy Markdown
Owner

The "Code City Apocalypse" dashboard has been significantly improved to correctly handle both live and archived telemetry. Key fixes include robust success detection, accurate leaderboard metrics that credit models only once per secured sector, and UI refinements such as themed "Beacon of Hope" styling in the Digital Bestiary. Code quality has been ensured through Ruff/Mypy compliance and the resolution of common Pandas warnings.


PR created automatically by Jules for task 5595286712531347712 started by @BleakNarratives

- Fix success detection for archived data in `dataset/failed_flag_submissions.csv`.
- Improve leaderboard logic to count unique secured sectors.
- Enhance Digital Bestiary with special styling for "The Beacon of Hope".
- Add defensive handling and avoid SettingWithCopyWarning in the emergency ticker.
- Fix UI regression in Bestiary card styling.
- Ensure all models are represented in the leaderboard.
- Clean up verification artifacts.

Co-authored-by: google-labs-jules[bot] <161369871+google-labs-jules[bot]@users.noreply.github.com>
@google-labs-jules
Copy link
Copy Markdown

👋 Jules, reporting for duty! I'm here to lend a hand with this pull request.

When you start a review, I'll add a 👀 emoji to each comment to let you know I've read it. I'll focus on feedback directed at me and will do my best to stay out of conversations between you and other bots or reviewers to keep the noise down.

I'll push a commit with your requested changes shortly after. Please note there might be a delay between these steps, but rest assured I'm on the job!

For more direct control, you can switch me to Reactive Mode. When this mode is on, I will only act on comments where you specifically mention me with @jules. You can find this option in the Pull Request section of your global Jules UI settings. You can always switch back!

New to Jules? Learn more at jules.google/docs.


For security, I will only act on instructions from the user who triggered this task.

Copy link
Copy Markdown

@gemini-code-assist gemini-code-assist Bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Code Review

This pull request updates the success detection logic to include a new flag, refines the UI styling for lore cards, and improves the robustness of the emergency ticker and leaderboard metrics. Key changes include using data frame copies to avoid SettingWithCopyWarning, ensuring the presence of the is_success column, and adjusting the leaderboard to include models with zero cleared sectors. Feedback was provided to improve the leaderboard data processing by handling potential null model names and ensuring type consistency for the sector counts.

Comment thread airtbench/frontend.py
Comment on lines +757 to +759
# Ensure all models in failure_df are included, even if they have 0 cleared sectors
all_models = pd.DataFrame(failure_df["model_display"].unique(), columns=["Model"])
model_stats = all_models.merge(model_stats, on="Model", how="left").fillna(0)
Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

medium

The current implementation of the leaderboard metrics can lead to data display issues if the telemetry contains missing model names. failure_df["model_display"].unique() includes NaN values if present, and the subsequent fillna(0) on the entire DataFrame will convert these NaN model names to the integer 0. It's better to exclude NaN models and specifically fill the Sectors Cleared column to maintain type consistency.

Suggested change
# Ensure all models in failure_df are included, even if they have 0 cleared sectors
all_models = pd.DataFrame(failure_df["model_display"].unique(), columns=["Model"])
model_stats = all_models.merge(model_stats, on="Model", how="left").fillna(0)
# Ensure all models in failure_df are included, even if they have 0 cleared sectors
all_models = pd.DataFrame(failure_df["model_display"].dropna().unique(), columns=["Model"])
model_stats = all_models.merge(model_stats, on="Model", how="left")
model_stats["Sectors Cleared"] = model_stats["Sectors Cleared"].fillna(0).astype(int)

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant