Description
Create utilities to visualize which weights/neurons/filters are being pruned by L0.
Motivation
Understanding sparsity patterns helps debug issues and provides insights into what the model considers important.
Proposed Visualizations
from l0.visualization import plot_sparsity
# 1. Layer-wise sparsity
plot_sparsity.by_layer(model) # Bar chart of sparsity per layer
# 2. Weight heatmaps
plot_sparsity.weight_heatmap(model.fc1) # Heatmap of gate values
# 3. Sparsity evolution
plot_sparsity.evolution(checkpoint_dir) # Sparsity over training
# 4. Filter importance (for CNNs)
plot_sparsity.filter_importance(model.conv1) # Which filters are kept
# 5. Attention head importance (for Transformers)
plot_sparsity.head_importance(model.attention)
Features
- Matplotlib/Plotly backends
- Interactive visualizations
- Export to images/HTML
- Integration with TensorBoard/WandB
Description
Create utilities to visualize which weights/neurons/filters are being pruned by L0.
Motivation
Understanding sparsity patterns helps debug issues and provides insights into what the model considers important.
Proposed Visualizations
Features