Shonali lux layer - #13
Conversation
Create layers directory, add reusable DirectedHypergraphLayer, regression model, and initial tests.
| """ | ||
| safe_column_normalise(M) | ||
|
|
||
| Normalise each column of `M` by its column sum. | ||
|
|
||
| Columns with a sum of zero use a denominator of one, preventing division by | ||
| zero while leaving those columns unchanged. | ||
| """ | ||
|
|
||
| function safe_column_normalise(M::AbstractMatrix) | ||
| col_sums = sum(M, dims = 1) | ||
| safe_sums = similar(col_sums) | ||
|
|
||
| for i in eachindex(col_sums) | ||
| safe_sums[i] = col_sums[i] == 0 ? one(eltype(col_sums)) : col_sums[i] | ||
| end |
There was a problem hiding this comment.
Why not combine this with safe_row_normalize and just have the dimension be an input parameter?
| """ | ||
| safe_row_normalise(M) | ||
|
|
||
| Normalise each row of `M` by its row sum. | ||
|
|
||
| Rows with a sum of zero use a denominator of one, preventing division by | ||
| zero while leaving those rows unchanged. | ||
| """ | ||
|
|
||
|
|
||
| function safe_row_normalise(M::AbstractMatrix) | ||
| row_sums = sum(M, dims = 2) | ||
| safe_sums = similar(row_sums) | ||
|
|
||
| for i in eachindex(row_sums) | ||
| safe_sums[i] = row_sums[i] == 0 ? one(eltype(row_sums)) : row_sums[i] | ||
| end | ||
|
|
||
| return M ./ safe_sums | ||
| end |
|
|
||
| A Lux-compatible message-passing layer for directed hypergraphs. | ||
|
|
||
| The layer accepts a species-feature matrix together with source and target |
There was a problem hiding this comment.
"species" is specific to CRNs. Remember that HyperGraphNeuralNetworks.jl is a generic hypergraph ML library.
| 1. A learnable transformation of species features. | ||
| 2. Separate aggregation of source and target species into reaction embeddings. | ||
| 3. A learnable transformation of reaction embeddings. | ||
| 4. Propagation of reaction messages back to participating species. | ||
| 5. A learnable update of the species embeddings. | ||
|
|
There was a problem hiding this comment.
See above. The layers that you develop should be general to any (di)hypergraph.
|
|
||
| # Arguments | ||
|
|
||
| - `species_in_dim`: Number of input features associated with each species. |
There was a problem hiding this comment.
Should be something like vertex_in_dim. You should also have a hyperedge_in_dim, I would think.
|
|
||
| struct DirectedHypergraphRegression{H, R} <: | ||
| Lux.AbstractLuxContainerLayer{(:hypergraph_layer, :regression_head)} | ||
|
|
||
| hypergraph_layer::H | ||
| regression_head::R | ||
| end |
There was a problem hiding this comment.
Again, think about your end user. Is this something that an average end user will want, or will they be designing their own network architectures? I think likely the latter. HyperGraphNeuralNetworks.jl should (mainly) be general-purpose features that could be used for HGNNs. We shouldn't be pigeon-holing folks into one particular architecture.
| - `hidden_dim`: Size of the hidden embeddings. | ||
| - `activation`: Activation function used in the hypergraph layer. | ||
| """ | ||
| function DirectedHypergraphRegression( |
There was a problem hiding this comment.
Here, you're assuming a single message-passing layer, we we discussed. At very least, you should have a variable number of layers.
| @@ -0,0 +1,47 @@ | |||
| using Test | |||
There was a problem hiding this comment.
Personal style, but unless your tests are massive, I prefer to have one test file with multiple @testsets
| ps, | ||
| st | ||
| ) | ||
|
|
There was a problem hiding this comment.
You're testing that the dimensions are what you expect and that you don't get any exploding infinite numbers. Is that all you need?
| using SimpleHypergraphs | ||
| using SimpleDirectedHypergraphs | ||
| using HyperGraphNeuralNetworks | ||
| include("layers/DirectedHypergraphLayer.jl") |
There was a problem hiding this comment.
If you're exporting properly, you shouldn't need to do this.
|
Solid improvement on the test. Unless I missed it, you're still not testing gradients/differentiation (which is important for backpropagation and ML). Take a look at how The Also, for the final PR, please remove your sandbox files (i.e., all of your numbered files, like examples/shonali_prototypes/12_formose_crn_case_study.jl). These shouldn't be part of the main codebase. |
Codecov Report✅ All modified and coverable lines are covered by tests. Additional details and impacted files@@ Coverage Diff @@
## main #13 +/- ##
==========================================
+ Coverage 92.19% 92.72% +0.52%
==========================================
Files 8 10 +2
Lines 1960 2102 +142
==========================================
+ Hits 1807 1949 +142
Misses 153 153 ☔ View full report in Codecov by Harness. 🚀 New features to boost your workflow:
|
Added additional unit tests for DirectedHypergraphLayer, including constructor validation, parameter/state initialization, forward-pass behaviour, and invalid input handling. All project tests pass successfully.
## Summary This PR introduces a new `DirectedHypergraphAttentionLayer` for Lux-compatible directed hypergraph neural networks. ### Features - Added a directed hypergraph attention layer supporting: - source and target attention mechanisms - optional hyperedge features - optional return of attention weights - Added helper functions for masked attention computation and safe normalization. - Added comprehensive unit tests covering: - constructor validation - parameter initialization - forward passes with and without hyperedge features - attention weight behaviour - deterministic zero-parameter behaviour - input validation and error handling All tests pass successfully.
3 participants
Description
This PR introduces an initial Lux-based implementation for directed hypergraph neural networks.
Summary of changes
layers/directory to organise neural network layers.DirectedHypergraphLayerusing Lux.jl.DirectedHypergraphRegressionmodel built on top of the directed hypergraph layer.This PR is intended as an initial implementation for review and discussion before further development.