ProjectTo(::Hermitian) keeps the imaginary part of a Diagonal tangent #722

The Diagonal method of ProjectTo{Hermitian} keeps the imaginary part of the diagonal, so its result is not Hermitian. The dense path drops the imaginary part. So the same tangent projects to different values depending on whether it comes as a Diagonal or a Matrix:

julia> using ChainRulesCore, LinearAlgebra  # ChainRulesCore v1.26.6

julia> H = Hermitian([1.0 2im; -2im 3.0]);

julia> dx = Diagonal([1.0 + 2im, 3.0 - 1im]);

julia> d = ProjectTo(H)(dx)
2×2 Diagonal{ComplexF64, Vector{ComplexF64}}:
 1.0+2.0im      ⋅
     ⋅      3.0-1.0im

julia> ishermitian(d)
false

julia> ProjectTo(H)(Matrix(dx))  # dense path: (dx + dx')/2 drops the imaginary diagonal
2×2 Hermitian{ComplexF64, Matrix{ComplexF64}}:
 1.0+0.0im  0.0+0.0im
 0.0-0.0im  3.0+0.0im

The cause is in src/projection.jl. The Diagonal method is generated in the same loop for Symmetric and Hermitian, and it projects dx.diag with the parent's element projector only:

function (project::ProjectTo{$SymHerm})(dx::Diagonal)
    sub = project.parent
    sub_one = ProjectTo{project_type(sub)}(; element=sub.element, axes=(sub.axes[1],))
    return Diagonal(sub_one(dx.diag))
end

That is correct for Symmetric. For Hermitian, the diagonal also has to be real.

The imaginary part pairs to zero with every Hermitian tangent, so pullbacks still give the right gradient value. But the result is outside the tangent space, so for example a gradient step can leave the set of Hermitian matrices. ChainRules.jl hit this in JuliaDiff/ChainRules.jl#856: the Matrix(::Hermitian) rrule now uses ProjectTo, while the hand-written rule it replaced returned real.(Δ) for a Diagonal cotangent.

A possible fix is to apply real to the diagonal in the Hermitian case, e.g. Diagonal(sub_one(real.(dx.diag))) (or real ∘ sub_one, whichever keeps the element type right), together with a test next to the existing Diagonal projection tests.