Fix dispatcher callback ownership and shutdown races

Fix dispatcher callback ownership and shutdown races #4975