Neural network design still relies heavily on manually specified architectural choices, such as operator types, layer composition, width, and depth, which limits adaptability across tasks and modalities. To address this issue, we propose E-KAT (Evolutionary KolmogorovArnold-Attention-Toeplitz Network), a unified neural framework that integrates multiple structural paradigms within a single self-adaptive neuron. Specifically, E-KAT combines cross-layer attention, Toeplitz-based local operators, identity mappings, residual propagation, and edge-wise nonlinear functions inspired by the Kolmogorov–Arnold framework, and learns their relative contributions end-to-end through adaptive structural coefficients. In addition, we introduce an edge-level threshold activation mechanism with continuous soft gating and a learnable temperature parameter, enabling differentiable control over connectivity and effective information flow. These neuronal and edge structures jointly form neural clusters for single-task learning, while multiple clusters can be further coupled to support cross-task and cross-modal feature integration.

Extensive experiments on MIT-BIH ECG, WAY-EEG-GAL, CIFAR-10, and DEAP demonstrate that E-KAT can automatically learn task-dependent structural preferences and sparse connectivity patterns. The proposed model achieves 98.63% accuracy on MIT-BIH-ECG, 96.77% on WAY-EEG-GAL, and 91.82% on CIFAR-10, while maintaining lightweight complexity, requiring only 0.54M parameters and 0.0246G MACs on ECG, 1.60M parameters and 0.1227G MACs on EEG, and 3.51M parameters and 0.10G MACs on CIFAR-10. Further analyses show that the learned connectivity patterns provide interpretable signals for effective width and depth, and that multi-cluster coupling improves robustness and feature integration in multimodal regression tasks. These results indicate that E-KAT provides an efficient and flexible alternative to manually designed architectures across heterogeneous domains.