Performance Comparison of Music, Root-Music, And Esprit Algorithms for DOA Estimation Under Rayleigh Fading Channels

Abstract

Accurate estimation of the Direction of Arrival (DOA) plays a critical role in mod- ern array signal processing applications, including radar, sonar, and wireless communications. In multipath-rich environments such as urban wireless systems, signal degradation due to Rayleigh fading poses significant challenges for DOA estimation. This study presents a comprehensive com- parative analysis of three prominent subspace-based DOA estimation algorithms—MUSIC, Root- MUSIC, and ESPRIT—under Rayleigh fading conditions. Using a Uniform Linear Array (ULA), simulations were performed with signals arriving from known directions at 30°, 0°, and +30°, across varying Signal-to-Noise Ratios (SNR), snapshot counts, and array element configurations. Performance metrics including Root Mean Squared Error (RMSE) and resolution capability were employed to quantify the effectiveness of each algorithm. Results show that while MUSIC offers high resolution under ideal conditions, it is sensitive to noise and computationally intensive. Root- MUSIC reduces complexity and improves stability, but ESPRIT emerges as the most robust and accurate algorithm across a wide range of parameters due to its closed-form solution and resilience to noise. The simulation findings offer practical insights into algorithm selection for real-world deployment in wireless and defense-related systems. Additionally, the proposed system leverages Rayleigh fading modeling to better reflect real-life propagation environments, providing a novel perspective often neglected in traditional DOA studies. Future work aims to integrate data-driven approaches to further enhance estimation accuracy. The outcomes of this study are valuable for engineers and researchers working on adaptive antenna arrays and next-generation communication systems.