Atrial fibrillation (AF) affects millions of people in the world, causing increased morbidity and mortality. Treatment involves antiarrhythmic drugs and catheter ablation, showing high success for paroxysmal AF but challenges for persistent AF. Experimental evidence suggests reentrant waves and rotors contribute to AF substrates. Ablation procedures rely on electroanatomical maps and electrogram (EGM) signals; however, current methods used in clinical practice lack consideration for time–frequency varying EGM components. The fractional Fourier transform (FrFT) can be adopted to capture time-varying frequency components, thereby enhancing the comprehension of arrhythmogenic substrates during AF for improved ablation strategies. To this end, a FrFT-based algorithm is developed to characterize non-stationary components in EGM signals from simulated AF episodes. The proposed algorithm comprises a pre-processing step to enhance the coarser features of the EGM waveform, a windowing process for dynamic assessment of the EGM, and a FrFT order optimization stage that seeks compact signal representations in fractional Fourier domains. The resulting order is related to the rate of frequency change in the signal, making it a useful indicator for frequency-modulated components. The FrFT-based algorithm is implemented on EGM signals from AF simulations in 2D domains representing a region of the atrial tissue. Consequently, the computed optimum FrFT orders are used to build maps that are spatially correlated to the underlying propagation dynamics of the simulated AF episode. The results evince that the extreme values in the optimum orders map pinpoint the localization of fibrillatory mechanisms, generating EGM activation waveforms with varying frequency content over time. © 2024 The Authors