You can miss a voice swapping out mid-conversation if you're not listening for it

Change deafness is the auditory version of failing to notice an obvious visual change: listeners asked to track one thing in a complex soundscape routinely miss another sound object changing entirely, even a speaker's voice swapping between sentences. Studies find that cueing attention toward where a change might happen sharply improves detection, while scenes with more than about four competing sound sources make missed changes the norm rather than the exception.

— Wikipedia contributors, Change Deafness — Auditory attention research, early 2000s onward