Brain damage can erase the left half of your imagination, not just your sight
People with right-parietal damage often ignore everything on their left, even in memory. Asked to imagine standing at one end of Milan's cathedral square and name the buildings around it, Edoardo Bisiach's patients listed only the ones on the right; asked to imagine the opposite viewpoint, they named the previously 'missing' buildings and dropped the ones they'd just given. The neglect followed imagined space exactly as it follows real space.
— Edoardo Bisiach and Claudio Luzzatti, Unilateral Neglect of Representational Space — Cortex, 1978