It started as both for me, but now it's mostly physical.
A girlfriend asked me to give her anal. It was just so unexpected and DIRTY. It turned me on like nothing had turned me on before. And it felt great physically, too.
A couple of years later, my best friend (whom I'd been giving blowjobs to for several years) out of the blue declared, mid-blowjob, that he wanted to fuck me! I was startled, as well as a little scared and aroused. I told him OK, but then I changed my mind and told him he could top me only if I could top him first. He didn't like that. We had a "battle of wills" about it for the next few months until finally, I gave in and let him take my anal virginity. It was a HUGE turn-on for him to top me and it was mostly mental. But it also felt great physically after I learned to relax.
Now I give my wife anal sex and she pegs me. It's mostly about the physical pleasure but when she pegs me, there is still a strong mental component. I like for her to be very dominant about it!