Some medications from alcohol (as some people have said) to some pain killers (Oxycodone is notorious for this, as is codeine, while Vicodin doesn't usually have this side effect, go figure) to SSRI antidepressants (Prozac being the most notorious) to marijuana to blood pressure medications and others can have an effect on the ability and time it takes to orgasm.
Or it could be just the way he is. I used to get (silently) frustrated with my husband, after we had been together a long time (he's older than I am) and he started taking a long time to come. I was, however really patient with him. He still often needs repeated types of stimulation, oral, vaginal, anal, manual etc to come.
God "Got me" as they say. About 3 or 4 years ago, I started having terrible difficulty orgasming, and he was right there helping me in any way he could. He bought me my first sex toy in our attempts to get this issue solved. I still sometimes take a while, but I learned, "what goes around, comes around." Despite the fact that I was frustrated with him at one point, (and once after an hour long blow job finally yelled, "OMG, quit enjoying it so much and just come !!!!") I helped him to every orgasm he could get, and we both learned sometimes you have to give up and try in the morning. That being said, when it started happening to me, he was there to help me through it.
Good luck. I don't have any suggestions, besides varying the stimulation and to tell you that if the only way he can come sometimes is to jerk off while you give him head (I've been accidentally punched in the mouth more times than I can count) then that's OK! At least he got there.
He doesn't have to come during PIV sex, and neither do you. Eventually you'll both get there and under what circumstances isn't really important.