It depends on the circumstances I think, say you're invited to a jam party with a few musos ... and the A$$#@^* is there too, it could happen, and despite your/my appreciation of the guy in question, something magical could happen, musically speaking of course. It also depends if the guy think of you/me the same way, or if he actually like you/me, if he likes you/me, it could also happen. Other than that, I don't think so.
I remember reading an interview of Simon Phillips back in the 80's, he was saying that people (which he didn't like) kept calling him back for session work, and he added that it didn't matter how obvious his behavior was when he recorded with them, they still called him back, of course he declined the offer, stating he had other projects, but he was amazed they called again, he thought he made a clear statement, personality wise, and that he would never had to anything with them anymore.
Which brings into the discussion, the world of session recordings, I don't think that these top-session drummers, like Phillips, Gadd, Colaiuta, Keltner and the likes are finding all the people they have to work for really nice persons, and I'm sure that sometimes they do think "what an A$$#@^* "
... I guess it's part of the job and they've been hired by the producers and not the artists themselves.