I don't go to private residences, and this is part of why. It's not the wife, exactly, but the idea of getting so intimately involved in the punter's personal life. My primary reasons for not going to residences is I don't feel safe and I'm afraid of finding a filthy house with a yapping dog - but this sort of thing is also part of it. I prefer either to see them in my own work-dedicated flat or in an anonymous hotel room. It just sits better with me.
That said, I'm not asked to go to residences very often, anyway. I work in a city centre and I won't go further than a few minutes by cab. So, hotels are numerous and that's enough for me.