As Eve said. Obviously none of us know your situation 100% so can only give advice and opinions, be careful. I’m sure his wife also thinks that they have emotional, committed, loving, maybe even passionate, sex.
I know to us we can say ‘well he chose to come so it’s not our fault he’s married’, and that’s true, if it wasn’t you it’d be someone else, but imagine what his wife would think. The woman who probably washes his clothes and cooks his meals and sleeps beside him peacefully every night thinking that only she gets those parts of him. Now, if he can do this to her, although a blank face to us, still a human who has given herself 100% to this man and would be destroyed by his actions and who he apparently loves, then this is not a good man. This is not a man you would want to gain feelings for or get confused about. Obviously again we don’t know the details 100%, but this is the type of man who will end up messing with your head further if given the chance.