For the soreness, how used are you to having a couple of partners in a night and how vigorous was the sex?
For the fingering, from what I gather, and amongst the reasons why I hate fingers in my bum, the problems are more often nails and roughness. But yes, I'd stop offering it, largely because you're not keen on it. If you were more ok with it, you could try gloves.