I have done this a few times with one of my regulars, so my advice would be:
Find a safeword. Make sure he knows if you use it, that everything stops immediately. Don't use "no/stop" etc. as the safeword, given the type of roleplay. I use red.
I see you're thinking of refusing the gag, but make sure you have a non verbal safeword (leg tapping or two fast head shakes for example) in case you can't speak for other reasons at any point.
Discuss beforehand how rough you are willing to get. Are you going to allow bruising? Slapping? Or are you only going to allow light markings/reddening of your skin and so on? All of this needs to be communicated and agreed before you begin.
Be very clear that he still needs to wear a condom. During my scenes, my regular 'forces' me to put the condom on him while making comments about fucking me. We can stay in character that way, while still being safe. If this falls out of character for you, he can use it as a humiliation play (if you offer that.) For example, he is using it to protect himself, not you.
Depending how rough you are willing to go, be prepared to ache the day after and possibly not work. Everywhere, not just vagina/ass. I'm quite sore each time over my entire body because of the 'struggle.'
And most importantly, make sure you trust him enough to be responsible with this type of roleplay. It CAN go horribly wrong if you're not BOTH vigilant. It can also be a whole lot of fun if you're both careful and on the same wavelength.
Good luck!