I would reply and say yes, you'll see him later and won't be able to reply to any further texts as you're busy.
If he's really becoming a pain just stop seeing him altogether, they're just not worth it when you spend more time texting them than actually seeing them.
You could also try telling him you're an escort and not a phone chat operator so you're not able to chat etc when it's not regarding a booking.