Personally, I think the worst aspect for me is the sporadic income. I've just recently went full time and am really struggling.
Lying to everyone about where I work is also a lot of effort. Got to try and remember all the fake co-worker names, and come up with some things that happened at work, etc.
Hanging around waiting for the phone to ring is a pretty annoying feeling.
I also find that it can take away from your regular sex life. One in that you just don't have the time. I used to have quite a bit of casual sex but now don't really want to see someone in case I get a booking. For me as well I don't want to ruin the things I actually really enjoy irl, I'm into dominating and don't do it professionally (though I'd probably earn a lot more money if I did) as it ruins my enjoyment of it to do it with guys I don't find attractive.