When I started if I'm completely honest, I did struggle with people I found unattractive. I think that's part and parcel of finding out if you can do the job. In our civvy lives we choose our partners / fuck buddies etc because we fancy them. We don't get the option of choosing in this job, in fact most of the time with someone new, I have zero idea of their appearance, size, age etc.
The difference is now that over time, someone's appearance matters less and less. As others have said, attitude and personality go much further towards an enjoyable booking and someone's appearance matters less. In fact if you do enjoy the job and stick with it, you quickly realise that appearance does not matter at all, and for me it's how I get along with them that matters, how comfortable I feel in their company. Appearance is irrelevant.
You have to try and see how you get on. You will get clients you do find attractive! But only time & experience will tell how you feel about being with men you don't fancy. Don't worry too much!