I know what you mean about "something about his voice". It shows in his voice. I refused a booking based on the sound of his voice when I first started out as a newbie. It turned out he had several aliases (which came up a few weeks afterwards, then followed by No show in red) and his booking was "malicious" - he wanted to cause trouble / gain access to my personal address etc. He left abuse in "feedback only" column. But it could have been a lot worse. If you are seen as "prolific", they might try to raid your premises or god knows what else.
If they refuse to park their car at the spot you specifically requested, they may well have the reasons why. Some could be perfectly "innocent" - (e.g. they just like to annoy you because they have to pay to have sex with you and you seem to be having fun at his expense - this line of thinking). Some might not because they don't want you to see what he was really up to.