I have always believed that if a client wants to see you, they will cover your costs.
Like you, am at the higher end price wise. I am also outcall only (London, zone 1), so my price is inclusive of travel. More often than not, particularly in winter/when the weather is bad, clients always add in extra money to cover my cabs. Totally unnecessary in my opinion, but they insist. (The beauty of only seeing regulars). Any travel outside of London zone 1 and overseas is always paid for by the client.
If you decide to start charging for travel, state it clearly in your ads.