No, I find most book me before, or immediately afterwards, which actually can make me busier. I don't usually get any complaints, apart from one person who thought because I didn't phone his as requested during a holiday), that I just couldn't be bothered. This was despite it being clear on my profile and website that I was taking a break, and him receiving an vacation reply.
I tend to have more problems with my regular day/days off each week not suiting, rather than taking a week or 3's break.