Guessing by the mention of BP you're US-based, and I have absolutely no experience with NA as a whole, but I figure I'll leave my opinion all the same.
I finish work at 8:00 pm normally, but I will take bookings around the clock as long as it's an advance booking.
The presence of crazies is honestly more down to where you live and how you act than when you're out, in my experience. I'm notoriously out and about at strange hours (I go out to train at around 4 am), and out of three countries I've lived in or stayed in for a decent amount of time, I've only ever been halfheartedly verbally harassed by drunk people a couple of times. Having been on the "hooligan" end of equation as well, I can tell you that I'd be way more likely to pick on people who look out of place (overdressed, skulking or looking uneasy). Not looking like an appealing target goes a long way.
If you live in a town that is known for being full of questionable activities or having high crime rate, you have reasons to be concerned in shoddy areas even during daytime. If the place you're in is relatively calm, then nights should not pose that much more of a risk.