I worked at home on and off a few years ago.
There were problems just the same as occur on tour.
Examples:
Breaking my shower.
Turning up without an appointment.
Upsetting neighbours with not following parking instructions.
Thinking it was okay to park outside my house if they had visited before.
Being overly nosey in my house.
I would do it if you have to but offer hour long appointments only for discretionary purposes.
Also, get them to park away from your house and stay on the phone with them whilst they're walking to you and make sure you can see them first before letting them in. My house number isn't visible from the road so this was a good option for me.