Hi Rose,
I work from home and I have young children. I do sometimes get a bit paranoid about people noticing, however my house is quiet discreet and unless the people opposite kept a vigil beside their front window 24/7 I'm not sure how obvious it is as I don't see many clients.
Regarding SS, give them a call and ask them their opinion, I did just that and their response was as long as children aren't at risk, it's not illegal.
I think you would have to worry if your house was a state and children looked like ragamuffins, then SS would have more on you than escorting.
At the end of the day this is how I make a living, if anyone doesn't like it, i'd suggest they pay my mortgage etc. It's not possible to work from hotels, as a single parent I simply can't afford to pay for rooms and then risk no bookings.
Plus, this is my home, I'll do what I want in it!!!