Once more than one person starts offering sex somewhere, even if that's not at the same time, it's a brothel.
That's illegal to own, run, or allow in premises you control... but not to work in. So often, only one of the two would be acting illegally - the one on the lease.
Whether anyone cares depends on a lot of things, particularly where you are and how much you annoy the neighbours.