Hi adilee and welcome,
It's not clear from your post what you think you could be charged with - what are your neigbours trying to 'prove'? It really depends on whether or not you were breaking the law (providing a place for others to work, even occasionally could get you saddled with brothel keeping charges, for example) but there isn't anything I can think of that specifically would relate to Adultwork and working as a prostitute is not illegal.
Have you been told what you are supposed to be 'guilty' of, or on what grounds they have contacted the police? If you are making a lot of noise at night, say?