Very worrying. I would try to discreetly take a picture next time. In the mean time log the time and day that it happened and if he turns up again, call the police. Also, check that your door is secure and that the camera facing the door is hidden and is working.
I've had something like this before, it is very worrying. Please stay safe. x