I would say ignoring him and not initiating any conversation would be best. You said you live in a central part of town, so you can bump into him again. Maybe if you deny any occurrence of the event, he will start believing that he spoke to the wrong person? Otherwise, if you write to him and acknowledge what happened, he'll know it's you, and he'll know he upset you, so he might try again in the future.