I would say definitely go with your gut.
But I do want to ask: did you give a postcode too? Because if I did incalls at home and gave out any one of SIX postcodes (all 7 digits) it would pin me down to one specific road - which isn't more than a couple of 100 yards long!
If you gave just a vague area - such as "Clapham Common" and he said "oh yes, Cedars Road?" (check this up on the map, you'll see what I mean) then he's either 1) really good at guessing, 2) an old client who wants to pretend he's new (for some reason), 3) a friend of an old client that's not very good at pretending he got your details from a friend, or 4) psychic.