Initially, you made some innocent "white lies" to cover it up. Lots of people do and on the other hand, there are people who see these white lies as "plain lies". Some people tell white lies because telling the truth hurts them.
It depends on how long you know this chap. If it's a casual date, why should you care? If he's a good BF material, he will eventually find out, not something you could simply keep it hidden. Some men mind (faithfulness is thought to be important amongst men) and some men don't. Lots of double standards, too.