Yes, I don't usually leave feedback for a cancellation but if it's several times in a row or more times than I've seen the guy, then yes I would be keeping the booking form and leaving neutral then if it continued, negative. I'd say something to the effect that I'd seen him, he was fine in the booking but had since become unreliable.
Quite a lot of guys on AW do continue to gain positive feedback, whilst double booking and messing others around.