It sounds like your agency isn't really set up for ladies like yourself. If an agency has nothing but young, slim ladies available then all their loyal clients will be men who prefer young, slim ladies. It will take a while before men who prefer your type notice that an agency that used to have nothing for them, now caters to their tastes.
I think if you want agency work you would do well to have a look around and see if there are any agencies that have more ladies of your age/body type working there - as that is where your client base will be. Being the new girl is usually a good way to get lots of interest so either your agency isn't working hard for your interest, or their client base just isn't the right one for you.
If there isn't an agency that you feel will represent you well, then definitely look into going indie. It can seem scary but the money is better and it gives you a lot more freedom.