Size isn't as big of a handicap as people present it. If you understand what you are doing then you can beat someone bigger, taller, stronger, heavier. Here's a few examples. Real fights of untrained people. One bigger the other smaller.
I train with a self-defense focus, so my training takes into consideration there will be a size, weight, height, age, and strength disadvantages for me. Granted that I way 200lbs now at 5'9. It doesn't mean that someone who weighs 160 at 5'9 can't beat me. In self-defense you have to have strategies for these type of scenarios.