I agree that every situation is different, and that some fights may last longer for all sorts of reasons (e.g. more people join in, the fighters don't know their limits and continue pummelling). Even if we were to look at MMA matches, entertainment wrestling matches and etc, there are just too many variables to consider. Some fights end in a few seconds with a single KO, while some can just go on and on.
On the question of fitness, stamina and endurance, there's no doubt I have found that to be important, and people don't give it as much emphasis than I think (i.e. people prefer to have more power than stamina). When I first entered my martial arts school, I thought I would be quite alright considering I have been weight-lifting and doing cardio for a measurable period of time. Some of the students are also fatter and older than me so I thought I should, at the least, be able to last longer.
I was not only sadly mistaken but humbled very much. Guys older than me by 20 years, shorter and fatter could kick harder and longer than me. As I read up more, I realised I was narrow-minded and mistaken to think my condition in running/sprint could be translated to better stamina in martial arts conditioning. It does help, of course, but it's just not the same thing. Further, as time goes by in a fight, you will find that you can't punch and kick as well due to the loss of energy.
Anyway, back to the street situation. Most likely, a trained person should be able to end a fight relatively quickly against someone who isn't trained, but I would not guarantee that the trained martial artist would win. A swinging hit from a brawler still hurts whether trained or not, and in a fight, other variables come into place (e.g. accuracy, timing, power, mental tougness). I also think luck has a lot to do with it.