Typically, the difference is as stated - the length of time a person convicted of a crime can be imprisoned. In some states, misdemeanor sentences are served in local jails, either city or county; whilst felony convictions are served in a state prison facility. If a crime cannot be punished with imprisonment, but only a fine, it is not a misdemeanor or a felony, it is an infraction. Examples of infractions are parking tickets.
Interesting side-note: most states do not have a provision for assignment of counsel (free lawyer) or a right to trial for some infractions. In other words, you can't 'take it to court' even if you wish to. It's administrative, not criminal.
Also, and again this varies by state, a person convicted of a felony loses some of their basic rights all US citizens have. Those are most often loss of the right to vote - in any municipal, county, state or federal election - and the right to own or be in possession of a firearm. In some states, this loss is permanent. In some states, the loss is for a certain time period. In some states, a convict can petition to have their rights restored after a period of time, which they may or may not get back.
There are also federal convictions, which although not technically classified as 'felonies', have the same approximate impact and are considered as if they were felonies.
Most states make it clear in their criminal statutes whether or not a crime is a felony or a misdemeanor, but sometimes it is a matter of either degree (1st degree assault versus 3rd degree assault for example) or it is left up to the prosecution (so-called 'wobblers') and could be prosecuted as either a felony or a misdemeanor, depending on how the DA feels about it.
Many felonies have a lesser included offense which is a misdemeanor, which gives judges, prosecuting attorneys, and juries the ability to either seek or deliver a lesser conviction if conditions warrant it.
Most misdemeanors are not extraditable across state lines - some not even from city to city within a state. This often stems simply from budget concerns. Felonies are often, but not always, extraditable. So a person accused of a misdemeanor could seek refuge in another state, but a person accused of a felony could not; if arrested, they would find themselves taken back to the state where the felony allegedly occurred to face trial.
All misdemeanors have a statute of limitations which preclude prosecution if the crime is more than a certain number of years old. Some felonies have statutes of limitations, but the most serious often do not. Felonies also often do not have a statute of limitations if an arrest warrant has been issued - so thirty-year-old warrants for felonies are sometimes still prosecuted.