I'm not sure that people can be classified as smart and not-smart anyway. People are just different. I think people's minds just work differently not faster or slower, or better or worse. Some people are good in thinking about and understanding abstact ideas, some have better spacial skills,others are perceptive, some can memorize facts, and some have good life skills that you can't learn in school, some can tell other people's emotions, some are inventive and creative, everyone is different. It's not really fair for someone to say that someone isn't as smart because it's not that they aren't smart it's that they think in a different way.
Besides, in a population where the the people are biased against "not-smart" people would be fair for the people considered "less smart" and there would be no real way to test for the inteligence of a person anyway, the IQ test itself has been considered "obsolete" for a while now.
Edit-sure put a lot of " " didnt' I? Sry about that.