It was Malcolm X I think who said "Racism is as American as apple pie." I think he was wrong, it's part of the human condition, not just American. Racism, and more generally fear and hatred of those not like us, is evolutionarily adaptive. If you can notice that that guy coming over the hill is not part of your tribe, from far away, you may be able to alert your tribe and they may be able to be ready for the attack and not get slaughtered. Other people's skin color among other things is direct wired through your neurons into some alert/thisisbad center in your brain.
Our task as modern humans (basically just barely crawled out of the caves) is to be damn aware that these brain connections exist, and simply choose to act not according to them, because we know intellectually that skin color is just skin color, nothing else, no real differences. Just like somewhere in our caveman brain we want to kill the guy who's with a woman and drag her away and rape her, but we don't do that because we know intellectually that society has organized itself with rules and traditions against that. Etc. etc. for all the other impulses our lizard brains feed us. THAT is called being not racist.