Racism is always present even if it doesn't manifest itself as overt prejudice. Even though the didactic messages I was getting when I was younger were quite the opposite, I knew intuitively that it was "bad" to be black--not that black people were bad, mind you, but that being black just wasn't as good as being white, no matter what kind of person you were. My confusion was entirely because I expected more connexion between slurs and actual physical characteristics than there was in this particular case.

I suspect the same with your family. They always knew it was "better" to be Spanish than to be some sort of dirty foreigner, it's just that there's no real need to point this out if there aren't any dirty foreigners around.