The United States was over 90% White at that time. You can't have racism where there is only one race and no one to be racist towards. Racism can by definition only exist in multiracial societies.
I'm not sure that you are correct in those numbers. And back in the 1850s-1860's, blacks in some areas outnumbered whites. Remember Slavery was system of free labor, and any business tends to have more laborers than the people who run it. Take a look at that car, that is a 1930s to early 1940s car--- you are then looking at Jim Crow Laws, lynchings and such, you might want to Read some of Richard Wright's earlier works (like Black Boy) which describe the era, for black as that time. You know, like eating in the back, after everybody else has finished.
Youeally need to do you research on our Laws and social morays at the time.
By the way, I am really feeling