Another quote by The Great Emancipator.

"I will say, then, that I am not, nor ever have been , in favor of bringing about in anyway the social and political equality of the white and black races - that I am not, nor ever have been, in favor of making voters or jurors of negroes, nor of qualifying them to hold office, nor to intermarry with white people; and I will say in addition to this that there is a physical difference between the white and black races...I, as much as any other man, am in favor of having the superior position assigned to the white race." -Abraham Lincoln
Furthermore, Virginia, Tennessee and North Carolina seceded because they feared an invading army.
Stonewall Jackson made it clear:
[ Invalid YouTube link ]