I have a feeling that somehow this is white people's fault.
Blame Hernán Cortés.  A Spanish Conquistador who brought slaves over from Africa to North America in 1526.  Once the English colonies were established in 1619 the demand for slaves increased exponentially.