There are differences across human populations, yes. For instance. Southeast Asians commonly have a mutation that causes them to be intolerant of alcohol. About 1 in 27 Jews are carriers for Tay-Sachs disease.
My point is that race as most people conceptualize it is not based in fact. I’ve recently watched a TED talk that pointed out that a Nigerian and a Kenyan are more genetically distinct than a Nigerian and a European (can’t recall the exact country in Europe). Our racial divides in society are primarily a product of biased psychological processes that automatically categorize a light-skinned and dark-skinned person as more different than they are. After all, the dark skin of African natives is a product of a single gene involved with melanin production, which is an adaptation to the intense climate and sunlight of Africa.
My point about genetic differences isn’t so much that species are more different than we think. My point is more that our brains are programmed to be keenly aware of differences, even small ones. A 0.3% difference between humans and Neanderthals would certainly be enough for our biased brains to lock onto minute differences and produce erroneous stereotypes and prejudices that would lead to conflict, especially since this happens with a 0.15% difference between human populations.