I’ve got a lot to say about this since public policy & education are my areas of expertise, but I’ll try to stay brief. No promises though!
As we ask why, like @HighEliteMajor says we have to do, the answers to this are very complicated. The state has pretty consistently botched creating equality of opportunity in this country, sometimes intentionally, sometimes with the best of intentions. Just a few examples. Jim Crow obviously was the kind of de jure segregation and oppression nobody should find acceptable in a liberal society. As the SCOTUS ruled in 1954, separate cannot be equal. Even today, by some measures school systems are as segregated as they were right after Brown. We know that integration efforts worked at first, especially in the South. After busing declined and the federal government stepped back in the 80’s (arguably betraying the mission of Brown), segregation increased.
Why is this important? Schools that are predominantly minority are generally worse than schools that are predominantly white. And school quality has a strong, causal relationship with future market and non-market outcomes. See the evaluation of the Perry Preschool program and the reams of studies evaluating charter schools as examples.
Pre-Brown, state and local governments intentionally underfunded school districts that were majority minority. As an example, (and to @Texas-Hawk-10’s point) Texas only spent 83 cents on each black kid for every dollar they spent on whites in 1960. Why? Racism. What happened post-Brown? Magically, funding equalized among schools. Not really magically (it was because legislatures didn’t want white kids to have to go to the old, decaying black schools). The other major piece is that higher quality teachers tend to work in wealthier districts. The district could be better at recruiting, offer a different quality of life, or other factors like rich kids being easier to teach than poor ones. Also to @Texas-Hawk-10’s point, across the country, poor & minority students receive about 2% more funding than their wealthier counterparts, though it’s pretty unlikely that’s enough to bridge the actual gap in terms of costs.
Since schools are residentially assigned here in the U.S., controlling who gets to live where has a strong relationship with school quality. As one of the primary channels through which any given individual’s opportunity in life is determined, this becomes very important. If we look at data on home ownership and lending, we see that blacks have historically faced severe discrimination in ability to receive mortgages and even if they were able to get a loan, many whites wouldn’t sell homes to blacks in good neighborhoods, a set of practices called redlining. Many whites who were dead broke could receive credit where middle- to upper-income blacks couldn’t ever get a similar loan.
This meant many blacks who would move couldn’t and were stuck in a bad neighborhood and possibly a bad school. In addition, blacks weren’t able to access the capital necessary to develop their own neighborhoods. This kind of hopelessness led to many of the societal ills that still exist there. Put simply, the channels to success that exist for whites exist to less of an extent for blacks.
But racism wasn’t the only culprit. I’m a libertarian so I’ll keep dumping on the state In response to this concentrated poverty (that the government in large part created) they tried to fix it with the War on Poverty and related efforts. Some of the programs were effective (SS and Medicare greatly reduced senior poverty, but have severe issues with solvency, thanks Boomers). One major program was AFDC, a means tested cash benefits program for families who had a male in the house who was unable to work for whatever reason. The program was started in the 30’s, but black women weren’t allowed to access it until the 60’s! Crazy. The program was wrought with administrative issues as well. It discouraged marriage since benefits could be cut if a beneficiary married someone with higher income. This drives some of the increase in out of wedlock births among those with low income (who are more likely to be black.) In addition, the phaseout of benefits was very sharp, which discouraged anyone to work since the marginal tax rate (when including lost benefits) became extremely high, more than 100% in some cases.
And we couldn’t talk about failed wars without bringing up the giant cash fire known as the War on Drugs (and the criminal justice system as a whole). Let’s use a specific example. Why does possession of crack cocaine carry much more severe sentences (about 1/3 longer) than powder? Could it have anything to do with the fact that crack is more commonly found in majority black neighborhoods and powder is more common to white neighborhoods? Even though powder cocaine is worth much more in street value? Not difficult dots to connect there. Another question. Why do blacks get pulled over at nearly twice the rate of whites, even controlling for factors like type of car, even though whites are more likely to carry contraband? Another fun fact: marijuana prohibition was largely based on the fear that blacks would “entice” white women to become addicted to weed. That legacy remains. Blacks are 4x as likely to be arrested for weed possession as whites even though they use at essentially the same rate. Blacks are 12x (!) as likely to be falsely convicted of drug crimes as whites. The U.S. Sentencing Commission found that, even controlling for other observables, blacks were given 20% longer sentences than whites for identical charges. Want a really huge part of family breakup and why young, black men have issues as a group? No wonder they don’t trust the institutions.
So what’s the result of all this? Well, glad you asked. Look at the maps here: https://www.opportunityatlas.org. It tells the story that both private and state actions have created a perverse kind of inequality in this country where it’s a lot harder for the average black kid to succeed in this country than the average white kid. When society continually beats a set of people down for, like, no reason it’s perfectly predictable that that set of people will have problems. One surely wouldn’t expect an abused kid to have the same outcomes as one from a perfectly healthy and safe family.
So what do we do about it? I’ve got a couple ideas. In a paper that hopefully will come out soon, I argue for universal school choice for every student in the country. If we believe primary and secondary education is positive right in this country (most states have constitutional provisions stating such) that it makes sense to allow people the freedom to exercise this right however they want, and allow for the marketplace to work to improve school quality, like we see in the charter sector. As these schools have grown and matured (and been allowed to innovate) their outcomes have gotten better. Now, it’s rare to find negative outcomes of charters, even with the strongest experimental designs.
Second, I believe strongly that we should repeal the entirety of the current welfare state and replace it with a universal basic income. The math works out to about even if we give everyone $800/month, deposited into a bank account, as soon as they graduate high school or turn 18, whichever is first. That’s enough to get above the poverty line for a couple and drastically increase bank usage.
Third, decouple employment and health care. Because of a quirk of WWII tax law, employer sponsored health benefits aren’t taxed like income, incentivizing employers to pay for health insurance instead of increasing salary. Replace it with expanded HSA’s or something similar. It would increase job mobility since your health insurance would be portable and not tied to your employment.
These three things have a common purpose: empower the individual to take control of their lives and live up to their potential unimpeded by the state. Other things like opportunity zones could help attract capital to the areas that need it.
Finally, everyone should read this book: https://www.amazon.com/Dignity-Seeking-Respect-Back-America/dp/0525534733/ref=sr_1_1?keywords=chris+arnade&qid=1551149213&s=gateway&sr=8-1. The author did the unthinkable: he went out and talked to people! Crazy thing to do these days. It really drives home the point that people in large swaths of the country feel neglected and hopeless, so they turn to drugs or other vices to escape.
Anyhoo, I’ve said too much. Have a lovely evening, everyone.