The Standard Story

In the standard account, American mass incarceration is a system of race-based social control. White elites constructed the carceral state in order to curry favor with ordinary white Americans who were worried by the changing character of the America around them. Yet there are at least three problems with this view.⁵

First, if American mass incarceration were a system of race-based social control, we should expect to see a rise in racial inequalities in punishment corresponding to the punitive turn (i.e., black incarceration rates should have increased substantially, and white rates much less or not at all). Yet white incarceration increased just as rapidly as black. Most of the growth in the ratio of black to white incarceration occurred in an earlier period of American history (1880–1970), after the end of slavery and during the first Great Migration.⁶ Since 1990, it has been declining.⁷

This figure shows trends in two ratios: (1) the ratio of black to white institutionalization rates; (2) the ratio of the institutionalization rates of high school dropouts to college graduates. We use the institutionalization rate rather than the incarceration rate to ensure consistency across IPUMS Census samples. This has the cost of including the institutionalized population as well as the incarcerated population. For this reason, we follow precedent and restrict our sample to nonimmigrant men aged eighteen to fifty (see Derek Neal and Armin Rick, “The Prison Boom and the Lack of Black Progress after Smith and Welch,” NBER Working Paper, July 2014). On the safe assumption that a very small proportion of this population is institutionalized in non-carceral facilities and that this proportion does not vary much over time and by race or education, these data can be used for this purpose.

This figure shows the percentage of men aged eighteen to fifty living in institutions between 1970 and 2017, disaggregated by education and race. Data are from IPUMS Census samples.

As Figure 1 shows, what has risen most dramatically over the last few decades is the disparity in incarceration between rich and poor. The incarceration rate among those with less than a high school education has skyrocketed, while the incarceration rate amongst college graduates (both black and white) has declined (see Figure 2).⁸ If white elites contrived mass incarceration to control newly enfranchised African Americans, why has the probability of a black college graduate going to prison halved over this period?⁹ In 2017, a white high school dropout was about fifteen times more likely to be in prison than a black college graduate.¹⁰

Second, to make the case that mass incarceration was a narrowly political project, the standard story has fixated on the War on Drugs. After all, the view concedes that black Americans have been arrested, charged, convicted, and sentenced for a crime. Proponents of this view argue that the criminalization of everyday drug use gave police, prosecutors, and judges the pretext to put blacks but not whites in prison.

By now, the problems with this argument have been widely documented.¹¹ Only a minority of American prisoners are incarcerated for drug crimes. At all levels of government — federal prisons, state prisons, and local jails — drug prisoners make up no more than one-fifth to one-fourth of inmates.¹² If one counts only the key victim of the standard story — the nonviolent, non-repeat user who has no ties to the drug trade — the figure is somewhere around 4 percent.¹³ A little less than half of inmates in prison or jail have been convicted or charged with various kinds of violent offenses (41 percent), another 17 percent with property crimes.

To prove that incarceration bears no relation to actual levels of crime, partisans of the standard story commonly claim that crime and punishment are uncorrelated at the national level. Between 1990 and 2008, they observe, the incarceration rate increased. It has since stabilized at very high levels. Over this same period, crime has declined precipitously.

This figure shows the crime rate over the period of its rise, between 1960 and 1995. These data come from the FBI Uniform Crime Reports, which compiles data on arrests from police agencies. Data for the homicide rate also come from mortality statistics (“Mortality”). As we discuss again later, these data show nearly identical trends over time, though the levels are always higher since some fraction of homicides never result in an arrest.

This figure shows the violent crime rate and the prison incarceration rate from 1960 to the present. These data show that the crime rate is not correlated to a measure of the stock of American prisoners, but that it is substantially correlated to a measure of the flows in and out of American prisons. Crime data are from the FBI Uniform Crime Reports and imprisonment data are from the Bureau of Justice Statistics.

To some, this is evidence that punishment must have nothing to do with crime. But this ignores the extraordinarily significant rise in crime that predated the punitive turn. From 1960 to 1990, as Figure 3 shows, the homicide rate doubled, the property crime rate trebled, and the violent crime rate roughly quintupled. Moreover, those who make this claim commit the mistake of comparing a stock (the total prison population in a given year) to a flow (the rate of crime per year). As Figure 4 shows, the violent crime rate is positively correlated with the flow of prisoners in and out of American prisons (i.e., the change in the incarceration rate).

To note all this is not to resurrect old arguments that American punishment is the necessary consequence of American crime. Defenders of the conventional view are right to emphasize that the state’s response was political. Most of this essay is devoted to substantiating this claim. But the rise in violence did detonate the punitive turn. Without the rise in crime and the ensuing public panic, the rise in incarceration would not have transpired.

Third, a deeper problem with the standard story is that its protagonists are a narrow cast of national, Republican elites driven by a single aim (to recapture the South from the Democrats). It shares this characteristic with the bespoke left-wing alternative, in which mass incarceration is a conspiracy not of white Republicans but of a wealthy elite seeking to “punish the poor.”¹⁴ The reality is that agency was diffuse. Mass incarceration unfolded in thousands of institutions across the country. These institutions were staffed by a diverse set of actors, all working under constraints set by the political economy of twentieth-century America and subject to an American electorate that was increasingly anxious about crime.

The standard story thus makes the common mistake of blaming a scandalous outcome on a cabal of scandalous actors. In the case of American incarceration, this is an especially egregious error. American criminal justice is distinguished by the degree to which members of local electorates have influence over criminal justice institutions and outcomes.¹⁵ In America, unlike other countries, state or local electorates vote for many of their prosecutors and judges; police officers are governed by elected mayors and sheriffs rather than unelected bureaucrats; and state legislatures make decisions that are elsewhere delegated to civil servants at the center.

In short, the standard story has led us astray. It has done so in three main ways. It mischaracterizes the population that languishes inside American prisons; it ignores the shaping role of violence on the politics of the punitive turn; and it overlooks the decentralized and atypically democratic character of American criminal justice institutions. It is ripe for replacement.

Crime

Advocates of the conventional view have suggested that the rise in crime was an invention of some combination of politicians, police, the media, and fearful white citizens. However, crime did rise dramatically in the 1960s. Figure 3 plots trends in homicide, property crimes, and violent crimes.¹⁶ It shows that between 1960 and the peak of the crime wave, the homicide rate roughly doubled, the property crime rate trebled, and the violent crime rate quintupled.¹⁷ There is also evidence that the increase in violence was concentrated in urban areas, with African Americans disproportionately likely to be both offenders and victims.¹⁸

The rise in crime was in some part the unsurprising result of demographic trends at mid-century. A “baby boom” had occurred in the aftermath of World War II, as couples who had put off having children during the war raced to start families during the prosperity of the postwar period.¹⁹ This led to more crime for two reasons: (1) most crime is committed by young men, so an increase in the share of young people in the population, all else being equal, should lead crime to increase; (2) a larger birth cohort may face more competition upon labor-market entry, stimulating conflict and demand for illicit forms of income generation.²⁰ In the US case, this demographic explanation seems to fit the shape of the crime wave, which began with a rise in “juvenile delinquency” in the late 1950s and ended in the “great crime decline” of the 1990s, just when the baby boomers were “aging out” of crime.²¹

But the baby boom cannot explain most of the crime rise. Age-adjusted crime rates show that crime rose considerably among all age groups.²² Why? Standard answers — a loss of political legitimacy,²³ or the rise of a “subculture of violence”²⁴ — raise more questions than they answer, not least because they are just as plausibly consequences of the crime wave. In point of fact, the rise in violence was incubated by the concentrated forms of deprivation that dotted America’s urban landscape by mid-century. These were the result of two peculiar features of American modernization: first, the unique character of its agrarian transition; and second, its distinctive fiscal and political geography, which inhibited cross-place redistribution.

Unlike other countries in the developed world, the United States experienced industrialization without large-scale rural-to-urban migration. Its labor force was not drawn from masses of peasants driven from their land, as in Britain. Instead, its nascent urban industries relied heavily on immigrant labor during the nineteenth century, while family farming continued to grow into the early twentieth century.²⁵ American industry only began to turn to its rural hinterlands for labor during World War I, and especially after European immigration controls came into effect in 1924. The cheapest homegrown source of labor was the African-American sharecropper in the South, whose living standards had been kept low by Jim Crow segregation and labor-repressive agriculture. The initial movement of rural blacks to cities in search of better-paying jobs contributed (along with the Agricultural Adjustment Act) to the collapse of the sharecropping system in the 1930s. This in turn led to a second and much larger wave of migration in the 1940s and 1950s. Around 40 percent of Southern-born blacks moved North in those decades, but the second great migration also had a counterpart within the South, as the African-American population of Southern cities also expanded rapidly.

The best available evidence suggests that this migration contributed to an increase in violent crime.²⁶ Claims that migrants brought with them “a subculture of violence” do not stand up to scrutiny.²⁷ But nor do accounts which indict a racist backlash from urban whites and their representatives.²⁸ The main culprit was structural rather than cultural or revanchist. As we will explain below, American labor and housing markets were in no state to absorb the new migrants. Those migrants had little or no wealth of their own due to the legacy of slavery, Jim Crow, and racial exclusion from education, jobs, and homeownership. Even if they had wanted to, city governments were in no position to address the resulting concentration of poverty and unemployment in predominantly black inner-city neighborhoods. Meanwhile basic social services were being undermined by the ongoing reallocation of people, jobs, and tax dollars to growing suburbs. It was primarily these factors that led to the explosion of urban crime rates.

The collapse of agricultural employment in the South was massive. In 1910, almost half of working-age black men in America were employed in the agricultural sector. In 1960, less than 8 percent were. Despite some decades of robust job growth, urban labor markets never replaced these lost jobs.²⁹ The problem only worsened as the flow of migrants increased, and urban economies began to change. Thus, while the first wave of migrants (during WWI and the 1920s) had largely been absorbed into industrial jobs, the second wave was invariably less likely to find work. Moreover, due to the segregated nature of urban labor markets, employment opportunities for the children of first-wave migrants were undermined by competition from the second wave.³⁰

Underlying the declining fortunes of rural migrants was a transformation in urban labor markets that was particularly consequential for unskilled men. In key areas like Detroit, deindustrialization began as early as the 1950s, as industry relocated first to the suburbs and then to the Sunbelt.³¹ The loss of key manufacturing jobs was exacerbated by automation and rising foreign competition. Figures 5 and 6 show the share (and change in the share) of the working-age male population living in central cities which was neither employed nor in school, disaggregated by skill level, race, and region. Those who dispute the idea that the rise in crime had economic causes commonly cite the fact that 1950 to 1970 was a time of general prosperity.³² And indeed, over this period, the unemployment rate was low, as was the percentage of the adult population without a job or not enrolled in school. But as these figures show, national prosperity masked severe and, soon, growing difficulties for unskilled and especially black men in central cities. Around a quarter of low-skilled black men between the ages of eighteen and fifty were neither in employment nor in school in 1960 and the number rose over the following decade.

This figure shows the proportion of men aged eighteen to fifty, living in central cities, who were neither in a job nor in school in the census year, disaggregated by skill level and region. “Lowest skill” refers to men whose educational attainment classifies them in the bottom quartile of the adult educational distribution in a given year; “highest skill” refers to men from the top quartile. These data are from IPUMS Census samples.

This figure shows the decade-on-decade change in the share of lowest-skilled men aged eighteen to fifty without a job and not in school, living in central cities, disaggregated by race and region of the country. Note that joblessness begins to increase slightly in the 1960s, and that this increase spans the South and not-South. Data are from ipums Census samples.

As the urban economy changed, the social prospects for those who remained in the cities plummeted further. William Julius Wilson provides the standard account of this transformation,³³ but, as our figures suggest, the story he tells begins earlier and is not limited to the Northeast and Midwest. The percentage of low-skill working-age men without a job began to increase rapidly after 1970, and it did so also in the South. While both white and black men were affected, trends amongst black Americans were categorically more severe, such that joblessness would soon become the norm for certain groups.³⁴ For many, cities went from being the place one moved to find a job to being the place one left to find a job.

Critically, not everyone was equally able to leave. By the 1960s, as is well known, white Americans began to flee the central city in droves. These decisions are typically attributed to their racist aversions to living alongside blacks. Such aversions were commonplace; embodied in restrictive covenants and a violent defense of the “color line.” But the growth of the suburbs in this period is arguably better understood as a case of capital flight, enabled by America’s peculiar fiscal geography. In the 1950s federal spending and subsidies redirected investment from cities to suburbs via a boom in highway and home construction.³⁵ Factories moved to the suburbs to take advantage of the new infrastructure, escape urban union strongholds, and benefit from lower taxes — and many skilled and white-collar workers followed them. Homeowners sought to take advantage of the federal subsidies, but they also moved to avoid the rising property taxes that were to fund citywide social programs won by progressive urban alliances. Thus white homeowners fled not only from areas into which blacks were making inroads, but also from neighborhoods that remained all white.³⁶ Importantly, many black homeowners also moved, taking advantage of recently won residential desegregation laws.³⁷ Thus cities became increasingly segregated and poor even as civil rights victories opened up new opportunities for the black middle class.³⁸

When these homeowners left the city, they took their tax dollars with them. The loss of revenue starved city-level social services, including education, public housing, and policing. The police, in particular, began to crack down under the strain, compensating for their inability to maintain order (as evinced by falling clearance rates) by exemplary acts of brutality.³⁹ The result was a vicious spiral: as cities hemorrhaged tax revenues, overcrowded schools lost funding, the housing stock deteriorated, and crime rose, the pressure to leave mounted. But the poor (disproportionately black) could not leave. They had no collateral and poor credit, and their access to the suburbs was further limited by zoning restrictions, minimum lot-sizes, and a deliberate lack of public transport.⁴⁰ They remained trapped in central cities that were being abandoned by both capital and the state, locked out of the consumption boom enjoyed by the rest of the country.

The result was a rise in violence that was historically unique in its speed and ferocity. Between 1960 and 1980 the US homicide rate had more than doubled to 10.7 per 100,000, the peak for the twentieth century (exceeding the previous peak of 9.7 per 100,000 in 1933). It remained at or around this level until the mid-1990s. These levels of violence were an order of magnitude more severe than anything observed in any other developed country. If crime rates had remained at their 1975–1984 level, the average American would have had an 83 percent chance of being the victim of a violent crime over the course of their lifetime.⁴¹

Moreover, the explosion of average victimization coincided with high and often rising inequalities in the distribution of violence. Violence rose in rural, urban, and suburban areas, but the rise was concentrated in central cities. Medium to large cities (200,000+) accounted for about half the growth of arrests in the 1960s, including 67 percent of the growth in homicide arrests and 72 percent of the growth in robbery arrests, despite making up only a third of the sample population.⁴² Rates of victimization and offending rose for both blacks and whites, but since racial disparities were already high in 1950, the explosion of violence in the 1960s led to unprecedented rates of violence in black neighborhoods.⁴³ By the early 1970s African Americans made up the majority of both victims and offenders in several categories of violent crime, and homicide had become the leading cause of death for young black men.⁴⁴

Rising urban un- and under-employment, especially for poor black men, together with a deterioration of education and social service provision, meant a reduction in legitimate forms of income generation. At the same time, opportunities for consumption and status attainment in the rest of society were rapidly increasing, leading to additional stigma and frustration for those stuck at the bottom. Finally, the strain on institutions of social control reduced the cost of crime by reducing the risk of getting caught. The net effect of these three changes was to increase the expected returns to illicit means of income generation. Crime began to pay more just as other sources of income dwindled for those who remained trapped at the bottom of deteriorating urban labor markets.

It is easy to see how this could lead to an increase in property crime, but why the rise in interpersonal violence? In part this is because illicit trades (e.g., drugs, gambling, prostitution, etc.) are regulated by violence. That said, at most, about half of the homicides in America’s largest cities are related to the illicit economy.⁴⁵ The other part of the story is sociological. The collapse of employment led to the collapse of communities, undermining the informal social controls that maintain order under ordinary circumstances.⁴⁶ Moreover, both of these changes most affected places in which policing had long been ineffective and brutal. Chronic mistrust and racialized neglect yielded low clearance rates.⁴⁷ This further incentivized violence, for people who believe they may be killed with impunity have a strong incentive to resort to preemptive violence.⁴⁸ It was the confluence of these circumstances that explains the rise in violence.

To summarize, American cities in the 1960s were characterized by the collision of two sets of facts, one stable and one changing. On top of an existing pattern of racial discrimination and the economic exclusion of African Americans came the transformation of the urban economy, the continued urbanization of Southern blacks, and middle-class flight. The result was the economic decline of the (central) city, particularly felt in historically black areas, while the rest of the country was prospering. Communities and social services were put under increasing strain, while law enforcement was unprepared for the consequences. The stage was set for an unprecedented rise of violence, which led to the highest homicide rates observed in any developed country in the twentieth century. How America (particularly the American state) would respond was as yet undetermined. We now turn to this response.

Punishment

Partisans of the standard story deem the rise in crime an invention of clever politicians. These politicians, the argument goes, used the language of “law and order” to transmute anxiety about the Civil Rights Movement into panic about a fictitious rise in criminal activity. But in the 1960s and 1970s there was nothing for politicians to invent. Crime rose, and it rose to particularly high levels in poor black neighborhoods.

The Public

We know that the public noticed the rise in crime, and responded to it by turning more punitive in its attitudes towards punishment. This point has been made most comprehensively by Peter Enns, who has gathered a large amount of public opinion data from different sources over this period.⁴⁹ Previous work on public opinion had studied idiosyncratic questions and often single snapshots in time, but Enns aggregates information from dozens of questions asked repeatedly over this period to estimate the public’s punitiveness. He finds that the punitiveness of the American public rose discernibly as crime rose, before falling in the late 1990s as crime, too, began to fall.

To be sure, the vast majority of the American public in this period was white. Thus, a rise in aggregate punitiveness is not obviously at odds with the standard story. Enns does show evidence from campaign documents at the time suggesting that politicians were reacting to, rather than fashioning, the public’s views, but one could still object that rising punitiveness might just have been a reaction to the Civil Rights Movement rather than a response to crime.

But, as we have argued elsewhere, there are at least two features of public opinion over this period that do not fit this view.⁵⁰

This figure shows trends in punitiveness by race, where punitiveness is defined as the probability that a respondent to a random question from a public opinion poll of the period answers that question punitively. These trends come from roughly 300,000 responses to thirty-nine different questions about crime and punishment, from almost 200 different public opinion surveys administered between 1955 and 2014. Data are from the Roper Center, the General Social Survey, and the American National Election Survey.

First, in our own analysis of data similar to Enns’s (Figure 7), we find that the rise (and fall) in punitiveness is characteristic of not just white but also black opinion. If the public’s punitiveness was nothing but a reaction to the gains of the Civil Rights Movement, it is odd that black Americans, who were the primary beneficiaries of these gains, should also turn punitive. The rise in crime, which hit black communities especially hard (because crime rose to much higher levels) is the more plausible explanation. This interpretation fits recent case studies of black communities in Harlem and Washington, DC, in which it is argued that public panic about rising, high crime rates came to dominate black politics in this same period.⁵¹

Second, if the standard story were right, over-time trends in white public opinion should mirror over-time trends in the strength of the Civil Rights Movement. As the movement peaked, so should have white anxiety (and thus punitiveness). But these trends do not coincide. Civil rights protests peaked in the late 1960s, declining soon after. The white public’s punitiveness, on the other hand, peaked in the mid-1990s, roughly twenty-five years after the peak of the Civil Rights Movement, not long after the height of America’s postwar crime wave. This is another reason to believe that crime, and not the conflict over civil rights, drove the public’s attitudes towards punishment.⁵²

Politicians

Observers of the politics of punishment in this period have noted that law-and-order concerns became commonplace among politicians — most prominently among Republicans, but among Democrats too. Historians and social scientists have argued that an emergent law-and-order coalition was cobbled together by politicians with disparate constituencies.⁵³

Together, these authors make two major arguments about this period. First, they argue that its protagonists were federal politicians, who engineered the public’s punitive turn. In these accounts, it is political entrepreneurs like Wallace, Goldwater, Reagan, and Nixon who catalyzed the racial anxieties of white Americans into demands for punishment. Second, they suggest that where conservatives led, liberals quickly followed. This view is particularly pronounced in two recent books about the period.⁵⁴ These authors argue that the liberal agenda was, in the end, not all that different from the conservative one. Both liberals and conservatives endorsed, if only implicitly, long-standing and racist theories about why crime was rising and why it was especially high amongst black Americans. And in response, liberals, like conservatives, clamored only to expand the punitive arms of the state.

On both points, this work overreaches. First, this account gets the causal sequence of the 1960s backwards. The public panicked not because political entrepreneurs emerged, but because crime rose precipitously. This panic defined the context in which all politicians of this period were operating. Talk of law and order became not just viable, but compelling. And it was in this context that the entrepreneurs of the period emerged. As Michael Flamm argues in his history of this period, it was precisely because the American public was growing fearful of crime that the conservative case against liberalism met with such success.⁵⁵

None of this is to dismiss the role of racism in fashioning a new punitive common sense. In selling “get tough” politics to white Americans, politicians profited from racist tropes about black Americans. But note two qualifications. First, as Flamm argues, racism was potent precisely because crime was rising and, especially, because black Americans were disproportionately represented amongst offenders. The racial overtones of political rhetoric succeeded because the white public was panicking about black crime. The public was not panicking about black crime because of the racial overtones of political rhetoric. Of course, conservatives pandered, sometimes explicitly, often implicitly, to cultural, moralizing, and racially coded interpretations of these disparities. They rejected the structural interpretations for rising crime and black-white disparities advanced by most liberals of the period. But they did not invent these disparities, or, indeed, invent public attention to them. Second, as Forman and Fortner have both shown, “get tough” politics became political common sense in black communities as well. It is not clear, in other words, that an America shorn of anti-black animus would have been an America without any brand of law-and-order politics. We will have more to say about the role of racism in American punishment at the end of this essay.

Second, while liberals could not avoid responding to people’s fears about crime, they initially responded very differently than did conservatives. In the relevant documents of the Johnson administration — the final reports of the Kerner and Katzenbach Commissions, for instance — the liberal view is plain. At root, the enemy is not poorly socialized teenagers, collapsing families, or the pathological choices of the urban poor. The root causes of crime, according to the leading liberals of this period, lay in limited labor-market opportunities for unskilled, young, and especially black men, a predicament made worse by the concentration of these young men in collapsing cities with underfunded public programs and an overstretched, under-resourced, and often abusive law enforcement apparatus.

It is thus no accident that these same documents called on liberals to conceive of the war on crime as a war on these root causes. The report of the Kerner Commission ends with four recommendations to fix urban disorder: expand welfare, expand housing, transform education, and create jobs. The Katzenbach Commission demanded that law enforcement be professionalized, centralized, and aggressively funded, while reminding its readers that the ultimate causes of crime lay in structural inequality. And in arguing this, liberals were right: because the rise in crime was a symptom of the failures of American modernization, its remedy lay in an aggressive expansion of the social-democratic state.

What Success Required

Yet, as this new scholarship on the carceral state emphasizes, liberals did fail. Crime rose inexorably in the 1960s, seemingly impervious to a variety of liberal initiatives, and despite almost continuous attention to the issue by the Johnson administration.

To understand liberal failure, one has to first appreciate what success would have required. Consider liberals’ choices. On the one hand, they had recourse to the state’s punitive arms (police, prisons, and the courts). Both conservatives and liberals agreed that these policies mattered. On the liberal view, however, the crime rate was additionally (and primarily) governed by a second set ofsocial policies: welfare, unemployment, housing, education, and health care. When politicians in the 1960s tried to wage war on the root causes of crime, it was to these tools that they turned.

In the abstract, anti-crime agendas can be usefully classified into the four quadrants that these two dimensions delimit: harsh or hands-off penal policy, paired with expansive or stingy social policy.⁵⁶ The conservative position in the 1960s was that the United States needed less social policy (in fact, conservatives attributed crime at least partly to the paternalism of the welfare state) and more punitive penal policy. The liberal argument was that crime required a dramatic expansion of social policy and a modernization of penal policy. What is not often appreciated about the liberal policy agenda is that, to succeed, it required an unprecedented redistributive effort.

This is a critical point, so it bears detailing. In either the punitive or social dimensions, expansion or contraction is generally a matter of dollars spent. This is obviously true of social policy, which mostly consists of redistributing resources, whether in kind or in cash, from rich to poor. But it is also characteristic of penal policy. While there are ways to make police, prisons, and the courts more punitive without spending more money on them (e.g., by cutting programs for prisoners), in general harsher policing, expanded imprisonment, and more efficient courts require more police, more prisons, more judges, more prosecutors, and so on.

Yet the costs of relatively generous social policy will always far exceed the costs of relatively harsh penal policy. The reason for this is simple: penal policy is hyper-targeted. Police arrest only that small fraction of the public that commits arrestable offenses; prosecutors charge that smaller fraction that commits offenses deemed worthy of being charged; and prisons harbor that even smaller fraction of the public that is sentenced to serve time. Moreover, contact with the criminal justice system is typically occasional. In contrast, social policy is indiscriminate in both dimensions. To be politically feasible, it must often be universal. Even at its most targeted, all poor people are eligible. And when they are eligible, they are usually eligible for significantly larger fractions of their life: time below the poverty line, while unemployed, if disabled, during childhood, or after retirement.

One will often hear criminal justice reformers argue that it costs $40,000 to incarcerate someone, but only $10,000 to educate a child.⁵⁷ The inference is that penal policy is actually more expensive than social policy, and thus, that the American state’s decision to fight crime with prisons and police has nothing to do with its aversion to redistribution. But while the statistic is correct, the inference does not follow. This is because the denominators are not equivalent. Penal spending is hyper-targeted, because the penal system makes less and briefer contact with the population than does the social arm of the state. And thus, it is much cheaper to build a harsh penal apparatus than to build a generous welfare state.

Consider some numbers. The United States combines the harshest penal state in the advanced world with its stingiest welfare state.⁵⁸ In the service of mass incarceration, it spends roughly $250 billion a year on prisons, police, and the courts, at all levels of government. This is considerably more than any other state in world history. Yet it also spends upwards of $3 trillion on social policy. Even if we count only that fraction of social policy which is spent on the poor (i.e., roughly that fraction which could strictly be tallied as part of the state’s war on the root causes of crime), the figure is at least $1 trillion.⁵⁹ To wit, the US government spends at least four and perhaps as much as twelve times more on programs that fight the root causes of crime than on repressing its symptoms.

This figure shows the ratio of social to punitive spending as a percentage of GDP in a sample of developed countries. Data are from the OECD. The classification of social and punitive spending thus corresponds to their definitions.

The point is not at all that the US welfare state is generous. It is well-known that it is not.⁶⁰ Rather, the point is that even underdeveloped social policy costs more than overdeveloped penal policy. In all other advanced capitalist countries, the ratio of social to penal spending is much higher. As Figure 8 shows, on average, governments in developed countries spend about twenty-two times more fighting the root causes of crime than they do on police, prisons, and the courts. By the OECD’s numbers, the ratio reaches almost forty in Denmark; the second-lowest (after the United States) is around thirteen (in Switzerland). The point is that waging an all-out war on the root causes of crime is equivalent to the task of building a large, redistributive welfare state that takes from the rich to give to the poor.

The problem in the 1960s was not even that liberals made no effort in this direction. In fact, liberals were not just verbally committed to policies that Hinton and Murakawa argue they disparaged. They were also committed to these policies in deed. In the 1960s, federal expenditures on social programs grew far more than federal expenditures on police, prisons, and the courts. In 1962, the Kennedy administration’s first full year in government, the federal government spent about $13.81 (or 0.37 percent of total spending) per person on punitive programs, and about $837.05 (or 22.5 percent) on social programs (all in 2016 dollars). In 1968, in Johnson’s final full year, the federal government spent $17.19 (or 0.36 percent) and $1,367.71 (or 28.8 percent), respectively. In real terms, this amounts to an increase of 25 percent in per capita punitive spending and 63 percent in per capita social spending. And because total spending was increasing significantly over this period, this small increase in per capita punitive spending could equivalently be represented as a decline in the percentage of total spending that went to punitive ends of about 2.6 percent (while even in these terms, social spending increased by 28 percent).⁶¹

These numbers should not be surprising. Many of the major advances in the American welfare state were products of this period (e.g., Medicare and Medicaid, a more generous Social Security program, increased federal aid to public education). It is true that the Johnson administration modernized and expanded the state’s punitive arms, but in general this was compatible with a genuinely liberal view of the source of social disorder. As the report of the Katzenbach and Kerner Commissions argued, some of the problems of urban crime could be attributed to the fact that existing law enforcement agencies were unprofessional, underpaid, and ignored black victims. In fact, this last complaint was made vociferously by Martin Luther King Jr in a 1965 piece on the Watts riots, writing: “The most grievous charge against municipal police is not brutality, though it exists. Permissive crime in ghettos is the nightmare of the slum family … Because no one, including the police, cares particularly about ghetto crime, it pervades every area of life.”⁶² Liberals’ desire to build a law enforcement apparatus free of these flaws is worth distinguishing from the punitive commonsense that would soon colonize American politics.

Thus, liberals did not fail to imagine what ought to have been done. Nor did they fail to attempt to do what ought to have been done. So why, exactly, did they fail? At root the issue is not one of attitudes or motivation, but capacity. The ultimate causes of liberal failure lie outside the state, in the incapacity of the American poor to compel redistribution from the rich. As we argue below, this incapacity was in part conjunctural. The social movements of the 1960s reshaped America, but they sought redistribution from a Johnson administration wedded to imperialist misadventure and to a Keynesian compact that pegged social spending to investor confidence. Yet much more important, we argue, were long-standing incapacities. By the 1930s, America was already well-established as a welfare laggard. The rise in crime that began in the 1960s was the bitter fruit of decades of a failed policy response to the problems of American modernization. Ultimately, the explanation of this enduring failure lies in the enduring constraints on social policy in the United States. And it is here, in these major and abiding limits to redistribution, that lies the key to understanding American mass incarceration.

Guns Or Butter

In the 1960s, left-wing elements inside and outside the Democratic Party were demanding a massive expansion of the welfare state. This was what the crises of the 1960s required, they argued. This is clearest in the exemplary ambitions of the Freedom Budget, which made a federally funded and federally administered jobs program the centerpiece of its policy agenda.⁶³ But this expansion was not forthcoming, for two kinds of reasons.

First, as the early 1960s turned to the late 1960s, the clout of the two constituencies demanding this expansive social policy, the labor and civil rights movements, were flagging. The labor movement was groaning under the weight of bureaucratization, after having been kneecapped by the McCarthyite attacks of the previous decades. And the Civil Rights Movement never found a way to move, in Bayard Rustin’s pithy mandate, from protest to politics.

Their weakness was exacerbated by the structure of the Democratic Party, which was never a social-democratic party on the European model, but a coalition of conservative Southern Democrats and Northern liberals. In the mid-1960s, thanks to the social movements that were bubbling around it, it had mustered something like a social-democratic agenda. But these movements never had more than a tenuous hold on the party establishment itself, which limited severely what they could win.

Second, the Johnson administration worried that redistribution from rich to poor would spook investors. During the boom-time economic growth of the 1950s and 1960s, the Kennedy and Johnson administrations had managed to expand government spending without burdening taxpayers — what Doris Goodwin calls “reactionary Keynesianism.”⁶⁴ This compact fell apart as the economy began to sputter. In the verdict of Bruce Schulman, Johnson’s administration had financed “simultaneous wars against communism and poverty … through a dangerous fiscal sleight of hand.”⁶⁵ It was only in 1967, several years into both wars, that Johnson finally did ask Congress for tax increases. And when he did, these were mostly to finance the war in Vietnam, in exchange for what Schulman calls “savage cuts in Great Society spending.” Social spending was profoundly limited by the demands of the war. Imperialism abroad killed reform at home.

And once the liberal moment of the mid-1960s had passed, any significant expansion of America’s social-democratic state was significantly less likely.⁶⁶ The labor and civil rights movements declined further. Republican administrations would not even try to fight the root causes of crime, and Democrats’ efforts to do so were ever more weak-kneed. The partisan divide on crime thus slowly closed, a trend most visible after the Dukakis debacle and under Bill Clinton in the 1990s. The American welfare state would never grow to do what liberals had hoped but failed to do in the mid-1960s.

A Tale Of Two Exceptions

However, the incapacities of the American state were not mainly the result of conjunctural facts about the 1960s. It is tempting to regard this decade as a missed moment, when the federal government failed to avert mass incarceration. This overstates the case. Less people would be languishing in American prisons had the Left won the battles it lost, but the struggles of the 1960s were not decisive as much as they were illustrative.

This figure shows the share of GDP devoted to social transfers in a sample of developed countries, over the course of their development. Data (and definitions) are from Lindert, Growing Public

By 1960, America was already well-established as a welfare laggard. As Figure 9 shows, the gap between the United States and the rest of the advanced world dates to the first few decades of the twentieth century. This was a period of massive working-class agitation in Europe with no real parallel in the United States. The United States was indistinguishable from other countries in the extent of public social transfers in 1890. In a world with very little redistribution, it spent about 1.3 times what the median developed country did on social transfers (as a percentage of GDP). By 1930, it spent half. This ratio would change very little over the next few decades.⁶⁷

As we have argued in this essay, modernization in America had yielded some unique social problems — most notably, the challenge of integrating the Southern, black peasantry into cities that had already passed through an industrial boom, populated by native and immigrant whites. High and rising rates of violence in American cities were a symptom of this problem, exacerbated by the postwar baby boom. These social problems demanded social policy remedies. What should concern us, analytically, is not the specific failure of 1960s liberals to take this path, but the long-standing failure of prior and successive American administrations to do so. It is the long-run underdevelopment of social policy over the twentieth century that yielded the high violence and harsh punishment that characterizes the United States today.

In short, we are arguing that American exceptionalism in violence and punishment is a symptom of America’s exceptional history. America has so many prisoners because its development path yielded some unique social problems, while its political economy prohibits redistribution from rich to poor on the European model. In a sentence, the story of American mass incarceration is the story of the underdevelopment of American social democracy.

The Origins Of Mass Incarceration

Of course, America’s carceral state was not constructed during the Johnson years. When Nixon took office in 1968, the incarceration rate was only 102 per 100,000. The seven-fold increase in the rate of incarceration happened subsequently, over several decades. So there is still something left to explain. How does the underdevelopment of social policy explain the metastasis of the American carceral state?

First, one must recognize that this state has not been built by the federal government. It is not the result of any one decision taken by a president or Congress. Federal actors may have made some difference on the margins, since they redirected funding, organized research and development, and fought a disproportionate share of the War on Drugs. But most of the federal bills are symptoms rather than drivers of the nationwide punitive turn. States and localities house 88 percent of America’s prisoners, employ about 81 percent of American police officers, and spend 79 percent of total money spent on police, prisons, and the courts.⁶⁸ Mass incarceration is better understood as the sum of all actions taken at these levels, by a cast of Republican and Democratic state legislatures, governors, district attorneys, police officers, and judges.

The failure at the federal level thus matters not because the federal government was the proximate agent of mass incarceration. It was not: neither under Johnson nor under later administrations. Rather, it matters because the persistent failure of the federal government to attack the root causes of crime left the task of managing the rise in crime to state and local governments. In this climate of high anxiety about crime, state and local legislators, mayors, city officials, prosecutors, and sheriffs made careers out of responding to a panicking public.

Of course, one might wonder why local and state governments all responded in punitive ways. Could not some of these governments have launched the affirmative, social policy response that the federal government could not muster?

One of the reasons for this is simply institutional. In the division of labor that characterizes American federalism, police, prisons, and the courts are mostly the responsibility of the states and municipalities, while most of the major social programs in American history have been invented and funded at the federal level. When local and state officials were bombarded by panicking electorates, it is no surprise that it was mainly to these tools that they would turn.

However, this is not the whole story. After all, some states and municipalities do attempt to craft their own social policies. They can raise taxes and spend in redistributive ways. Thus, another answer is that they were subject to the same constraint that bound the federal government: the absence of a constituency that could force the rich to give to the poor.

But consider, in addition, two other facts that make redistributive policy difficult for states and localities. First, the rich live in certain areas but not in others. Thus, local officials in poor areas cannot raise the kind of revenue that the federal government can. Even if the mayor of Ferguson had the gall to tax and redistribute to fight the root causes of crime in his area, he could never tax San Francisco’s billionaires. The perverse consequence of American federalism is that it is those areas in which violence concentrates that have the least resources to fight it at its root. Second, as was evident in the 1960s, the local state is especially vulnerable to the flight of its tax base. The costs of fleeing the federal state are much higher than the costs of fleeing local taxes, since the rich only have to jump across jurisdictional boundaries (e.g., move to the suburbs). This, too, condemns localities to cheap and thus punitive solutions.

Racial Inequality

Our argument thus far has explained why incarceration grew and also why America is exceptionally punitive, but we have yet to say very much about inequalities in exposure to police and prisons. Why are certain groups of Americans — and in particular, black Americans — so much more likely to fall foul of America’s carceral state? Racial disparities have declined slightly in the last two decades, but even over this period the black-white ratio has never fallen below five. There are few more important questions to pose about American punishment than this one.

One common answer is that these disparities are explained by the biases of police officers, prosecutors, juries, judges, and politicians. This amounts to the claim that, conditional on having committed an offense, black defendants are more likely to be arrested, more likely to be charged, more likely to be convicted, more likely to receive longer sentences. There is certainly evidence that each of these disparities exist.

Yet, what is relevant is not just whether they exist but how much they can explain. For instance, evidence from federal courts suggests that judges sentence black defendants to sentences that are 10 percent longer than otherwise-equivalent white defendants.⁶⁹ One can take this as a rough index of judges’ biases, explicit and implicit. But given that the overall disparity in the stock of prisoners is about 500 percent, this would be equivalent to noting that the biases of judges explain only about 2 percent of the total racial disparity in incarceration (10/500 = 2 percent). In fact, our best evidence suggests that biases at every stage, from arrest to sentencing, explain much less of the total racial inequality in punishment than is commonly assumed.⁷⁰ One estimate, which compares baseline estimates of offending to disparities in incarceration rates, finds that around 70–75 percent of the black-white disparity in incarceration is explained by the fact that black Americans are more likely to commit criminal offenses.⁷¹

Here, of course, it is natural to worry that we have no reliable measures of disparities in offending. After all, police reports and arrest records may be prone to racial bias either because police are individually prejudiced (and thus more likely to arrest African Americans) or because police departments focus their activities in black neighborhoods (and are thus more likely to come into contact with African Americans). But concerns about racial bias in police data can be mitigated by relying on multiple and independent sources. We know from court records and witness reports, for instance, that the vast majority (roughly 90 percent) of homicides are intra-racial. Arrest records of suspects can thus be reasonably checked against racial disparities (in victimization) derived from coroner’s reports. In 1970, for instance, coroners indicate that African Americans were nine times more likely to be murdered as whites, while they were eleven times more likely to be arrested for murder in the same year. By 1980 the ratio had fallen to 5.7 for victimization and 5.9 for arrests.⁷²

Furthermore, if cross-sectional disparities in offending between blacks and whites are consistently very large, they are also historically specific. Racial disparities before the twentieth century ran in the other direction. White Americans killed each other at higher rates before 1900.⁷³ Thus, any explanation of racial disparities in violence must account for their twentieth century provenance, which is but one reason that biological or other racist explanations of these disparities are a nonstarter.

So where do these disparities come from? The arguments of the previous sections furnish an answer. Behind racial disparities in offending lies long-standing inequality in life circumstances. African Americans are overrepresented in crime because they are more likely to live in America’s worst neighborhoods, at the bottom of its stretched class structure, with few opportunities to escape, and few public resources available for their self-development or safety.

Critics of mass incarceration often ignore crime because they worry that acknowledging it would be to blame mass incarceration on individual choices.⁷⁴ But this does not follow. To blame individuals, one must make the additional, nonobvious argument that they are responsible for the antecedent causes of their crime. In a society in which the vast majority of criminal offenders are drawn from the bottom of its class structure and trapped in its worst neighborhoods, this is a risible proposition. Crime is an index of oppression. Blame thus misses the point. It is altogether unfortunate that those who are alive to this oppression would deny its consequences. The Left is not wrong to denounce accounts which blame criminals for crime (or African Americans for their disproportionate representation in the ranks of offenders). Yet nothing in this denunciation requires us to ignore the reality of crime and violence.

This neglect of crime by critics of mass incarceration has costly political consequences. In the absence of serious critical commentary on the issue, conservative common sense has thrived. Most importantly, by leading progressives to misdiagnose the source of racial disparities in punishment, it makes it impossible to wage effective war against them. Racial disparities are mostly not a result of the injustice of biased treatment inside the criminal justice system, but rather the foundational injustice of American racial inequality outside it. The remedy must be equivalently foundational: not merely the retraining of police, prosecutors, and judges, but a redistributive attack on the roots of inequality by race and place.

One might argue that racism matters because it has blocked precisely this magnitude of redistribution. Forman suggests this in his recent book on Washington, DC, in which black elected officials failed to win their “all-of-the-above” policy agenda (both social policy and punitive policy) because of the racism of the white establishment.⁷⁵ And in fact a host of work notes the unpopularity of American social programs when they are perceived to benefit black Americans.⁷⁶

But this revision does not take the barriers to social policy seriously enough. As we have argued, nonpunitive remedies to American violence demand massive, unprecedented redistribution from rich to poor. In Washington, DC and like cases, what is relevant is not the prejudice of the rich and the white but the powerlessness of the poor and the black. As Stokely Carmichael once quipped, “if a white man wants to lynch me, that’s his problem. If he’s got the power to lynch me, that’s my problem.” What African Americans specifically (and poor Americans, more generally) lacked, as the 1960s turned to the 1970s, was leverage over American elites. As the economy sputtered, cities hemorrhaged revenue, states reorganized around powerful suburban constituencies, and the labor and civil rights movements collapsed, the prospects for American social democracy grew ever fainter. In a world in which, somehow, black Americans had acquired new, sufficient leverage, no amount of white prejudice could have stood in their way. Tragically, the same forces that decimated black communities and yielded the rise in violence (deindustrialization, flight of white Americans and many middle-class blacks from city centers) also sapped poor, working-class black Americans of most of their economic and political power.

Given this, how should we think about the causal relevance of race to American mass incarceration? In our view, it matters mostly in ways both less direct and more basic. Race is relevant because it is our best explanation for the absence of a working-class movement on the American continent, and thus the persistent underdevelopment of American social policy.⁷⁷ American slavery and then Jim Crow delayed the proletarianization of African Americans, with the result that they arrived in Northern cities after the first wave of American industrialization, in urban environments in which pivotal, scarce resources (jobs and housing) were hoarded by the first and second generations of established white ethnics. This was an environment destined to yield working-class disunity. Black Americans strove to penetrate well-protected labor and housing markets. It was no surprise that established incumbents would craft caste-based remedies to exclude them. Such strategies were rational, even if suboptimal in the long run.

In this sense, partisans of the standard story are not wrong to link American mass incarceration to American slavery. But they are connected not because slavery established some transhistorical imperative that America be always a land of white domination. Rather, they are connected because the plantation economy tied African-American labor to the land until 1940. Blacks were thus bypassed by America’s industrial boom. They are connected also because slavery was largely responsible for an American federalism which assigns law enforcement to those parts of the state least capable of paying the higher costs of redistributive remedies.⁷⁸ In the final reckoning, the story of the twin exceptions that have been the subject of this essay starts with this history.

Looking Forward

What is to be done about American crime and punishment today? Remedies run in two domains: criminal justice reform and reforms to social policy. Most of the contemporary discussion has focused on the first. Those on the Left (but also some on the Right) have argued that the criminal justice system has become too overbearing, too harsh, too intrusive, and too punitive. This is undoubtedly true. The American carceral state is degrading and inhumane. Reforms to make it less so — humane conditions of confinement, shorter sentences, easier parole, decriminalization of key offenses, less aggressive policing — are urgently needed.

But there are two important points that the current reform conversation often elides. First, most of the people inside American prisons have committed serious offenses. Overbearing punitive intervention in their lives is an injustice, but so too would be hands-off neglect. A judicious treatment of their problems will require a radical rethinking of the very purpose of the penal system. Sadly, the views of the American left on punishment are often more American than left-wing. Leftists, too, talk of victims’ rights, and demand retribution from guilty offenders. These impulses are understandable, but they are also draconian. Those who commit violent offenses are themselves victims of facts beyond their control, whether of inheritance or circumstance. The appropriate response is not to punish or to shame, but to repair and rehabilitate while safeguarding the public.⁷⁹ In this process, victims have no position of privilege; they deserve care and compensation for their trauma and loss, but society’s debts to them should be paid separately. The specifics might be arguable, but the overall point is that reformers, left and right, have washed their hands of this problem by centering reform efforts on the easy cases (e.g. the nonviolent drug offender). This can actually amplify reigning intuitions about punishment in harder cases, which is where reform efforts will have to reach if they are to be effective.⁸⁰

Second, and more importantly, if we are right that the overdevelopment of the American penal state is a symptom of the underdevelopment of the American social policy, meaningful reform is in large part the task of winning redistribution from ruling elites . It will be costly. And there will thus be losers, who will resist it. The end of American mass incarceration is not a technical problem for which there are smart, straightforward, but just not-yet-realized solutions. Rather, it is a political problem, the solution of which will require confronting the entrenched power of the wealthy. In this sense, the task before us is to build the capacities of poor and working-class Americans to win redress from their exploiters.