Shortly after the resurgence of Black Lives Matter, I found myself connecting with friends and acquaintances who seemed prepared to admit that we’ve enjoyed privileges interlinked with race (and age), and that right about now would be a good time to start unhanding these privileges. For the most part, they’re people like me—white baby boomers, with a skew toward males who attended Catholic schools long ago. The feelings among them are genuine and might well reflect a moment of realization for many Americans, not just white liberals but also others.

Still, there are questions. What privileges are we talking about? And what exactly is it that we’d be giving up?

My friends and interlocutors are speaking of the now-familiar advantages. These include an assurance that I could approach authorities, including the police, and expect a fair hearing; that I could browse aimlessly in shops without clerks monitoring my every movement; that if I move to a new locale, the neighbors won’t eye me with suspicion; and many other privileges of membership in my race.

These are surely advantages (they’re also the more visible ones, catchable on video). But what would it mean for me to no longer have them? Recalcitrant cops aren’t going to start manhandling me just because they’ve decided to go easy on Blacks. My new neighbors wouldn’t look at me warily, by virtue of having lowered their guard against Black newcomers. Shopkeepers won’t start tailing me after they’ve turned their gaze off patrons of color.

In other words, I can lose all of these privileges, at no cost to myself. They’re easy to renounce (if not necessarily change at the social level). Could the same be said for some other advantages, harder to see and therefore acknowledge? I’m thinking mainly of the pecuniary benefits made possible by systemic racism, or what the Latin American liberation theologians call “structures of sin.” Here’s one little corner of a structure: how my whiteness affects my property taxes. It’s something I scarcely thought about until seeing a Washington Post article under the headline, “Black families pay significantly higher property taxes than white families, new analysis shows.”

I’d have thought Black homeowners pay less in property taxes than their white counterparts, relative to home prices. That’s because the value of their homes tends to appreciate more slowly, in neighborhoods often viewed as less desirable. So, local tax assessments ought to be lower. In fact, Black families pay more, adjusting for market value—13 percent more than white families, according to a new study of 118 million homes nationwide by Indiana University and University of Utah economists. Though the causes are varied, it’s reasonable to conclude that assessors haven’t been particularly worried about overtaxing Black families.

My takeaway: I’ve likely reaped a white-family discount on property taxes, and have done so through 31 years of homeownership, thanks to a heftier burden on Black families. Renouncing this privilege isn’t like giving up an exclusive claim to fair handling by police, which wouldn’t cost me a dime. If Black families pay less in property taxes, white families have to pay more (maintaining the tax base). The solution isn’t free of charge.

And does that even graze the surface of white, middle-class privileges?

According to numerous studies, typical middle-class Black households have barely one-tenth the wealth of their white counterparts. (The pandemic, which has done inordinate harm to Black and Latino workers, won’t make it easier to narrow the gap.) For most Americans, the chief asset is their house, which many Blacks don’t have. Indeed, researchers have traced much of the racial wealth divide to public policies that deliberately excluded Blacks from homeownership.

In his 2017 book The Color of Law, Richard Rothstein recounts a hair-raising history. During the post-World War II housing boom, developers who received federal loan guarantees were barred from selling homes to African Americans in places like Levittown, New York. In 1948, those modest single-family houses in Levittown went for around $8,000 each with no money down; today they can sell for up to a half-million dollars. That translates into substantial equity for generations of white families—and a lost bounty for generations of Blacks.

These are structures of sin that haunt us today, bolstering economic inequality and eating away at social solidarity. In light of the injustices, what sorts of public policies are called for? What transfers of wealth would begin to right these wrongs, and from whom should they primarily come? Rothstein lets his mind play over the possibility of the government buying homes in white suburbs—and reselling them to Blacks for the going price in 1950, adjusted for inflation (around $75,000 today for a typical Levittown home). This is offered as a thought experiment, not as a politically feasible measure.

Who’ll decide which privileges are undone and how that’ll happen? The scale of these questions is more than most of us bargained for, when we started counting our white privileges. And yet, where does a serious conversation about the inequities and remedies go, if not in that direction?