As passions rage nationwide over monuments to the Confederacy, it’s easy for New Yorkers to feel removed, as if this history and our commemoration of it don’t affect us.

They do.

After the recent deadly violence in Charlottesville, Virginia, where white supremacists opposed the removal of a statue of Confederate Gen. Robert E. Lee, monuments to the Confederacy in several places — including Annapolis, Maryland; Durham, North Carolina; and Austin, Texas — have been removed.

In Brooklyn, a smaller monument was removed on Aug. 16. It was a churchyard plaque at St. John’s Episcopal Church in Bay Ridge commemorating a tree Lee planted in the 1840s. The bishop of the Episcopal Diocese of Long Island told the Daily News that the violence in Charlottesville, and the removal of Confederate statues nationwide, inspired the move.

On the same day, Bronx Community College said it would remove busts commemorating Lee and Stonewall Jackson, also a Confederate general, from its Hall of Fame for Great Americans. Gov. Andrew Cuomo has asked the Army to change the names of two streets in honor of Lee and Jackson on the Fort Hamilton base in Brooklyn.

New York fought for the Union in the Civil War. But in Colonial times, as much as 20 percent of New Yorkers were enslaved, and the city’s financial institutions were involved in the Southern slave trade throughout the early 19th century.

We shouldn’t forget that history. New Yorkers need sites like Plymouth Church in Brooklyn Heights, a hotbed of abolitionist organizing, or lower Manhattan’s African Burial Ground National Monument, where many enslaved New Yorkers are buried.

But unlike these sites, the street names and the monuments honoring Lee and Jackson as Great Americans don’t merely document part of history. Rather, like similar monuments around the country, they glorify those who fought to defend the institution of slavery.

Organized white supremacy has been on the rise in the United States, according to the Southern Poverty Law Center, and has become particularly visible now. We must stop honoring it in public spaces.

Liza Featherstone lives and writes in Clinton Hill.