PALOS VERDES ESTATES (CBSLA) — A retired attorney and self-proclaimed historian is coming under fire again for raising a replica of the Confederate flag on his Palos Verdes Estates property, but he says he does it to honor the men who died during the Civil War and that the United States has not come to grips with its racist past.

Civil War expert Joe Ryan flies the Confederate flag of the Army of Northern Virginia, which has the names of the battles of Gettysburg, Sharpsburg, Cedar Mountain and Groveton in each quadrant, about four times a year, the Daily Breeze reported this week.

READ MORE: Fire Breaks Out In Restaurant Of Downtown LA High-Rise, One Hurt

However, Ryan, who talks in depth about the Civil War on his YouTube channel and website, says the display is not meant to be racist.

“They’re responding to different flags and signals of which side is theirs and which side’s not, but they’re all Americans dying on the same battlefield,” Ryan told CBS2 News Wednesday. “We’re not facing our racism, let me put it to you that way.”

He says the reasons for Civil War were complicated, but that the conflict was rooted in racism.

“We’re pretending like the problem is solved,” said Ryan, who has lived in PVE for more than 40 years. He said that in that time, he’s only received one complaint about the flag.

Still, current neighbors don’t feel this is the way to address those issues.

READ MORE: Suspect Shot, Wounded By Deputies During Standoff; Climbs Onto Roof Of Walnut Park Home

“The Confederate flag, to me, is representative of a painful and hateful time in our society, one that I do not want to look to the past for but I want to move past,” said Cindy Dunbar.

“It’s not the time to put up that flag,” said another woman who wished to remain anonymous.

Civil War expert Joe Ryan flies the Confederate flag of the Army of Northern Virginia about four times a year. (SOURCE: CBS2)

The Daily Breeze reported the PVE City Manager began looking into the matter after the topic was brought up on the Nextdoor website.

Ryan said he doesn’t understand what the city would be looking into, since he is exercising his First Amendment right to free speech in raising both the Confederate flag and the American flag, the latter flying in the former’s place on the Fourth of July.

MORE NEWS: Claremont Unified Cancels All Classes To Assess Windstorm Damage, Power Out At 2 Campuses

Ryan claimed he was already planning to retire the flag, but not because of any complaint; he said the flag is old and tattered.