Their names—at least the ones we know—were Lucy, Anarcha, and Betsey. There were other women, but their identities have been forgotten.

The man whose name appears in medical textbooks, whose likeness is memorialized in statues, is J. Marion Sims. Celebrated as the “father of modern gynecology,” Sims practiced the surgical techniques that made him famous on enslaved women: Lucy, Anarcha, Betsey, and the unknown others. He performed 30 surgeries on Anarcha alone, all without anesthesia, as it was not yet widespread. He also invented the modern speculum, and the Sims’s position for vaginal exams, both of which he first used on these women.

That Sims achieved all this has long won him acclaim; how he achieved all this—by experimenting on enslaved women—started being included in his story much more recently. And on Tuesday morning, in the face of growing controversy, New York City moved a statue honoring him out of Central Park.

The move came after decades of concerted effort by historians, scholars, and activists to reexamine Sims’s legacy. Medical professionals, especially gynecologists, have not always taken kindly to criticism from outsiders. Sims was one of their own. To implicate him, his defenders implied, is to implicate medicine in mid-19th century America.

The first serious challenge to Sims’s lionization came in a 1976 book by the historian G.J. Barker-Benfield titled The Horrors of the Half-Known. Barker-Benfield juxtaposed Sims’s “extremely active, adventurous policy of surgical interference with woman’s sexual organs” with his considerable ambition and self-interest. The man who once admitted “if there was anything I hated, it was investigating the organs of the female pelvis,” took to gynecology with a “monomania” once he realized it was his ticket to fame and fortune, writes Barker-Benfield.

In response, during the 1978 annual meeting of the American Gynecological Society, doctors took turns vigorously defending Sims against Barker-Benfield’s book. The most fervent of them was Lawrence I. Hester Jr., who said, “I rise not to reappraise J. Marion Sims, but to praise him.” He then announced that his institution, the Medical University of South Carolina, which Sims also attended, was raising $750,000 for an endowed chair named after J. Marion Sims.

Another doctor, Irwin Kaiser, in a more tempered defense asked the audience to consider how Sims ultimately helped the enslaved women he experimented upon. The surgery that he practiced on Lucy, Anarcha, Betsey, and the other enslaved women was to repair a vesicovaginal fistula—a devastating complication of prolonged labor. When a baby’s head presses for too long in the birth canal, tissue can die from lack of blood, forming a hole between the vagina and the bladder. The condition can be embarrassing, as women with it are unable to control urination. “Women with fistulas became social outcasts,” said Kaiser. “In the long run, they had reasons to be grateful that Sims had cured them of urinary leakage.” He concluded that Sims was “a product of his era.”

This did not quell criticisms, of course. Over the next few decades, scholars continued to criticize Sims’s practice of experimenting on enslaved women. The story became well-known enough to join a list of commonly cited examples—along with the Tuskegee experiments and Henrietta Lacks—of how the American medical system has exploited African Americans.

Medical textbooks, however, were slow to mention the controversy over Sims’s legacy. A 2011 study found that they continued to celebrate Sims’s achievement, often uncritically. “In contrast to the vigorous debate of Sims’s legacy in historical texts and even in the popular press, medical textbooks and journals have largely remained static in their portrayal of Sims as surgical innovator,” the authors wrote.

In recent years, one of the most prominent defenders of Sims’s legacy has been Lewis Wall, a surgeon and an anthropologist at Washington University in St. Louis. Wall has traveled to Africa to perform the vesicovaginal fistula surgery that Sims pioneered, and he has seen firsthand what a difference it makes in women’s lives. “Sims’s modern critics have discounted the enormous suffering experienced by fistula victims,” he wrote in a 2006 paper. “The evidence suggests that Sims’s original patients were willing participants in his surgical attempts to cure their affliction—a condition for which no other viable therapy existed at that time.” Wall also defended Sims on the charge that he refused to give anesthesia only to black patients. Anesthesia was not yet widespread in 1845, and physicians who trained without anesthesia sometimes preferred their patients to be awake.

There is debate over whether Sims’s specific surgical practices were unusually gruesome for his time. But his practice of operating on enslaved women was certainly not unusual. He wrote about it openly. It is this ordinariness that is noteworthy.

Sims was able to advance so quickly, argues Deirdre Cooper Owens, a historian at Queens College, City University of New York, in her book, Medical Bondage, because he had access to bodies—first enslaved women in the south, and later also poor Irish women when he moved to New York. “These institutions that existed in this country, which allowed easy access to enslaved women’s bodies [and] poor women’s bodies, allowed certain branches of professional medicine to advance and grow and to also become legitimate,” she says. The history of medicine has often been written as the history of great men. Owens wants to the turn the focus from the doctors hailed as heroes to the forgotten patients.

This first part—taking the focus away from Sims—is happening. In 2006, the University of Alabama at Birmingham removed a painting that depicted Sims as one of the “Medical Giants of Alabama.” In February, the Medical University of South Carolina quietly renamed the endowed chair honoring J. Marion Sims—the one announced by Hester after the publication of The Horrors of the Half-Known. The minutes of the board of trustees meeting where it happened did not even mention Sims’s name—just the new name of the endowed chair. “The decision was made in recognition of the controversial and polarizing nature of this historical figure despite his contributions to the medical field,” a MUSC spokesperson confirmed in an email to The Atlantic.

The J. Marion Sims statue that stood in Central Park is being relocated to Green-Wood Cemetery in Brooklyn, where Sims is buried. There, the New York Times reports, the statue will be demoted to a lower pedestal and displayed with a sign explaining the statue’s history. There may be an opportunity, now, to use the statue to tell the full story—to tell the stories of Lucy, Anarcha, Betsey, and the other enslaved women and their place in the history of medicine.