New York will enact major changes to its use of solitary confinement in prisons as part of a settlement with the New York Civil Liberties Union, the state announced Wednesday. The announcement from one of the nation’s largest prison systems caps the most successful year yet for solitary-reform advocates.

Under the agreement, about one-quarter of the state’s 4,000 prisoners in solitary confinement will be placed in less isolated housing. New York will also reduce the use of solitary for future inmates by limiting both the reasons they can be placed in it and the time they spend in it. Some of solitary confinement’s more troubling aspects will also be curtailed: Prison officials will no longer be allowed to use food as punishment, and pregnant inmates won’t be placed in solitary “except in exceptional circumstances.”

The agreement, which will needs approval from a federal judge before it goes into effect, was reached after two years of negotiations following a NYCLU lawsuit.

“Today marks the end of the era where incarcerated New Yorkers are simply thrown into the box to be forgotten under torturous conditions as a punishment of first resort,” Donna Lieberman, the NYCLU executive director, said in a statement announcing the settlement, “and we hope this historic agreement will provide a framework for ending the abuse of solitary confinement in New York State.”

New York’s powerful Correctional Officers and Police Benevolent Association expressed skepticism.

“While we have not had the opportunity to review the details of this settlement, our state’s disciplinary confinement policies have evolved over decades of experience, and it is simply wrong to unilaterally take the tools away from law enforcement officers who face dangerous situations on a daily basis,” Michael Powers, the union’s president, said in a statement.

New York first experimented with solitary confinement in 1821 in an attempt to replicate penal reforms in neighboring Pennsylvania. Quakers and other religiously minded prison advocates of the era supported keeping prisoners in prolonged isolation during their sentences. There, it was argued, criminals could mediate upon their crimes and become “penitent” before returning to society. They viewed this as a humane alternative to the deplorable conditions of American and European prisons at the time.

Intrigued by the idea, France’s restored Bourbon monarchy dispatched two writers, Alexis de Tocqueville and Gustave de Beaumont, to the United States in 1831 to study the penitentiary model’s potential for French prisons. De Tocqueville later wrote his celebrated book Democracy in America based on his travels for this project. When he and de Beaumont returned to France in 1832, their report endorsed some of the efforts, but also described New York’s failed experiment at Auburn Prison a decade earlier.

The northern wing having been nearly finished in 1821, eighty prisoners were placed there, and a separate cell was given to each. This trial, from which so happy a result had been anticipated, was fatal to the greater part of the convicts: in order to reform them, they had been submitted to complete isolation; but this absolute solitude, if nothing interrupt it, is beyond the strength of man; it destroys the criminal without intermission and without pity; it does not reform, it kills,

During their prolonged isolation, de Tocqueville and de Beaumont wrote, the prisoners “fell into a state of depression so manifest that their keepers were [also] struck with it.” Five of the inmates died; others went insane. Eventually, the governor of New York intervened and ended the experiment by pardoning 26 of the inmates. Prison officials allowed the rest to leave their cells during the day to toil alongside other prisoners in Auburn’s workhouse.

By the late 19th century, solitary confinement largely fell out of use. The Supreme Court first noted its disturbing impact on prisoners’ mental health in In re Medley in 1890 when it was imposed on death-row inmates in Colorado. After a long quiescence, prolonged isolation enjoyed a modern renaissance in the 1970s and 1980s as the nation turned towards mass incarceration. Precise statistics are unavailable, but estimates suggest that between 70,000 and 100,000 inmates are kept in prolonged isolation in the United States.

In June, Justice Anthony Kennedy wrote an unusual concurring opinion that all but urged the legal community to bring a case about long-term solitary confinement to the Supreme Court’s docket. Invoking Medley, Kennedy wrote that recent scholarship “still confirms what this Court suggested over a century ago: Years on end of near-total isolation exact a terrible price.” He described it even more bluntly during a congressional hearing three months earlier. Solitary confinement, Kennedy told legislators, “literally drives men mad.”

New York’s settlement follows a string of legal victories by other solitary-reform groups in the nation’s largest prison systems. In May, Illinois said it would sharply curtail the use of solitary confinement for juveniles as part of a settlement with the ACLU. California, which struggled with hunger strikes over its solitary policies in recent years, approved a broad slate of changes in September to settle a lawsuit brought by groups seeking changes in prisons.