A federal judge expressed sympathy on Tuesday for thousands of federal employees who were suddenly fired by the Trump administration earlier this year, but he also voiced skepticism about whether reinstating them to their jobs was a proper remedy and questioned what the courts could ultimately do.

Judge J. Harvie Wilkinson made the comments during a hearing in the 4th U.S. Circuit Court of Appeals in Richmond, Virginia, as part of a lawsuit brought by 19 states and the District of Columbia. The suit alleges the states were harmed by the mass firings of probationary workers, because states did not have warnings required by law to help them prepare for the unemployed.

“Part of me thinks that the voters may be the ones to render a final verdict on this,” Wilkinson said. “There are limits to what the courts can do, but there are also gigantic political costs to throwing out so many employees, and in one sense the final verdict may be one for the voters to render. I’m not saying that’s the whole answer, but maybe it’s a part of it.”

While a federal judge in Baltimore issued an order in March to require the federal government to reinstate the employees, the federal appeals court halted it. The states are seeking further review as the lawsuit plays out, but Wilkinson said full reinstatement of the employees sounded like an overly broad remedy during Tuesday’s hearing.

Caroline Van Zile, the solicitor general of the District of Columbia, argued that the District of Columbia and states have been harmed due to the lack of 60 days’ notice and information about layoffs required in the law when the federal government conducts a reduction in force. The time and information was set by Congress to help states respond, she said.

“They have made us the ones responding to unemployment crises like this, not the federal government — the states,” Van Zile said. “We are the ones left holding the bags by statute, and that’s exactly why they gave us this right to information.”

But Sarah Welch, an attorney for the appellants, said information has been provided, and she also contended that the probationary employees in question were not dismissed due to what’s known as a reduction in force, which triggers the 60-day warning and information requirements that the states say they did not receive.

“These agencies did not conduct (reductions in force.) They haven’t eliminated positions, and there’s been not a shred of evidence that agencies through the terminations of probationary employees that are challenged here have eliminated a single position or changed their agency functions, which would be the calling cards of a RIF,” Welch said.

Probationary workers have been targeted for layoffs across the federal government because they’re usually new to the job and lack full civil service protection.

While expressing sympathy for the workers, Wilkinson said “there seems to be a disconnect between the harm alleged and the remedy imposed.”

“The remedy has to be tailored to the violation, and here the remedy is very broad, and much broader than the actual violation, and as just as a basic matter of equity, that’s a difficult proposition,” the judge said.

But Van Zile said she believed U.S. District Judge James Bredar got it right in his order, when he said that agencies can’t engage in far-reaching illegal activity “and then complain that the remedy is too burdensome.”

“They are the ones who terminated 24,000 employees in a single go,” Van Zile said. “That harm was wrought on the states. We had no agency, no way to avoid that harm.”

Bredar was one of two judges appointed by Democratic presidents who found that the Trump administration violated federal laws in carrying out the terminations at 20 agencies in the states that sued.

The Supreme Court blocked another order from U.S. District Judge William Alsup in San Francisco, finding that nonprofit groups lacked legal standing to sue over the firing of probationary workers.

During Tuesday’s hearing, Wilkinson also questioned whether reinstating the employees would give states too much influence over the composition of the federal workforce. While the judge said he has sympathy for how individuals have been affected, “there’s a real question here of what the federal remedy could be that wouldn’t really significantly damage the federal government’s control over the composition of its own workforce.”

The states suing the Trump administration in the Baltimore case are Arizona, California, Colorado, Connecticut, Delaware, Hawaii, Illinois, Maryland, Massachusetts, Michigan, Minnesota, Nevada, New Jersey, New Mexico, New York, Oregon, Rhode Island, Vermont and Wisconsin, along with Washington, D.C.