() President Donald Trump’s approval rating numbers are moving in the wrong direction as he approaches the 100-day mark of his administration on Tuesday, with nearly 53% of Americans disapproving of the president’s performance, according to Decision Desk HQ.

But polls can provide as much movement as the stock market, according to Larry Sabato, the director of the University of Virginia’s Center for Politics, who says the November election represents the true measure of voter approval.

“So, this is window dressing in a sense,” Sabato told .

The polls only matter when the White House begins to panic and decides that the numbers should matter, Sabato said.

“Generally, when people are alienated as a president or angry at him for some reason or another, everything tends to drop a little and sometimes, more than a little,” he added.

Sabato said that ultimately, Independents will decide the direction Trump’s popularity moves. However, the first marker of where the president sits with voters will come during the mid-term elections, particularly in the House of Representatives, Sabato said.

“The Senate is another animal entirely,” he said.