President Obama is pressing lawmakers to raise the federal debt ceiling and says he won’t be drawn into negotiations.

“When it comes to the debt ceiling, we’re not going back there,” Obama said during a press conference in the State Dining Room of the White House on Friday.

His comments come after Treasury Secretary Jack Lew set a Nov. 5 deadline on Thursday for raising the debt limit to avert a potential default on current loans. That gives Congress just a few weeks to hammer out an agreement on the current $18.1 trillion limit.

Obama acknowledged that Speaker John Boehner’s (R-Ohio) plans to leave Congress in October will complicate budget negotiations with congressional leaders, but said he believed there is a path to an agreement.

“I do think there is still a path for us to come up with a reasonable agreement that raises the spending caps above sequester to make sure that we can properly finance both our defense and non-defense needs, that maintains a prudent control of our deficits and that we can do that in short order,” Obama said.

“It’s not that complicated. The math is the math,” he added, saying he has spoken with Boehner, Senate Majority Leader Mitch McConnell (R-Ky.), Senate Minority Leader Harry Reid (D-Nev.) and House Minority Leader Nancy Pelosi (D-Calif.).

Congress on Wednesday passed a short-term bill to keep the government funded through Dec. 11.

Obama said Friday that “extraneous issues” should be kept out of budget talks, mentioning the current fight over funding Planned Parenthood.