Not sure he is worried maybe his advisors who might loose their jobs.
check it out:

As is always the case with anything involving former-Alaska Governor Sarah Palin, our unbiased, objective, not-at-all-liberal media couldn’t wait to mock her impeachment case against President Obama — but not everyone is laughing. According to reporting from The Hill, The White House fears exactly that should the president take executive action to grant amnesty to thousands of illegals.

Senior White House advisers are taking very seriously the possibility that Republicans in Congress will try to impeach President Obama, especially if he takes executive action to slow deportations.

Dan Pfeiffer, a senior adviser to Obama, said Friday that the White House is taking the prospect of impeachment in the GOP-controlled House more seriously than many others in Washington, who see it as unlikely.