It started with a warning, proceeded to a walkout and drew to a close with IRS officials walling off the agency from congressional interrogators whose patience appears to be wearing thin.

Democratic and Republican lawmakers were visibly frustrated Wednesday at the third congressional hearing on the IRS’ practice of singling out conservative groups. Members of Congress asked, again and again, how the program started and why nobody in the agency’s upper echelons notified Congress despite knowing about it at least a year ago.

One lawmaker, Rep. Stephen Lynch, D-Mass., warned at the top of the hearing that if witnesses did not start to open up, a special prosecutor would be appointed.