The prospect that thousands of drones could be patrolling U.S. skies by the end of this decade is raising the specter of a Big Brother government that peers into backyards and bedrooms.

The worries began mostly on the political margins, but there are signs that ordinary people are starting to fret that unmanned aircraft could soon be circling overhead.

Jeff Landry, a freshman Republican congressman from Louisiana’s coastal bayou country, said constituents have stopped him while shopping at Walmart to talk about it.

“There is a distrust amongst the people who have come and discussed this issue with me about our government,” Landry said. “It’s raising an alarm with the American public.”