Imagine a most upstanding human, righteous, kind and lawful. That human gets elected to hold some office. Should that person not worry about government spying because they would never find anything compromising about him?
Surveillance can be used for more than just discrediting imperfect people. Even meta info, much less whole texts of email, can be used to learn who is important to your target. Then pressure would be put on those important people to make the target — their father or spouse or friend — comply with some government dictate or another for fear of harm to those he loves. Typical mafia approach made even easier by NSA and their handlers.