Read In Spanish Read In Portuguese If you owe enough in delinquent American federal taxes, the U.S. government can take your passport. The Internal Revenue Service certifies “seriously delinquent” tax debt to the U.S. State Department. This is an individual's unpaid, legally enforceable federal tax debt, including interest and penalties, that totals more than $55,000 (that amount How You Can Lose A Passport Due To Tax Debt