A taxpayer's problems with the Internal Revenue Service didn't end with the April 15 tax filing deadline thanks to scammers impersonating the IRS.

The IRS warns Americans against an old phone scam that has surprised the agency with its new sophistication and wider reach. The Treasury Inspector General for Taxpayer Administration described the scam as "the largest scam of its kind that we have ever seen."

The IRS said that over 20,000 persons have been contacted by scammers. Those who fell for the scam have together forked over more than $1 million to the thieves. Immigrants have been frequently targeted and threatened with arrest or deportation, said the IRS.

In this scam that first surfaced last year, someone claiming to be an IRS agent calls a person and tells him that he owes more taxes than he paid. The scammer then tells the victim to load his payment onto a prepaid card or to send the money via a wire transfer.

The IRS said these thieves have manipulated caller ID information to make it appear like it's the IRS calling. In order to make themselves appear legitimate, the thieves reveal all the personal information they have about the victim. They tell the victim the last four digits of his Social Security number and home address, among other personal information.

Thieves will resort to threats and accuse the victim of hiding income from the IRS, They'll also threaten the victim with arrest, seizure of his property and freezing of his bank accounts.

The IRS has been issuing warnings about this scam since last year but this hasn't prevented scammers from boosting their efforts.

"This has been going on for quite a while," said IRS spokeswoman Peggy Riley. "But now these callers will be calling people to say, 'We've refigured your tax bill and you owe us more'."

How do you know it's a scam? "The IRS doesn't normally correspond by email or call unless they sent a prior notice by mail," Riley said. "That's how you'll know."

Here's how the real IRS operates. The IRS usually contacts taxpayers about unpaid taxes by mail and not by phone or email. It won't ask for payment via wire transfer or prepaid card or ask you to pay up immediately or use threatening language.

If you've been scammed, file a complaint with the Treasury Inspector General for Taxpayer Administration (800-366-4484) and with the Federal Trade Commission at FTC.gov, using the words "IRS Telephone Scam" in your complaint.