Article Lead Image

Gage Skidmore/Flickr (CC BY SA 2.0) | Remix by Jason Reed

The case for abolishing the IRS

Inside the radical plan to remake American government.


Aaron Sankin


Posted on Apr 15, 2015   Updated on May 29, 2021, 1:57 am CDT

When Sen. Ted Cruz (R-Texas) stepped onstage at the Conservative Political Action Conference in Oxon Hill, Md., last year, he knew precisely what the audience wanted to hear.

Asked by Fox News talking head Sean Hannity about the top theoretical agenda items for a theoretical Ted Cruz presidency, the senator replied, “Number one, repeal every blasted word of Obamacare.”

The audience erupted in cheers. One does not become a right-wing hero, rising from a freshman senator to a serious presidential candidate in a single term, without being able to feed red meat to the hungry base as effortlessly as breathing.

“Number two,” Cruz continued, “abolish the IRS; take all 125,000 IRS agents, and put them on our southern border.”

This time, a mixture of cheers and chortles. The image is clearly absurd. A bunch of pencil-necked Internal Revenue Service bureaucrats in ill-fitting suits patrolling the U.S.-Mexico border, the province of trigger-happy paramilitary tough guys, sounds like the setup to a Kevin James movie. But it worked for Cruz, who has now officially entered the 2016 race, because it allows him to not only hammer the Obama administration on failing to secure the border, but it also lets him rag on one of the right’s favorite targets.

For a party whose unifying plank is an antipathy to taxes, hating on the IRS isn’t all that surprising. The agency may only be enforcing tax rules imposed by Congress, but, in politics, the messenger almost always gets shot. Nevertheless, calling for the IRS’s total elimination seems, well, insane. Cruz may be for a smaller federal government, but he’s not advocating for no federal government. As long as there is a federal government doing things even Cruz approves of, like awarding patents, sending military aid to Israel, and, yes, even guarding the border, there’s going to be a need to generate enough funds to pay for those initiatives. In short, we need the IRS—right?

On the surface, Cruz’s statement reads like former Texas Gov. Rick Perry‘s famous “oops” moment during the 2012 campaign, when he couldn’t remember which three federal agencies he wanted to eliminate. It seemed like Perry was reflexively spouting rhetoric about downsizing the government without putting much thought into what that actually entailed. But Cruz’s idea is not quite as ridiculous as it initially seems. In fact, the push to vaporize the IRS is one that’s been slowly gaining momentum for decades.

While Cruz’s specific plan involves implementing a flat tax, which eliminates all deductions and charges everyone from Warren Buffett on down the exact same rate, the argument has largely crystallized into something far more radical. It’s called the Fair Tax, and it would completely reshape both how the government collects revenue and the fundamental relationship individual American citizens have with their government.

Cruz spokesperson Catherine Frazier told the Dallas Morning News that the senator views the Fair Tax as “ideal” but believes a flat tax is more politically realistic. However, that political calculus may soon change. In 2013, Fellow GOP presidential contender Sen. Rand Paul (R-Ky.) joined with Citizens United—yes, that Citizens United—for a video calling for the IRS’s abolition.

If Republicans hold onto both houses of Congress and retake the White House in 2016, the Fair Tax may sit near the top of their agenda. When a Fair Tax bill was was introduced in Congress earlier this year, it attracted 67 cosponsors, all of them Republicans. This level of support makes it something that everyone needs to start taking very seriously.

Nuts and bolts

The first thing that pretty much everyone agrees on, no matter where they sit on the political spectrum, is that the current tax system is a confusing mess. The tax code is 2,600 pages long. Even IRS Commissioner John Koskinen admitted to reporters last year that the system is far too complex and urgently needs to be simplified.

Calling for the IRS’s complete and total elimination seems, well, insane.

Americans spend an estimated 6.1 billion hours each year filing their taxes at an estimated total cost of $37 billion. Fair Tax advocates argue that their proposal would eliminate the whole mess.

The premise behind the tax is relatively simple. First, eliminate the income tax completely, both for individuals and corporations. Second, replace it with a sales tax on every product sold in the United States, which would be collected by each individual state. Every single person and organization—from Madonna to Obama, from Church’s Chicken to the Catholic Church—pays the exact same rate on everything they purchase, no exceptions. Third, the states pay that money to the federal government. Fourth, replace everyone at the IRS with a single employee in the Treasury Department who would double-check that states made their payments successfully.

No more filling out tax returns, no saving receipts, no more shelling out a few hundred bucks to H&R Block to do the damn thing for you, no more audits, no more companies stashing $2.1 trillion overseas to avoid having to pay taxes when those international earnings are repatriated.

The moral case

When Craig Bergman decided to make a movie about abolishing the IRS through the establishment of a Fair Tax, his focus wasn’t on the economics. Unfair: Exposing the IRS is a labor of love for the Iowa-based activist. He wrote, produced, and narrated his broadside against the American taxation system because he noticed a divide among conservatives. Bergman had been a conservative virtually since birth, but only became active in the religious arm of the movement when Southern Baptist Minister Pat Robertson beat the eventual Republican nominee, then-Vice President George H. W. Bush, in the 1988 Iowa caucuses.

“I didn’t understand why people of faith aren’t understanding this tax issue. Why are they always worried about babies and marriage and not about oppressive taxation?” he says in a phone interview. “I decided to make the movie focus not about the economics, because I think that case has been made completely and totally … [I wanted to] make the moral case. Let’s tell people that tax policy is moral policy and wake up this 30-40-50-60 percent of Americans who vote on social issues and don’t vote about taxes because it’s way over their heads.”

At its core, Bergman’s argument is that the income tax, which was first established during the Civil War, violates the inalienable right of Americans not to have the government nosing into their personal lives without a very good reason.

“Five of the principles in the Bill of Rights are violated by the income tax system,” he says. “First, the framers didn’t want an income tax system. In Article 1 Section 8 [of the Constitution], they said the government can collect [revenue] with trade, tariff, and excise taxes. This is the way most governments have collected taxation throughout history. They are considered legitimate forms of taxation.”

Since 2004, the number of people who rate the IRS’s performance as “poor” jumped from 15 percent up to 42.

Furthermore, Bergman argues, the First Amendment guarantees the right to free association for everyone regardless of creed. By giving different treatment to a church than to a union or a for-profit business effectively favors certain types of associations over others, he says. The Fifth Amendment, which guarantees the right to due process, puts the burden of proof on government prosecutors to prove that someone did something wrong. However, income taxes demand that people disclose information about their personal finances no matter what. “The IRS says you will fill out these forms. You will testify against yourself. If you make any mistakes, we will use it against you and, again, you go to prison,” Bergman insists.

He adds that there’s evidence that a system based entirely on consumption taxes works. Prior to the establishment of the income tax, all taxes collected by the federal government were on consumption. He points to the state of Texas, which has no income tax and funds its programs with a 8.25 percent sales tax and is the fifteenth largest economy in the world. Granted, this assertion does come with some important qualifications. Like a lot of red states with low or even nonexistent income tax rates, Texas programs suck up far more money out the federal tax pool than the state puts in.

In a macroeconomic sense, consumption taxes tend to do a better job of growing the economy than do income taxes. Bergman asserts that if the U.S. makes the switch to the Fair Tax, it’ll put the American economy far ahead of its international competitors. “It’ll be like when America was king of the world at the turn of the century, and the rest of the world was still peeing in their drinking water,” he says.

Roberton Williams, a fellow at the Urban Institute’s Tax Policy Center, who is far from bullish on the Fair Tax, noted that, in general, economists tend to favor consumptions taxes over those based on income. “Economists don’t like to take savings and investment,” he explains over the phone. “Our current income tax taxes the money that you’ve saved and invested and the return on your investments. That is income; it’s perfectly sensible to tax that, but it does have some negative incentive effects on people’s savings and investing. You need a higher return than you otherwise would need to make your investment worthwhile.”

“The problem with the Fair Tax,” he says, “is making the Fair Tax fair.”

Fairness and simplicity

In the late 1980s, U.K. Prime Minister Margaret Thatcher had an equally radical plan to transform how her nation collected revenue. She wanted to abandon the U.K.’s progressive tax system, which charged people different rates based on how much money they earned, with a poll tax levied at a uniform rate on every single household.

The tax would have hit everyone at the same rate no matter how many pounds they had in the bank. It struck a very large number of people as a deeply unfair transfer of wealth to the rich, who would have had seen their tax bills drop dramatically.

To say that the idea didn’t go over well would be an understatement. There were riots in the streets. The backlash to the proposal was largely credited with Thatcher’s political downfall and led to the end of her tenure as prime minister.

While not uniformly progressive, one of the core tenets of the American system is that taxes are supposed to hit rich people harder than poor ones. The simple arithmetic of taxation is that if you tax something, you get less of it. As such, taxes are one of the clearest ways in which a society shows what its values are.

One of those deeply American values embedded in the DNA of the tax code is fairness. When 2012 GOP presidential candidate Mitt Romney was caught on camera saying that the 47 percent of American citizens who don’t pay any federal income taxes were lazy bums living off the government dole, it was major a scandal because it showed a lack of understanding that making rich people pay a larger share of their income in taxes is, at it’s heart, a deeply moral position. The rich benefit from American society more than the poor, so paying their fair share is precisely that—fair.

Slapping a one-size-fits-all sales tax on everything would necessarily hit those less well off a lot harder than the affluent because poor people tend to spend a far higher percentage of their income on a paycheck-to-paycheck basis than rich ones. Tacking on the 30 percent sales tax proffered in the most recent version of the Fair Tax introduced in Congress would be about as regressive as tax policies come.

“It’s really, really hard not to give a tax break to the rich with this kind of system.”

The solution, explains Steven Hayes, is something called a prebate. A self-described “recovering tax attorney,” Hayes is the president of Americans For Fair Taxation, a group formed in the early 1990s by a cadre of Houston businessmen—including the billionaire future owner of the Houston Texans, Bob McNair—to study and advocate for the Fair Tax.

A prebate is a cash payment going to every American each month as a way to make up for the inherent unfairness of the Fair Tax. The prebate, Hayes says, would effectively exempt a certain level of monthly spending from taxation by making up for the loss with a check. “The amount given in the prebate is based on the federal poverty level,” Hayes says. “Everyone would file with the Social Security Administration and receive a prebate based on their family size.”

What’s notable here is that everyone in the country gets the same dollar figure with no consideration given to things like money in the bank or an area’s cost of living. The check you get each money and the check that Bill Gates gets each month are exactly the same size. It seems unfair, especially because $500 a month goes a hell of a lot further to Tulsa, Okla., than it does in New York City (and Bill Gates doesn’t need to ever get any money from the government), but the goal here is simplicity. Giving everyone the same size prebate means there’s no reason for people to report income or for the government to manage a complex bureaucracy to determine how much is given to each individual citizen.

Additionally, Bergman argues, the guaranteed income from the prebate could be used as collateral for a loan:

Let’s talk about John, living in the inner city. He didn’t even graduate high school. He’s not good enough playing football to get picked up anywhere. His life expectancy in a major city is that he’s going to be dead or in prison by the time he’s 30. He can’t get a job, nobody’s going to hire him. He has no future except to sit on the government dole, get caught up in the drug culture, and die. It’s terrible. Under the prebate, he turns 18. He’s got that $500 per month that’s guaranteed to come in. He can sit down and say, ‘My brother and I are going to buy a used pickup truck and go into the moving business with the money borrowing against this guaranteed income stream.’ The opportunities are almost limitless.

However, according to critics of the Fair Tax, the prebate is half the problem. “[The prebate] is a single instrument that’s very blunt. It certainly cannot replicate the kind of progressive we have in the tax system now,” Williams insists. “It’s hard to make a prebate big enough … to get the same kind of distribution we have right now.

“To get a truly progressive tax code under this system, you’d have to have rebates that vary greatly across the income distribution,” he adds. “One prebate is not going to be able to mimic the current distribution with one tax rate. It’s really, really hard not to give a tax break to the rich with this kind of system.”

The other half of the problem is setting a tax rrate that raises the same amount of revenue as the current system. While Fair Tax advocates have spent decades, not to mention a giant heap of cash, running the numbers, a 2005 analysis by the Institute on Taxation and Economic Policy found that a sales tax rate of 50 percent would be necessary make the Fair Tax revenue neutral from the current system, which would make the necessary prebate enormous. Another study by the Brookings Institution put that number at 60 percent.

“Do we want something simplified? Yeah, everybody would love to have a simpler system,” Williams says. “Is what we have very inefficient? Yes, its absolutely inefficient. Do we want to go to something simple? Well, once we start talking about the details of a consumption tax, it may not be as simple as it looks.”

Starving the beast

The Fair Tax, in its various iterations, has been kicking around Washington since the 1990s, starting with Louisiana Congressman Billy Tauzin, who pitched an early version of the idea. Yet, decades later, the IRS hasn’t exactly been abolished. Instead, the agency’s conservative enemies in Washington have worked out a half-measure: cutting its funding to the bone.

Over the past five years, the Republican-led Congress has cut the IRS’s budget by 17 percent—$1.2 billion.

That’s over 1,000 people, most of them wealthy, who owe hundreds of thousands of dollars to the IRS and are getting away scot-free.

“We deliberately lowered IRS funding to a level that will make the IRS think twice about what … [they’re] doing and why … [they’re] doing it,” Rep. Ander Crenshaw (R-Fla.) said at a congressional hearing last month.

The reason for these cuts has as much to do with a general antipathy towards taxation as it is a reaction to a series of scandals that have rocked the agency in recent years. There were revelations that IRS managers spent $4.1 million on an expensive conference in Anaheim, Calif., in 2010. There were also the allegations that IRS officials had targeted Tea Party groups for extra scrutiny when they applied for tax-exempt status on the suspicion that if a group had the words “tea party” in its name and claimed the majority of its activities were non-political, there might be something fishy going on.

The IRS didn’t exactly help itself in the latter case when Lois Lerner, the official at the center of the scandal, claimed to have deleted tens of thousands of emails and then invoked her Fifth Amendment right against self-incrimination and refused to testify before Congress.

Nevertheless, the result is an agency tasked with doing more with less. With the taxable population growing and the implementation of the Affordable Care Act, which handles compliance with the individual mandate to carry health insurance through the IRS, the agency has had to significantly scale back basically everything it does.

The number of audits, both on individual returns and at the corporate level, are down considerably. The IRS’s enforcement arm has been cut by 5,000 revenue agents and investigators and officials have estimated that some $2 billion will be left on the table because the agency simply doesn’t have the resources to pursue the cases.

A Washington Post report found that the IRS has been forced to stop going after people who owe under $1 million in back taxes. In the city of Dallas alone, that’s over 1,000 people, most of them wealthy, who owe hundreds of thousands of dollars to the IRS and are getting away scot-free.

The area in which the cuts are having the most noticeable effect on the general public is customer service. The IRS employs an army of staffers to assist taxpayers with any questions they have about their returns. Thanks to budget cuts, 60 percent of people who call the IRS with questions never get to speak to a human being. Instead, they receive “courtesy disconnects” after waiting on hold for hours. The term is a fiendishly polite euphemism for having the IRS just hang up on them. There have been 5 million courtesy disconnects so far this year.

To quote former Secretary of Defense Donald Rumsfeld, “You go to war with the army you have, not the army you might want or wish to have at a later time.”

Speaking on NPR’s Diane Rehm Show last week, IRS National Taxpayer Advocate Nina Olson explained how the lack of service hits the poor especially hard:

Over 45 percent of the individual taxpayer population has income at or below 250 percent of the federal poverty level, which is essentially the working poor. That population tends to need assistance. We have a lot of data that they can’t go online … They want to either call or go to a walk-in site. With the budget cuts this year, taxpayer service declined about 5 percent from the year before. We can’t hire people to temporarily staff the lines during the filing season. We get 10 million pieces of correspondence and 100 million phone calls every single year.

If you can’t get through to the IRS on the phone, you write the IRS a letter. We don’t respond to that letter in time; over 50 percent of our correspondence is over age. You pick up the phone again, and you call again, and then you don’t get a response again. It becomes an endless cycle.

IRS Commissioner John Koskinen recently called the state of the agency’s customer service “abysmal.”

Thomas Wilson, a long-time senior IRS official, who has since retired, notes that not being able to provide the same level of assistance to ordinary taxpayers triggers a litany of problems, both in people’s ability to file accurate returns and also to avoid identity theft.

“That whole process has downstream effects in terms of overall compliance—depending on taxpayers’ financial situation and whether or not they’re able to get professional help,” he says. “Some, who are affluent enough, will get professional help, but a lot of the other folk won’t. The other thing that’s really impacted by this, on the compliance side, the IRS has been … dealing with false returns. It creates a refund, such as an earned income credit, where a person who really does not even pay tax can get a refund.”

As these types of scams become more prevalent, the IRS has had to create a whole new internal system to protect against it—which, in turn, has further strained its budget.

Polls have shown this downgrade in the IRS’s ability to do its job has had a serious effect on the public’s perception of the agency. A Gallup poll found that, since 2004, the number of people who rate the IRS’s performance as “poor” jumped from 15 percent up to 42. The survey found that the IRS was the most unpopular government agency of the nine pollsters asked about.

Representatives from the IRS did not respond to an interview request for this article.

The ultimate result is a self-fulfilling prophecy. Republicans want to abolish the IRS but can’t quite make it happen politically. The alternative is to defund the agency to the point where everyone else is as fed up with the income tax system as they are, to create a moment where abolishing the IRS may seem like a radical idea, but not an entirely unreasonable one.

That’s where we are right now: When two of the leading Republican candidates for president regularly call for eliminating the IRS, it doesn’t seem all that far off from one where a Fair Tax proposal could reach the president’s desk and beyond. In a world like that, well, Ted Cruz’s calls to “abolish the IRS” suddenly don’t sound all that insane.

Photo via Gage Skidmore/Flickr (CC BY SA 2.0) | Remix by Jason Reed

Share this article
*First Published: Apr 15, 2015, 9:15 am CDT