I don't think they should be banned for making bad calls. Refs see what they see, punishing won't make them see any better if it's offside or not. You don't bann a player for performing poorly either, you bann them if they do something that goes against the rules. If someone is consistently making bad calls, then either give him a weekend off to settle down or demote him, but I'm against banning someone over 1 or 2 bad games.
However I do think it would be fair to bann a ref that is consistently making poor interpretations of the rules. As in refs that see what happens (or what doesn't happen) and react in a very questionable manner. Losing control of the game and carding everyone, or not carding anyone despite the players butchering each other... or even worse, refs that are highly inconsistent with the way they punish the same offenses... imo they are much more to blame for that kind of stuff than for missing out on a penalty or whatever, as they are human and might not see what we see on tv.
I know that this season there have been a lot of poor performances by the refs, but it has only been one season. There's a bad run, it happens. I'm sure that if we looked at a sample of 1000 games instead of 100 there wouldn't be much of a statistical difference.