The Epistemological Argument
The first argument contra could be called the 'epistemological' argument or the argument from ignorance: it can't be known with certainty that one accused of a capital crime is guilty. The argument sometimes takes this enthymematic form:
P2. Capital punishment is sometimes inflicted on the innocent.
C. Capital punishment ought to be banned.
But this argument is invalid without the auxiliary premise:
P1. Any type of punishment that is sometimes inflicted on the innocent ought to be banned.
In the presence of (P1) the conclusion now follows, but (P1) cannot be accepted. For if we accept it, then every punishment ought to be banned. For every type of punishment has been at some time meted out to the innocent. Obviously, to be found guilty in a court of law is not the same as to be guilty of the crime with which one has been charged.
Our first argument, then, suffers from probative overkill: it proves too much. I reject the argument for that reason, and you ought to too.
If the wrong person has been executed, that person cannot be restored to life. Quite true. It is equally true, however, that if a person has been wrongly imprisoned for ten years, then those years cannot be restored to him. So the cases are exactly parallel. At this point liberals will often say things that imply that their real objection to capital punishment is that it is capital. Well, yes, of course: it has to be.
For the punishment must fit the crime, and anything less than capital punishment for certain crimes violates the self-evident moral principle that I put in italics. Justice demands capital punishment in certain cases. If you don't agree, then I say you are morally obtuse. On this issue either you see that justice demands capital punishment in certain cases or you are morally blind. End of discussion. To argue with the morally blind is as pointless as arguing with the color-blind and the tone-deaf. The morally blind need to be defeated by political means. There is no point in further discussion with them.
The Consistency Argument
Another argument repeatedly given against capital punishment is that it involves doing to a person what in other circumstances would be deemed morally wrong. We could call it the 'consistency argument.' The argument is that, since killing people is wrong, then the state's killing of people is also wrong. The trouble with this argument, however, is that it, like the preceding argument, 'proves too much.'
For if the argument were sound, it would show that every type of punishment is impermissible, since every type of punishment involves doing to a person what otherwise would be deemed morally wrong. For example, if I, an ordinary citizen, demand money from you under threat of dire consequences if you fail to pay, then I am committing extortion; but there are situations in which the state can do this legitimately as when a state agency such as the Internal Revenue Service assesses a fine for late payment of taxes. (Of course, I am assuming the moral legitimacy of the state, something anarchists deny; but the people who give the sort of argument I am criticizing are typically liberals who believe in a much larger state than I do.)
The state is a coercive entity that limits the liberties of individuals in all sorts of ways. It has to be coercive to do its job. If you hold that the state is practically necessary and morally justifiable, then you cannot reasonably balk at the state's killing of certain of its citizens in certain well-defined circumstances.
The main purpose of the state is to protect life, liberty, and property. It cannot do this if it does not have the power to punish those who take life and liberty and property.
If justice demands the execution of certain miscreants then this justice must be administered by some agency. It had better be an agency dispassionate and impartial hedged round by all sorts of rules and safeguards. Otherwise vigilantism. The job falls to the state.
The Cost Argument
And then there is the 'cost' argument. The idea is that capital punishment is not cost-effective. It is claimed that the benefit to society does not outweigh the cost. A utilitarian might be able to rig up such an argument, but I am not a utilitarian. The issue is one of justice. Justice demands capital punishment in certain cases, and it doesn't matter what it costs, or whether there is any benefit to society, or even whether there is any society to benefit. Recall Kant's last man scenario from Metaphysics of Morals, Part II (emphasis added):
 But whoever has committed murder, must die. There is, in this case, no juridical substitute or surrogate, that can be given or taken for the satisfaction of justice. There is no likeness or proportion between life, however painful, and death; and therefore there is no equality between the crime of murder and the retaliation of it but what is judicially accomplished by the execution of the criminal. His death, however, must be kept free from all maltreatment that would make the humanity suffering in his person loathsome or abominable. Even if a civil society resolved to dissolve itself with the consent of all its members--as might be supposed in the case of a people inhabiting an island resolving to separate and scatter themselves throughout the whole world--the last murderer lying in prison ought to be executed before the resolution was carried out. This ought to be done in order that every one may realize the desert of his deeds, and that blood-guiltiness may not remain upon the people; for otherwise they might all be regarded as participators in the murder as a public violation of justice.
In any case, there is nothing necessary about high costs. They could easily be reduced. A limit could be set to appeals -- and ought to be set to them. Endless appeals make a mockery of justice. And if malefactors were executed in a timely fashion, the deterrent effect would be considerable. Thus a fourth argument, the 'no deterrence' argument, is also worthless in my opinion. Apart from the suicidal, people love life -- criminals included. Swift and sure execution for capital crimes could not fail to have a deterrent effect.
I will add that if it could be shown that in some jurisdiction the capital penalty was not being applied fairly and justly (due to prejudice against people of Middle Eastern descent, let us say), then I would support a moratorium on the penalty in that jurisdiction. But this question is distinct from the question of principle. That alone is what I have been discussing.