Do I pay for answers that fail?

No. If an answer fails, its credits are refunded automatically and you can press "Retry (free)". If you stop an answer before it writes anything, it’s free; once it has started, it costs its normal message price.