Give the agency time (how much is up to you) to address your issues with them, which you must set out clearly. A red card can sometimes transform an agency's performance on your business, for a time at least.
Use a 'marriage guidance' counsellor to keep your relationships with your main agencies on track. These companies will act as early-warning systems when relations are breaking down, and can advise how to get things back on track. If all else fails, they can help manage the break-up.
But don't shirk your responsibilities. The one who did the hiring should also do the firing, and you will need to hear from your agency why they think the partnership has collapsed: there will be lessons for both sides to learn.
Don't engage another agency before you've actually fired the incumbent. Well, that's pretty obvious, financially if not morally. But don't even indulge in some serious flirting unless your agency is clear that you're unhappy. In fact, if you really want to play with a straight bat, put your incumbent agency on notice before you even start sniffing round their competition; the more open you are, the easier it will be to part cleanly and with a degree of mutual respect.
Stay friends. People move on and you may end up working together again. And anyway, who wants to get a reputation as a bad client (there are enough of those around).