The political journalist quit the show saying that he wanted to stop the joke from going too far as there was a "real danger that I might win the competition", despite being consistently rated bottom of the judges' leader board.
The BBC has received 2,094 complaints so far from viewers upset about Sergeant leaving, compared with 377 complaints about him remaining in the show before he decided to quit.
There was speculation online that he had been forced to quit by the BBC as the judges of the show applied pressure to stop the latest series of 'Strictly Come Dancing' becoming a farce.
The judges placed Sergeant and his partner Kristina Rihanoff at the bottom of the leader board week after week, but thanks to votes from the public the pair remained in the competition.
Lead judge Len Goodman said that Sergeant's dancing made a "nonsense of the show" while fellow judge Arlene Phillips said he was "outstanding at dancing really badly".
On last Saturday's show, actress Cherie Lunghi was voted out of the competition after finding herself in the bottom two with model Lisa Snowdon.
Jay Hunt, controller of BBC One, said in a statement: "John has told us of his plans to leave 'Strictly Come Dancing' and we are very sad to see him go.
"He has been an entertaining contestant and is hugely popular with the viewers. We would have liked him to stay but we respect his decision to leave."