The US cereal manufacturer said it would not renew its contract with Phelps, which expires at the end of the month.
Kellogg's signed Phelps soon after his eight-medal performance at the Olympic Games in Beijing for an undisclosed sum.
The agreement put Phelps' face on the boxes of Kellogg's Corn Flakes and Frosted Flakes in an exclusive cereal deal, keeping him off rival General Mills Wheaties box -- the traditional landing spot for Olympic athletes.
Kellogg spokesperson Susanne Norwitz said in a statement: "We originally built the relationship with Michael, as well as the other Olympic athletes, to support our association with the US Olympic team.
"Michael's most recent behaviour is not consistent with the image of Kellogg. His contract expires at the end of February and we have made a decision not to extend his contract."
Kellogg's has also decided to not to renew its contract with the US Olympic team, which ended last year.
Phelps today received a three-month ban from competitive swimming, handed down from USA Swimming, which will also withdrawal its financial support for Phelps for the duration of the ban.
Phelps acknowledged that the photo was authentic and made a public apology, also saying the controversy might affect his decision to compete in the 2012 Games in London.
The swimming star reportedly earns $5m a year from endorsements and has a number of agreements with other companies, including Subway, Visa, Speedo, Hilton Hotels and Omega watches.
After the photograph surfaced, Speedo and Omega issued statements saying they are still supporting Phelps with no plans to terminate its contracts.
Subway, which nabbed Phelps for a lucrative deal ensuring he wouldn't sign with McDonald's, currently has a large Phelps television campaign nearing launch, but would not comment on the photograph.