Schmidt joined Apple's board of directors in August 2006. His resignation comes as rivalry between the two Silicon Valley giants continues to grow as they encroach on each other's territory.
Last month Google revealed it was developing a computer operating system to compete with Apple's own OS and Microsoft's Windows.
Separately, Google's web browser Chrome has also taken a market chunk out of Apple's Safari but most importantly Google wants to be a serious rival to Apple's iPhone with its Android phone.
The iPhone, however, has a huge start having sold more than one million units in the UK alone.
Tensions increased again last week when from its App Store, citing the fact that it duplicates a number of features which already come native to the iPhone.
The Federal Communications Commission has launched an investigation into Apple's rejection of the app and has sent letters to Apple, Google and AT&T, which is the sole distributor of the iPhone in the US.
Steve Jobs, Apple's CEO, said: "Eric has been an excellent board member for Apple, investing his valuable time, talent, passion and wisdom to help make Apple successful.
"Unfortunately, as Google enters more of Apple's core businesses, with Android and now Chrome OS, Eric's effectiveness as an Apple board member will be significantly diminished, since he will have to excuse himself from even larger portions of our meetings due to potential conflicts of interest.
"Therefore, we have mutually decided that now is the right time for Eric to resign his position on Apple's board."