It is understood that the healthcare giant is not renewing its TOP (The Olympic Partnership) sponsorship for 2009 to 2012 because it was disappointed with the lack of people admitted to the sponsor's area in Beijing due to security restrictions.
Johnson & Johnson's sponsorship deal for 2005 to 2008 was worth between $60m and $80m and it is understood that the renewal would have brought the IOC another $100m over four years.
A spokeswoman for J&J said that it was not renewing its sponsorship to enable the company to focus on other business priorities.
J&J's decision leaves the IOC without a partner in the healthcare products category and follows the departure of Kodak, ManuLife and Lenovo from the list of Olympic sponsors.
The IOC has replaced Lenovo with Acer but it returned ManuLife's insurance category rights back to the 200 national organising committees and added Kodak's digital camera category rights to Panasonic's sponsorship.