Nelson, a former managing director of Guinness GB, will be responsible for reversing the fortunes of the UK's third-biggest supermarket.
He has been handed the job following a restructure of senior management.
Iceland managing director Mike Coupe will replace Nelson as trading director.
Nelson joined Sainsbury's in July 2003 from Diageo, where he was president for the South-West US. Prior to that he held a number of senior positions at Diageo, including commercial director.
Former Sainsbury's joint managing director Sara Weller, who quit to join Argos in March, was previously in charge of marketing activity.
Chief executive Justin King announced changes to the board during the supermarket's results presentation last week. The marketing director position was the only role that had remained vacant.
A Sainsbury's spokeswoman said Nelson would not be reviewing its creative agency arrangements.
Abbott Mead Vickers BBDO has worked with the supermarket for 24 years and recently began working on the latest series of Jamie Oliver ads, after the celebrity chef signed a further 12-month contract for £1.2m.
Coupe's responsibilities at Iceland ended on 21 May, but a start date at Sainsbury's is still being negotiated. Nelson will oversee trading in the meantime.