Delaney Lund Knox Warren & Partners, J. Walter Thompson, Clemmow Hornby Inge, DDB London, Euro RSCG London and WCRS will battle it out for the business, with a decision expected by 1 May.
Cilla Snowball, AMV's chief executive, said the agency felt it had "given its best" and had chosen not to take part after much consideration. She added that the agency hoped there would be no redundancies as a result.
"The decision not to enter the pitch is a tough one but the right one for the agency at a time when our client and new client agenda is busy and we have genuinely given our best," she said.
The move ends an eight-year relationship, during which time AMV produced memorable television ads starring the Only Fools and Horses actor Nicholas Lyndhurst.
But the retailer has struggled to forge a strong brand identity in an increasingly competitive market. Disappointing sales figures over the festive period forced it to issue a third profit warning in less than a year.
Katie Spriggs, the brand communications director for WH Smith, said: "We have decided to invite a number of agencies to give us their best objective thinking on how we can further build upon these brand assets and position WH Smith most competitively."
The media incumbent, Starcom Motive, is competing with Walker Media, MediaCom, Universal McCann and Carat for the £9 million media account.