The Icelandic investor, which is believed to hold a stake of about 30 per cent in Moss Bros via Unity, is understood to have had tentative discussions about privatising the business. It is also thought to have talked to Sir Tom Hunter about rolling in his retail businesses, which include branded fashion chains USC, D2 and footwear businesses Office and Qube.
Kevin Stanford, who holds a 6.6 per cent stake in Moss Bros, is also likely to add his All Saints chain to the group, which would be headed by Moss Bros chief executive Philip Mountford.
Industry sources told Retail Week that discussions were at a very early stage and that Baugur was focused on finalising the House of Fraser deal first, which is due to complete on November 8.
One source close to Baugur said: 'Mountford was considered for the House of Fraser job, but that didn't work out. However, there is no doubt that Baugur was really impressed by him and it wants to build a new business around him.'
However, industry observers questioned whether there were synergies between Moss Bros and Sir Tom's businesses. The chains serve a diverse customer base from designer shoppers to middle-market customers.
One source close to Baugur said a move to buy Moss Bros could be more beneficial to Baugur's House of Fraser deal in the long term, because it could boost the department store group's buying power for menswear.
Moss Bros posted pre-tax profit down 50 per cent to£800,000 for the six months to July 29, on sales of£63.3 million. Like-for-like sales for the first 10 weeks of the second half were flat. Shares were trading at 77p as Retail Week went to press, valuing the business at about£71.7 million.
A source close to Moss Bros said: 'Either someone has to buy Moss Bros or it has to buy something to make it a viable business proposition. However, the price is questionable given recent performance.'
Any deal would have to win the support of the Moss and Gee family members, who have large shareholdings in the business.
Baugur and Hunter were both unavailable for comment.