"We didn't miss cellphones, but the way that we went about it didn't allow us to get the leadership. It's clearly a mistake," Gates, Microsoft's former CEO, said in a rare interview with CBS.
Gates hedged questions on whether he was happy with Chief Executive Steve Ballmer's performance. He said Ballmer had achieved a lot but that both he and Ballmer were not satisfied.
"(Ballmer) and I are two of the most self-critical people I know," Gates said.
"There are a lot of amazing things that Steve's leadership achieved - Windows 8, the Surface computer, Bing, Xbox. Is it enough? No. He and I are not satisfied that in terms of breakthrough things we are doing everything possible."
Ballmer took over as Microsoft's CEO in February 2000. The company's shares have dropped 45 percent since then.
A former senior Microsoft executive, Joachim Kempin, said in a book he wrote about his time at the company that Ballmer was not the right leader for the world's largest software company but held his grip on it by systematically forcing out any rising manager who challenged his authority.
His criticism echoes that of investor David Einhorn of Greenlight Capital, who called for Ballmer to step down in 2011.
Microsoft has faced criticism for its latest Windows 8 versions for different devices, while its Bing search engine has won only a small market share.