Icahn, 76, who has gone from feared corporate raider to activist investor in more than three decades of dealmaking, called Ackman, 46, a "liar" and "the most sanctimonious guy I ever met in my life."
In a tirade that included expletives, Icahn said he would never invest with Ackman and predicted investors in his Pershing Square hedge fund would lose a lot of money on the Herbalife bet. U.S. news outlets have reported that Icahn has a long position in Herbalife, although he has not confirmed that.
Ackman, who is usually much more restrained than Icahn when speaking publicly, got in a few verbal shots himself. Icahn "does not have a good reputation" and "is not an honest guy," he said.
The altercation quickly became the talk of the financial world, both on trading floors and on Twitter.
"I will never delete today's Ackman vs Icahn CNBC debate from my DVR. Even when DVRs are like Betamaxes, I'll force my grand kids to watch," tweeted Eric Jackson of hedge fund Ironfire Capital LLC.
Business Insider Deputy Editor Joseph Weisenthal tweeted: "It's never going to get any better than what we just saw."
The influential blog ZeroHedge called it the "Ultimate Hedge Fund Deathmatch: Icahn And Ackman As The Real Billionaire Husbands Of CNBC Going Wild."
BusinessInsider held an online poll: "Who Won The Brawl Between Carl Icahn And Bill Ackman?" As of 5 p.m. New York time, 70 percent of the more than 1,800 people who voted declared Ackman the victor.
Neither Ackman nor Icahn could be reached for comment after the CNBC show.
Ackman has called Herbalife a "well managed pyramid scheme" that he predicted will collapse. Herbalife has denied the allegation.
"On CNBC today, Mr. Ackman continued to misrepresent Herbalife," an Herbalife spokeswoman said on Friday.
"Herbalife is a financially strong and successful company, having created meaningful value for shareholders, significant opportunities for distributors and positively impacted the lives and health of our consumers over our 32-year history."
The genesis of their feud stems from a nasty contractual dispute involving a real estate company deal 10 years ago, when Ackman was running his former hedge fund, Gotham Partners, which he co-founded in 1993 with former Harvard Business School classmate David Berkowitz.
After eight years of litigation, a court ruled in Ackman's favor and Icahn was forced to pay Gotham $4.5 million, plus interest.
Indeed, the discussion on CNBC often had little to do with the merits of Ackman's view of Herbalife. At times, Icahn also feuded with CNBC host Scott Wapner, accusing him of "bullying" him when he was pressed on whether he had gone "long" on Herbalife shares.
Icahn may have little to lose from the dustup given that he is simply managing his own money these days. Ackman, on the other hand, manages about $11 billion at Pershing Square, including money from many prominent institutional investors.
Herbalife shares briefly surged over 5 percent when Icahn said during the row that Ackman, by going public with his big short position, would cause the "mother of all short squeezes" in the stock.
A short squeeze is when short sellers are forced to cover their position, a move that pushes a stock higher.
Ackman and Icahn's dislike of each other is well known in financial circles. Some of the tension stems from the fact they both specialize in the same game - taking big positions in companies and agitating for management changes.
Ackman's bet against Herbalife is also being challenged by another big hedge fund player, Third Point manager Daniel Loeb, who has said he holds a big long position on the stock.
(Reporting By Sam Forgione, edited by Matthew Goldstein, Jennifer Ablan, Martin Howell and Andre Grenon)