The 29bn euro (£24.5bn) merger of the London Stock Exchange and Deutsche Boerse
could collapse after the LSE said the deal was unlikely to be approved by the European Commission.
The commission had ordered the LSE to sell its 60% stake in MTS, a fixed-income trading platform.
However, the LSE said the request was "disproportionate".
It warned investors it would struggle to sell MTS and that such a sale would harm its ongoing business.
As a result, the LSE said: "Based on the commission's current position, LSE believes that the commission is unlikely to provide clearance for the merger."
Who killed the LSE-Deutsche Boerse deal?
The two rival exchanges announced plans for a "merger of equals" about a year ago, aiming to create a giant trading powerhouse that would better compete against US rivals.
They had already agreed to sell part of LSE's clearing business, LCH, to satisfy competition concerns before the commission's surprise demand concerning MTS earlier this month.
The Commission had given the exchanges until Monday to come up with a proposal to meet that demand.
LSE said that such a sale would need regulatory approval from several European governments and would hurt its wider Italian business.
"Taking all relevant factors into account, and acting in the best interests of shareholders, the LSE Board today concluded that it could not commit to the divestment of MTS," the exchange said on Sunday night.
London Stock Exchange/Deutsche Boerse merger in doubt
Dominic O'Connell, Today business presenter
When the London Stock Exchange trumpeted its merger with Deutsche Boerse last year, the chairman, Donald Brydon, was remarkably relaxed about the Brexit vote. He and his German counterparts said the deal would go ahead regardless of the outcome. What they didn't know was that it would eventually fall foul of another aspect of the single market - Brussels' control of competition laws.
The two sides have made no bones about the fact the merger would create a company with a powerful presence in key markets, and have offered a number of sops to competition authorities in an attempt to win their approval. But it was not enough; Brussels insisted the LSE cede control of MTS, an important Italian clearing house with a crucial role in the trading of Italian government debt.
For the LSE board, this was a step too far. A sale of MTS would be too harmful, the directors decided, and they have chosen not to meet an European deadline to say how they would get rid of it. The Brits say they want to go ahead with the merger, but it looks like an insurmountable obstacle. The LSE will either go it alone in a post-Brexit world, or - as has often seemed possible before, and even more so now - fall into the arms of one of the giant American exchanges.
sourse: bbc.com