Little more than a week after the first stone was laid, the two main characters in the project are mired in recrimination.

Amid great fanfare Mr Libeskind's design for the Freedom Tower won the public competition to redevelop the site. He was chosen in February 2003 by the Lower Manhattan Development Corporation and the Port Authority of New York and New Jersey. But within three months another architectural firm working for Silverstein Properties had taken over.

"While Silverstein disingenuously claimed to support the masterplan, his actions, then and up to the present time, bespeak a clear intent to derail the project wherever he perceives a conflict with his personal financial interests," claims the lawsuit filed in the New York state supreme court on behalf of Mr Libeskind, his wife, Nina, and their company, Studio Daniel Libeskind.

A spokesman for Silverstein Properties said the developer was surprised by the lawsuit. "It has always been our desire to avoid a senseless courtroom fight," Howard Rubenstein told the New York Times. "That being said, we fully expect to prevail."

Mr Silverstein is already in a legal battle with his insurers over how much money he will receive for the destruction of the twin towers, which will determine how much money he has to rebuild the site.

In a sign that there may be room for negotiation, if not compromise, the Libeskinds' suit says they are prepared to defer talks about payment until Mr Silverstein has completed discussions with the insurers.

And while the suit says the design of the 541-metre (1,776ft) tower had been changed considerably it concedes that it still "retained the spirit and key elements of Libeskind's vision".

Ms Libeskind said Studio Daniel Libeskind "has been paid nothing for its work". Mr Silverstein's company does not deny it has not paid the company, but says Mr Libeskind has already received money from elsewhere.

Mr Rubenstein said: "[Mr Libeskind] has already been paid many millions of dollars for his work by the Lower Manhattan Development Corporation and the port authority and is unable to provide any industry-standard timesheets or other documentation to justify an additional payment of more than $800,000."

Ms Libeskind acknowledged that the office kept no time sheets but said it was standard industry practice for architects to bill a percentage of total construction costs.

This friction has existed from the outset although it has never before reached this level.

Mr Silverstein's representatives pointed out that even before a design had been chosen their boss had said he was "willing to work with people".

But the developer will decide everything about the design. We're the ones who have to [let] these buildings".