US officials said investigators were assessing the credibility of the alleged plot against Crown Prince Abdullah described by an American Muslim and a Libyan intelligence officer.
The allegations, which have been dismissed as nonsense by Mr Gadafy's son, are one reason why the US has yet to remove Libya from its list of nations that support terrorism, despite negotiations that led to the lifting of sanctions this year, officials told the New York Times.
If the claims are true Mr Gadafy's bid for international rehabilitation could lie in ruins.
In separate statements to American and Saudi officials, Abdurahman Alamoudi, a Muslim in jail in Virginia, and Colonel Mohamed Ismael, a Libyan intelligence officer in prison in Saudi Arabia, revealed details of Mr Gadafy's plans to assassinate the crown prince in a bid to destabilise Saudi Arabia.
Mr Alamoudi's statements were offered as part of a plea bargain with US prosecutors.
He was indicted last October for violating US sanctions by travelling to Libya and receiving money. Mr Alamoudi told prosecutors that he met the Libyan leader twice last year.
At a meeting in June Mr Gadafy supposedly told him: "I want the crown prince killed either through assassination or through a coup." When they met again in August Mr Gadafy asked why had had not seen "heads flying".
There is enmity between Mr Gadafy and the crown prince, which flared up just before the Iraq war. After accusing each other prostituting themselves to colonial domination, the crown prince told Mr Gadafy: "Your lies precede you and your grave is in front of you."
A government official told the New York Times that if there was compelling evidence of a plot to kill one of America's allies, it could prompt a "180 degree" change of US policy towards Libya.