French football federation audit: Le Graet's no longer has legitimacy as president
The audit, whose conclusions were made public on Wednesday, also "highlighted the inappropriate behaviour of Mr Le Graet towards women".
Last month, Le Graet took a step back as FFF chief with Philippe Diallo taking charge on an interim basis, after Le Graet criticised France great Zinedine Zidane and came under fire over the extension of coach Didier Deschamps's contract until 2026 - two years after his mandate expires.
The Paris prosecutor's office opened an investigation for moral and sexual harassment against Le Graet, who denies any wrongdoing.
"It is up to the federation's authorities to play their part in managing this crisis and allow the FFF to make a fresh start," French sports minister Amelie Oudea-Castera told a news conference on Wednesday.
"The status quo is impossible. Noel Le Graet is a man of decisions and I hope he will make the right one, for the federation and for himself," she added, although the French government must tread carefully as world governing body FIFA's regulations prohibit interference by political authorities.
A meeting of the FFF's executive committee is expected to take place on Friday.
The executive committee last month voted against asking Le Graet to step down after the legal investigation was opened, saying it was waiting for the conclusions of the audit.
"On the basis of specific testimonies and documents provided by certain witnesses, the rapporteurs believe that this behaviour cannot be considered respectful of the dignity of these people and does not comply with the obligation to set an example expected of the president of a federation," the audit stated.
"The hearings conducted by the mission revealed that the inappropriate and insulting nature of Mr Le Graet's remarks may have been accentuated by excessive alcohol consumption."