A former employee of one of the leading agencies representing baseball players sued the owners of the firm in federal court, alleging they knew of players using performance-enhancing substances and even directed those players to people who could give them the PEDs.

Juan Carlos Nunez, who worked for the ACES baseball agency, sued Seth and Sam Levinson in New York State court on Monday, asking for more than $3 million in unpaid fees and expenses.

The Levinsons, who in the past have said Nunez was a rogue employee, defended themselves Tuesday.

“This is nothing more than a shakedown by a man broken by his own criminal actions,” the Levinsons said in a response to an inquiry from the MLB Trade Rumors website. “We will take the fight to Mr. Nunez for any meritless and defamatory claims, and we will seek all available remedies and damages that his criminal behavior has caused.”

Nunez served time in prison after pleading guilty to conspiracy to distribute testosterone as part of the Biogenesis scandal, which resulted in multiple suspensions in 2013, including one for Alex Rodriguez, who was initially suspended for 211 games.

Nunez admitted that he referred players to Biogenesis founder Anthony Bosch, who served time in prison as well as part of the case. Nunez was banned from baseball in 2012 after he took responsibility for creating a fake website that Melky Cabrera said he used to buy a form of synthetic testosterone. Cabrera was suspended for 50 games.

Nunez’s lawsuit claims that ACES was involved in providing PEDs to Major League Baseball players. It also details what he says is the Levinsons’ relationship with Bosch, and says that the brothers were aware that Bosch was providing HGH, testosterone and performance-enhancing substances to their clients.