The academic world is currently grappling with a controversy surrounding AI-generated studies submitted to the International Conference on Learning Representations (ICLR). Intology and Autoscience, two AI labs, have come under fire for submitting AI-generated papers without giving workshop organizers the option to reject them. This has sparked criticism from numerous AI academics, who argue that these actions co-opt the scientific peer review process.
Ashwinee Panda highlighted the issue by stating that submitting AI-generated papers without the right for organizers to refuse them showed a "lack of respect for human reviewers’ time." This sentiment was echoed by Prithviraj Ammanabrolu, who expressed frustration over the use of peer-reviewed venues as human evaluations without consent.
“All these AI scientist papers are using peer-reviewed venues as their human evals, but no one consented to providing this free labor,” said Prithviraj Ammanabrolu.
In response to the criticisms, Intology claimed that their papers received unanimously positive reviews. However, Sakana, another lab involved in AI-generated studies, took a different approach by withdrawing its ICLR paper prior to publication to maintain transparency and uphold ICLR conventions. Sakana admitted to making "embarrassing" citation errors and had informed ICLR leaders before submission, obtaining peer reviewers' consent.
This controversy has brought to light the existing problem of AI-generated content within academia. An analysis revealed that between 6.5% and 16.9% of papers submitted to AI conferences in 2023 likely contained synthetic text. In light of these findings, Alexander Doria emphasized the need for a regulated entity to conduct "high-quality" evaluations of AI-generated studies.
“Evals [should be] done by researchers fully compensated for their time,” stated Doria.
Doria further argued that academia should not be used as a platform to outsource free AI evaluations.
“Academia is not there to outsource free [AI] evals,” said Doria.
Despite the criticisms, Intology and Autoscience continue to defend their actions, pointing to positive feedback as justification. However, the debate raises broader questions about how AI-generated content should be integrated into established academic processes.
