If reviewers are named then they’ll refuse to give critical reports, even when deserved and necessary. The reviewer giving a critical report could easily make enemies, and yet has nothing to gain by reviewing. So they won’t.
If an anonymous reviewer is being unfair, then it’s the editor’s job to spot that and do something about it, such as ask another reviewer.
I don’t agree at all. The reviewers are chosen because they have previously shown at least some expertise in the small topical area that the manuscript covers. But these reviewers have a body of work they feel good about. If the manuscript attacks consensus conclusions or methods, that is a direct attack on the reviewers’ sense of accomplishment. The more radical the manuscript, the worse the bias of the reviewers will be. Ai doesn’t have a reputation or ego to defend. Thus I would much rather have ai review my work.
The foregoing assumes the reviewers are conventional scientists. By and large, of course they are.
100% agree. Good piece. I am sure you have heard of this benign looking and very popular 'club' of universities seeking broader assessment research criteria. (ie., 'DORA', and CAORA out of the EU). DORA is committed to moving beyond impact rankings. Alone, true- they have been grossly misused. But we need to be a bit more nuanced, as your piece suggests. Our institution was trying to ram DORA signing through without any consideration of consequences. Seems to open the door to the dangerous 'E for effort' mentality that is destroying the last gasps of evaluation based on merit.
The parts about AI and peer reviews stand out to me the most. I also think that transparency is important. Thank you for writing this piece.
If reviewers are named then they’ll refuse to give critical reports, even when deserved and necessary. The reviewer giving a critical report could easily make enemies, and yet has nothing to gain by reviewing. So they won’t.
If an anonymous reviewer is being unfair, then it’s the editor’s job to spot that and do something about it, such as ask another reviewer.
I don’t agree at all. The reviewers are chosen because they have previously shown at least some expertise in the small topical area that the manuscript covers. But these reviewers have a body of work they feel good about. If the manuscript attacks consensus conclusions or methods, that is a direct attack on the reviewers’ sense of accomplishment. The more radical the manuscript, the worse the bias of the reviewers will be. Ai doesn’t have a reputation or ego to defend. Thus I would much rather have ai review my work.
The foregoing assumes the reviewers are conventional scientists. By and large, of course they are.
100% agree. Good piece. I am sure you have heard of this benign looking and very popular 'club' of universities seeking broader assessment research criteria. (ie., 'DORA', and CAORA out of the EU). DORA is committed to moving beyond impact rankings. Alone, true- they have been grossly misused. But we need to be a bit more nuanced, as your piece suggests. Our institution was trying to ram DORA signing through without any consideration of consequences. Seems to open the door to the dangerous 'E for effort' mentality that is destroying the last gasps of evaluation based on merit.