Here is my peer review story. Two senior academic colleagues of mine working in physics were asked to review the work of another. They were given two options: to rate the work as "highly significant" or as "very significant". So we spent some time at lunch batting back and forth over what the difference between "highly" and "very" might be, but of course the real problem was that in truth, the two reviewers felt the work was in fact "not significant at all". But there wasn't a ticky-box for that opinion.
Furthermore, both reviewers felt that it would be a bad idea to tick such a box in any case, because that would imply that the area of study itself was insignificant, and as it was their own area they didn't want to say that. So in the end they went for "very". Even though this was an inaccurate representation of what they thought.
That is why peer review as an evaluative practice may, in fact, simply suck.
![[personal profile]](https://www.dreamwidth.org/img/silk/identity/user.png)
timeasmymeasure
no subject
Date: 2011-01-10 05:23 pm (UTC)Here is my peer review story. Two senior academic colleagues of mine working in physics were asked to review the work of another. They were given two options: to rate the work as "highly significant" or as "very significant". So we spent some time at lunch batting back and forth over what the difference between "highly" and "very" might be, but of course the real problem was that in truth, the two reviewers felt the work was in fact "not significant at all". But there wasn't a ticky-box for that opinion.
Furthermore, both reviewers felt that it would be a bad idea to tick such a box in any case, because that would imply that the area of study itself was insignificant, and as it was their own area they didn't want to say that. So in the end they went for "very". Even though this was an inaccurate representation of what they thought.
That is why peer review as an evaluative practice may, in fact, simply suck.