NRLgate -
Plagiarism by Peer Reviewers

Sections 2 thru 2.7

This page is part of the NRLgate Web site presenting evidence of plagiarism among scientific peer reviewers involving 9 different peer review documents of 4 different journal and conference papers in the fields of evolutionary computation and machine learning.

This page contains sections 2 through 2.7 of "Evidence of plagiarism in reviews A and B of a paper on empirical discovery submitted to the Machine Learning Conference."


Go to top of NRLgate Home Page
Go to Abbreviated Table of Contents
Go to Detailed Table of Contents
Go to Complaint Letter to the Evolutionary Computation journal
Go to top of Previous Page
Go to top of Next Page

2. Evidence of plagiarism in reviews A and B of a paper on empirical discovery submitted to the Machine Learning Conference

This section presents numerous pieces of evidence indicating the need for an impartial investigation and determination of whether there was plagiarism among the 2 scientific peer reviewers who reviewed the paper that I submitted to the Seventh International Machine Learning Conference (MLC) on the subject of applying genetic programming to empirical discovery and concept formation (Paper No. 151).

As the reader proceeds, two alternatives will repeatedly present themselves: The initial assumption is that scientific peer reviewers are ethical, that the peer reviewers carefully will read a submitted paper that is placed into their hands by a conference or journal, that the peer reviewers understand that the outcome of the peer review process can have an important effect on the career of the author of the submitted paper, and that the peer reviewers will act independently in rendering their scientific judgment of the paper. As will be seen, this initial assumption of ethical behavior will become increasingly difficult to sustain in light of the evidence appearing on the face of the peer review documents.

It is not for me to argue with the scientific peer reviewers as to whether they are right or wrong in the judgments expressed in their reviews. It is not for me to argue about where they choose to locate their comments within their paper review form, what verbs they choose, what punctuation they use, or how they spell. However, I will have a lot to say about the extraordinary coincidences that repeatedly occur inside the boundaries of these very short review documents. In particular, there are numerous instances where two peer reviewers express the same thoughts using the same words, the same phrases, the same punctuation, the same sentence structure, the same punctuation errors, the same spelling errors, the same ordering of thoughts, the same demonstrably inaccurate statements, and in the same parts of their paper review forms.

Collusion can occur in many ways, including a conversation in the lunchroom, a telephone conversation, or an actual exchange of an entirely written review document from one reviewer to another (via paper, computer screen, or e-mail). As will be seen in the case of this MLC paper (as well as the other conference and journal papers herein), one reviewer had the actual already written text of one review in front of him when he constructed his review. However, the plagiarism here did not involve "cutting and pasting" entire sentences of one review in order to construct a second review. Instead, the second reviewer used the first already written review as a template for thoughts, choice of words, punctuation, grammar, grammatical errors, and placement of items within his paper review form. In each case, the second reviewer reworded and retyped the first review in order to make it look different from the first review. That is, conscious effort was expended (albeit somewhat mechanical effort) to disguise the plagiarism.

The next 16 sections detail numerous similarities between the short 218-word review A and the short 138-word review B that I received for the submitted MLC paper on empirical discovery. (The full text of reviews A and B are found in the sections below).

2.1. Both reviewers A and B put quotation marks around the same insignificant and unmemorable 3-word prepositional phrase "In one run, ..."

There are hundreds of different things that a reviewer might say about a 12-page 3,118-word paper and there are thousands of different ways to express those ideas.

Quotation marks are ordinarily used to highlight words that are so memorable and important that only the author's exact words do justice to the idea involved. It is rare to use quotation marks to highlight a mere prepositional phrase --- particularly one as insignificant and unmemorable as the one quoted below.
NOTE: Because quotation marks, ellipsis, and other punctuation marks play an important role in analyzing these documents, direct quotations are presented in a distinctive type font (illustrated below), without adding quotation marks.
Review A said, in the section 3 of the paper review form for the submitted MLC paper on empirical discovery,
It is not sufficient to say "In one run, ...".

(Quotation marks and ellipsis in the original).
(Emphasis added).
Reviewer B's entire response to section 3 on the paper review form for this same paper,
The paper needs to be strengthened by presenting more formally than "on one run ... "

(Quotation marks and ellipsis in the original).
(Emphasis added).
Isn't it improbable that two reviewers would have independently decided that this insignificant 3-word prepositional phrase from a 3,118-word paper was worthy of direct quotation?

There is simply no reason for a direct quotation of these 3 words.

Isn't this joint action particularly improbable given that the total size of review A is only 218 words and review B is only 138 words?
NOTE: Although it is not important in this section at this time, reviewer B made a typographical error in quoting this 3-word prepositional phrase. The words "one run" appeared only once in the submitted paper and they appeared as part of the phrase "In one run". The words "on one run" did not appear in this submitted paper. Reviewer A quoted the 3 words accurately, but reviewer B did not. As will be seen, a 31-word review (T2) of an entirely different paper on an entirely different subject submitted to a different scientific conference later in the same year (the TAI conference) also contained "on one run ... " (with both the quotes and with ellipsis). However, the phrase "on one run" did not appear anywhere in the paper submitted to the TAI conference! This error by reviewer B (and by reviewer T2) will be discussed later because it suggests the time sequence of the plagiarism by reviewers A and B at the MLC conference. It also points to the identity of the plagiarizers (since only 2 persons were reviewers for both the MLC and TAI conferences and since these same 2 persons are the only persons from the MLC and TAI conferences who are also among the editors and editorial board of the Evolutionary Computation journal). It, of course, also indicates that TAI reviewer T2 didn't read my submitted TAI paper, but instead extracted the erroneous quotation "on one run ... " (with both the quotes and with ellipsis) from a previously written review of my earlier MLC paper (which was on an entirely different topic). See section 5 through section 5.2.

2.2. Both reviewers A and B made the same punctuation error in adding ellipsis in their quotation of "In one run, ..."

Review A said,
It is not sufficient to say "In one run, ...".

(Quotation marks and ellipsis in the original).
(Emphasis added).
Review B said,
The paper needs to be strengthened by presenting more formally than "on one run ... "

(Quotation marks and ellipsis in the original).
(Emphasis added).
The purpose of the ellipsis is to alert the reader to the possible deletion of words (typically inside a running quotation) that would otherwise not be self-evident from the face of the quotation. Since a prepositional phrase is always a fragment of a sentence (hence there is no possible uncertainty about the existence of other words in the sentence), since nothing was deleted inside this 3-word prepositional phrase, and since the quotation was so short, the use of the ellipsis here is not grammatically correct.

Isn't it improbable that two independently-acting reviewers --- without the suggestive power of somebody else's already written review in front of them --- would both make the grammatical error of incorrectly adding the ellipsis to this 3-word quotation?

2.3. Both reviewers A and B converged on the same section of the paper review form to locate their quotation of "In one run, ..."

Why did both reviewers A and B happen to put their complaint about this unmemorable 3-word prepositional phrase in the "Accuracy" section of the 5-part paper review form?

What does this 3-word prepositional phrase have to do with "Accuracy"?

Reviewers A and B could, with greater logic, have decided to place their joint thought in at least 3 other sections of the 5-part paper review form, such as the The MLC paper review form (like most) contains open-ended questions. Thus, a particular comment can typically be located in numerous different places within the paper review form.

Isn't it improbable two independently-acting reviewers would converge on the "Accuracy" section (a particularly inappropriate choice) of the paper review form?

2.4. Both reviewers A and B devoted significant fractions of their reviews to a complaint about a 3-word reference to an operational detail (the Ethernet)

There are hundreds of different things that a reviewer might say about a 12-page 3,118-word paper whose intellectual content was concerned with automatically building mathematical models from raw economic (and astronomical) data and about creating decision trees using genetic programming.
Reviewer A's entire response to section 4 of the paper review form is as follows:
The presentation suffers from an abundance of irrelevant details about where the author got his economic data and how he transferred to the Explorer by Ethernet (who cares?). This space could be better spent showing more data.
(Emphasis added).
Reviewer B began, in section 4,
The paper is annoyingly spotty. In the middle of a technical discussion, the author tells us how many lines of Lisp code his program is, and how he used a Mac II to pull a data set over the Ethernet!! This is not what he should be spending his precious 12 pages on.
(Emphasis added).
Isn't it improbable that two independent-acting reviewers would both zero-in on a 3-word reference to the Ethernet in a 12-page 3,118-word paper?

Review A and B were both very short. There were only 218 words (16 sentences) in review A and only 138 words (8 sentences) in review B.

Isn't it improbable that reviewer B would devote 39% of the words in his 138-word review and reviewer A would devote 17% of the words in his 218-word review to this 3-word reference to Ethernet?

There was, in fact, only one 58-word sentence in the submitted 3,118-word paper covering the paper's operational details.
The time series data used here were obtained from the CITIBASE data base of machine-readable econometric time series (Citibank 1989), accessed by an Apple Macintosh II computer using software provided by VAR Econometrics Inc. (Doan 1989), moved into the microExplorerTM half of the computer, and then transferred to the Texas Instruments ExplorerTM II+ computer using an Ethernet.
(Emphasis added).
Of course, scientific papers are ordinarily required to provide both references and operational details to enable other researchers to replicate the reported work (especially concerning potential operational pitfalls). The last half of this 58-word sentence described a necessary "work around" "trick" for transferring economic data from a conventional CD-ROM into the file system of the non-conventional LISP computer (the ExplorerTM II+) involving using a certain expansion board (the microExplorerTM) inside the Macintosh computer (which could easily handle the CD-ROM) to create a bridge from the Macintosh file system to the non-conventional file system of the ExplorerTM II+ LISP machine.

It should be remembered that a complaint about plagiarism is very different than a complaint about bad judgment or even bias. In a case about plagiarism, the central issue is the circumstances of creation of the documents --- not the judgments (or bias) contained in the documents. Looking back on this submitted paper, I acknowledge that the extra 3-words "using an Ethernet" were an unnecessary detail in my 58-word sentence on operational details in my 3,118-word paper about automatic machine learning.

If the phrase "using an Ethernet" were not in bold-faced font in the above quotation from my paper, would this 3-word phrase be the most salient point of this 58-word paragraph? And, isn't the joint action by the 2 reviewers in zeroing-in on these 3 words particularly improbable given that the total size of review A is only 218 words and review B is only 138 words?

2.5. Both reviewers A and B couched their complaint about the offending 3-word reference to the Ethernet in a 3,118-word paper as a matter of "spending space"

The semantically straight-forward way for reviewer A and B to express their joint complaint would have been to say that it was However, both reviewers A and B couched their complaint about the offending 3-word reference to the Ethernet as matter of "spending space." It is, of course, preposterous to claim that the offending 3-word reference to the Ethernet (in its 58-word sentence) constituted a significant amount of space in a 3,118-word paper.

Isn't it improbable that the two reviewers --- acting without the suggestive power of phraseology originating from somebody else's already written review --- would independently converge on a characterization of the problem posed by the offending 3 words as a matter of "spending space"?

2.6. Both reviewers A and B converged on the same verb in complaining about the matter of "spending space" on the 3-word reference to the Ethernet

There are many different choices of words that can be used to express the same idea --- even a preposterous idea.

Why did both reviewers A and B jointly employ the same verb ("spend") instead of, say,

2.7. Both reviewers A and B converged on the same section of the paper review form to place their complaint about "spending space" on the 3-word reference to the Ethernet

Why did both reviewers A and B converge on section 4 of the paper review form to locate their shared complaint about the 3-word reference to the Ethernet?

This complaint could, with equal logic, have appeared in at least 3 other sections of the 5-part paper review form.

Author: John R. Koza
E-Mail: NRLgate@cris.com
Go to top of NRlgate Home Page
Go to Abbreviated Table of Contents
Go to Detailed Table of Contents
Go to Complaint Letter to the Evolutionary Computation journal
Go to top of Previous Page
Go to top of Next Page