|
February 5 |
Submission deadline |
|
February 19 |
Papers assigned to AEs |
|
February 26 |
Deadline for AEs to assign papers to reviewers |
|
February 26 |
Deadline for AEs to summarily reject papers |
|
March 23 |
Deadline for reviewers to submit reviews |
|
April 5 |
Deadline for AE final reports |
|
April 12 |
Deadline for Editor endorsements of AE reports |
|
April 30 |
Executive PC Meeting (ICPRB Editor-in-Chief attends) |
The 2014 IROS Conference Review Board (ICPRB) is organized in the same way as the ICRA CEB. There is one Editor-in-Chief (EiC), approximately 15 Editors (EDs), and more than 200 Associate Editors (AEs). Each paper to be reviewed will be assigned one Editor and to one of the AEs that that Editor supervises. The AE will be responsible for obtaining a minimum of two high quality reviews for each paper they handle, and for preparing an AE recommendation that explains the recommended decision regarding the paper acceptance and presentation type. The AEs will also help to identify papers to be considered for awards. The Editors will be responsible for reviewing and endorsing the work done by the AEs on the papers for which they are responsible.
Assignment of Papers to Editors and AEs. For IROS 2014, the keywords are partitioned into sets, each of which is the responsibility of one Editor. When a paper is submitted to the conference (via PaperPlaza), the author's choice of keywords will be used to determine to which Editor and AE the paper will initially be assigned. This initial assignment, which is done automatically by the system, is then reviewed by the Editor in Chief and the Editors to avoid conflict of interests (COI), to improve the matching of the expertise of the AE with the paper, and to provide load balancing across the AEs. Finally, the AEs must review the papers assigned to them and inform the supervising Editor of any COI they may have with their assigned papers.
AE Tasks and Responsibilities. Once assigned papers to handle and notified that they should get started, the AE should follow the following steps:
Note about Organized Sessions: From the perspective of AEs and Editors, papers submitted to organized sessions will be handled in the same way as any other paper. The decision regarding whether an Organized Session Proposal (ISP) will be accepted, and if so, which papers will be included in it, will be handled by the Senior Program Committee (SPC).
The review process for IROS is managed using the PaperPlaza system. PaperPlaza provides a wide variety of tools to help AEs manage the review process. Reviewer assignments, review entry, AE reporting, and final decisions are all managed using PaperPlaza.
To access the system, go to the PaperPlaza page, click Start and then Log in. If you have forgotten your login information, you can retrieve it using the PIN management page.
It may be useful to spend a few minutes looking over the help pages, and in particular the Associate Editor's FAQ at the PaperPlaza site: PaperPlaza Help Page.
In certain special cases, an AE may recommend that a paper be rejected without sending the paper for review. In such cases, the AE writes a brief summary review giving the rationale for this decision, and gives the paper an Unsatisfactory rating. This is done via the usual AE Report mechanism (described below). The Editor assigned to the paper will then make the final determination as to whether the paper should be rejected without further review, or should go through the formal review process.
A paper should be rejected without review in any of the following cases:
A paper should not be rejected without review merely because it makes only an incremental contribution, because it fails to report real-world experiments or because of poor writing quality. Further, a paper should not be summarily rejected because the AE feels its subject lies outside the scope of IROS (this judgment is left for the Program Committee). If there is any doubt as to the decision, the paper should be sent for review.
No more that 10-15% of submitted papers will be rejected without review.
The ICPRB has a strict review quality policy: every submitted paper must receive at least two substantial reviews. An AE should reject any sub-standard review received - e.g., too short or shallow. It is also the AE's responsibility to see that reviews are constructive and not diminishing the authors' efforts, even when they have to be negative or very negative. Relevant information about the reviewing process is described in the page 2014 IROS Reviewer Information.
Selecting appropriate reviewers is critical in order to have a quality and timely review process. The most important criteria is for the reviewer to have the appropriate expertise. Candidate reviewers may include leading researchers in the topic area, colleagues who are familiar with the topic, authors of papers on the reference list, authors of relevant papers that have been previously presented at IROS or in other high-profile journals and conferences. Generally, a key idea to secure reviewers is to align the interest of reviewers with ours: the paper is on a topic that they would really like to see or to say something about. Having reviewers who are interested in going over the manuscript helps alleviate the need of repeatedly sending reminder messages, and avoids the need for AEs to write detailed comments themselves to supplement shallow and not-to-the-point reviews.
In addition to having appropriate expertise, the following provides some other requirements and guidelines in selecting reviewers.
IMPORTANT NOTICE: Paperplaza has a very nice feature to help you choose your reviewers, which however has a potential for misuse. The feature is that you can get a list of registered users whose keywords match the paper at hand, so that a reviewer can be recruited with a single click. The abuse of this feature is that the AE selects a reviewer whom the AE does not know, and whose expertise is not proven. It may thus happen that junior members of the community (e.g., undergraduate students), or simply people that are associated to a keyword by chance or mistake, are asked a review in a field where they are not competent. This is clearly unacceptable, and ultimately counterproductive in terms of time - as the reviewer will probably decline, or anyway the Editors will discard the review.
Also, please note that you should not add new users to the system unless you are certain that they are not already registered. Multiple PINs for a single user can cause significant difficulty and confusion.
Review requests are handled by the PaperPlaza system. After you have compiled your list of reviewers, a review can be requested by following the Reviews link for a submission, then clicking the Request a review link. You may edit the standard form letter as you please. You must click Send to actually generate the e-mail invitation to the reviewer.
You can find details about this process in the PaperPlaza help pages: Compiling my reviewer list.
Once you have collected all reviews, you should prepare and submit your AE report. You can find details about the report process in the PaperPlaza Help pages available after you have logged in. Please note that we will not be using the rebuttal process.
We would like to have the AE reports have a common structure that:
Here is guidance and reminders on what to put or not to put in your report and in determining your scores:
Request revision of the review. After checking a review that has been submitted this link may be used to request the reviewer to prepare and submit a revised review. The form letter "Request to submit a revised review" is available for this purpose. This is a conference wide form letter that may be individualized on the Personalized form letters page. Revision of a review may be requested if the status of the submission is Under review or Decision pending. When this request is sent to the reviewer the status of the review is reset to 'Saved' so that the reviewer may re-access and resubmit the review. The usual option to save a review is not available to the reviewer for such reviews.
Restoring reviews to original after revision requested. To Restore a review for which revision was requested to the original review click on the link "Revision requested on..." for the review in question. This opens an information box with a link to view the original review and another link to restore the original review.
NOTE: Disposition of unreceived revised reviews. Reviews for which a revision was requested but not received are considered reviews that were requested but not submitted by the system. If they are not restored to the original review they are automatically canceled when the Review summary report is submitted for the submission.
IROS 2014 features a number of awards. The ICPRB is asked to help in the selection of these awards. In particular, the ICPRB will designate a number of outstanding papers from which the Senior Program Committee will draw finalists for Conference Awards. Accordingly, Associate Editors are asked to identify which papers in their assignments they would consider as potential candidates for an award. To do so, please use the "Confidential comments to the Program Committee" textbox in your AE Report form. It is not necessary to identify which award would a paper be a suitable candidate for. The actual selection of finalists for each award will be made by the Senior Program Committee, using your inputs.
The AE's recommendation, expressed in the report, will be reviewed by the supervising Editor, who will be responsible for checking that the quality standards of the review process (including number and depth of reviews, significance of AE's reports avoiding undecisiveness) have been met. The Editor will issue a brief statement for each recommendation, whereby the correctness and completeness of the revieweing procedure is endorsed.
Editors will also overview and endorse the processing of award candidates by the AEs.
An ICPRB Editor, Associate Editor and a ICPRB Reviewer is deemed to have a conflict of interest in a submitted paper if
A. he or she is a (co-)author of the paper; or
B. one (or more) of the authors of the paper:
B.i. is, or has been, a student or advisor of that person, or
B.ii. has co-authored a paper or has closely collaborated in a research project with that person
in the previous five years, or
B.iii. is employed at the same institution (at the Department or Division level) as that person;
C.
there are any other circumstances which may create an appearance that
the person might have a bias in the evaluation of the paper.
Plagiarism cases involve serious accusations, which should be dealt with carefully. IEEE has clear policies to follow. IEEE defines plagiarism as the reuse of someone else's prior ideas, processes, results, or words without explicitly acknowledging the original author and source. It is important for all IEEE authors to recognize that plagiarism in any form, at any level, is unacceptable and is considered a serious breach of professional conduct, with potentially severe ethical and legal consequences (source: Section "8.2 Publication Guidelines" of the IEEE PSPB Operations Manual, "Guidelines for Adjudicating Different Levels of Plagiarism." )
IROS 2014 has access to the CrossCheck database, and initiative to prevent scholarly and professional plagiarism. Every submission will receive a plagiarism similarity score. The score and scan reports are generated by an external provider (iThenticate) and the scan reports are stored on the iThenticate servers and not downloaded to the conference submission system servers. Eventually the reports are deleted from the iThenticate servers at a time determined by conference and provider policy, after which they are no longer available.
IMPORTANT: It is very important to note that it is not possible to draw any conclusion from the iThenticate numerical score alone. Unfortunately, due to the output from iThenticate algorithms, there will be a number of false positives. One issue is that it represents a a cumulative score so that, e.g., a 1% similarity with 40 papers is shown as 40% similarity. Another issue is that there may be large similarity, but still not plagiarism. For example, if an author has a version of their paper as an technical report or in a public dropbox someplace, it might get a very high (e.g., 99% or 100% similarity score). Hence, it is necessary that the detailed report be scanned to make sure that there is indeed a case of plagiarism. Also, some of the papers will not scan properly due to font problems.
The iThenticate reports are available to you as an Associate Editor and to your Editor, but they are NOT available to reviewers. To access the report for a particular paper:
Self-plagiarism. The definition of self-plagiarism is that the paper includes substantial overlap with another of the authors published paper(s) and that the previously published paper is not cited in the references and/or the contribution of the current paper over that other papers is not described in the current submission, both of which are required by IEEE policy.
The AE should be able to determine if self-plagiarism is a concern by reviewing the paper and the plagiarism report. If this is considered to be the case, then the AE should set one of the plagiarism flags and inform their managing Editor. If the Editor agrees, then they will inform the Editor-in-Chief who will also review the paper and the report. If the EiC concurs, then the paper is a candidate for summary rejection and the AE will be asked to prepare a report describing the reason for the summary rejection.
Plagiarism may also be spotted/reported by reviewers (recall,
they don't have access to the iThenticate report).
If a reviewer detects a potential case of plagiarism,
then they should document their concerns in
the confidential comments portion of their review
and should alert their AE.
It is important that the Reviewer is factual in their remarks, and
that as much and detailed evidence is provided as possible.
For instance, this could be a copy of the supposedly plagiarized
paper with the copied parts highlighted. It should also be noted
that there exist freely available software that can detect
plagiarism automatically: if this was used, details on the query and
its outcomes would also be useful.
Based on this information, the AE should use the information
provided by the reviewer in the same fashion in which they
would have if the alert was prompted by the iThenticate report.
| Score | Recommended Action | Interpretation |
| A / 5.0 | Definitely accept | Top 15% of accepted IROS papers, an excellent paper. I advocate and will fight for acceptance. |
| B+ / 4.5 | Accept | A great paper. I will strongly argue for acceptance. |
| B / 4.0 | High Borderline | I am leaning to accept. This paper should be accepted, although I would not be upset if it were rejected. |
| B- / 3.5 | Borderline | I am undecided, I would not be upset if it were accepted or rejected. |
| C / 3.0 | Low Borderline | I am leaning to reject. This paper should be rejected, although I would not be upset if it were accepted. |
| C- / 2.5 | Reject | The paper needs substantial improvements. I will strongly argue for rejection. |
| D / 2.0 | Definitely Reject | The paper is trivial or wrong or known. It is clearly below IROS conference quality, I assume no further discussion is needed. |
| U / 1.0 | Inappropriate | This does not fit the conference or its standards. |
The reviews will help to maintain the quality of IROS. Each review is required to meet the 1,200 non-white character threshold, which has the purpose to provide useful and constructive feedback to the authors. Please refrain from filling the space with meaningless or repeated text. The reviewers should respect the time invested by the authors and should not accept to review unless they are willing to at least provide a meaningful review.
Last modified: Sat Feb 15 14:04:46 CET 2014