Changing review assignments mid-review
During the review, you may have to assign/re-assign reviewers to submissions. This may be because they cannot complete their workload or because your deadlines are approaching and you need reviews completed quickly. In re-assigning reviewers, emails will be sent to the reviewer who has been removed and the reviewer who has been assigned.
Step 1) Browse Submissions
Click on Browse Reviews to see the submissions being reviewed:
Step 2) Find the reviewer to remove or the reviewer place to assign
First, find the submission in the list and then click on the reviewer name or No Reviewer slot on the submission:
Step 3) Choose the reviewer
You will be presented with a screen that will ask you, "Who should review this paper?"
You will have 4 options to choose from:
- Recommended Reviewers: Choose one of the "recommended reviewers" as determined by Ex Ordo's statistical analysis of the underlying submissions and available reviewers.
- Search by Name or Email: Browse or search the list of available reviewers available to you.
- Invite a Reviewer: Invite an external reviewer that is not currently available in the list of reviewers. This is useful if you would like to invite a colleague that has not yet been invited to be a reviewer for this conference.
- No Reviewer: For whatever reason, you may decide to leave one of the review slots vacant. The submission will then receive one less review than the other submissions in your list.
a) Choose a recommended reviewer
Based on our statistical analysis of the underlying submission, the system will present a list of 3 "recommended reviewers". It should be stressed that our recommendation engine does not take into account a reviewer's intellectual ability, reputation or academic qualifications. Instead, it bases its recommendations on the statistical analysis of a paper vs. the available reviewers.
You are guaranteed that a recommended reviewer; will be a reviewer with the highest relevance to the paper, will not be overburdened with review assignments and will not have an identified conflict of interest with the submission.
If you are not satisfied with the reviewers recommended, you should choose the Search by Name or Email option instead.
b) Search by name or email
You can choose to search by name/email or browse the list of available reviews. If you select this option, you will be presented with a screen that looks something like this:
Search
In the top right hand corner, you can use the search engine to search through the list of available reviewers. Using this tool, you can enter the reviewer's first name, last name, organisation or email address. The system will perform a search for you in real-time and present the results as above.
List of Reviewers
You can also choose which list of reviewers you want to search/browse using the drop down option in the right hand corner of the screen. You can choose to search by all reviewers, recommended reviewers, related reviewers or conflicted reviewers.
Star Rating
Each reviewer is given a "star rating" based in his/her suitability for the given submission. This rating is based on the number of topics shared between the reviewer and the submission. A reviewer with 3 stars, has the maximum number of common topics shared with the submission. A reviewer with no stars has no topics shared with this submission. The star rating does not take a reviewer's intellectual ability, reputation or academic qualifications into account.
Workload
Each reviewer also has a workload rating, which represents the number of submissions assigned to each reviewer.
c) Invite a new reviewer
If cannot find a suitable reviewer using the Recommended Reviewers or the Search option, you can invite a new reviewer using the following screen:
Simply enter the reviewer's email address, first name and last name into the box above. Once you click on the Invite Reviewer button, the system will send the reviewer an automatic email with links that they can use to setup their account, login and access their newly assigned review.
d) No Reviewer
You can choose to assign No Reviewer for a given review slot. This might be useful towards the end of the conference review process when you can remove non-performing reviewers from the review panel for a given submission. Simply click on their name, choose No Reviewer and the system will un-assign the reviewer from the submission.
Step 4) Complete
Depending on what route you take, you will ultimately end up assigning a reviewer to the submission.
Changing review assignments mid review will send two automated emails. One to a reviewer that has been Unassigned from the submission to let them know they have been removed as a reviewer and one automated email to a Newly assigned reviewer to let them know they have a new submission to review.