Area Chair Guidelines

Thank you for agreeing to serve on the program committee of COLM 2025! As an area chair (AC), your job is to ensure that all the submissions you are assigned have high quality reviews and good discussions. You should become familiar with the technical contents of all your submissions and are responsible for making the initial acceptance decisions.

If you encounter a situation that you are unable to resolve on your own:

Please consult the important dates page for the expected timeline for the tasks below. We will communicate each task as the process progresses.

Main Tasks

  1. Preparation:
    • Please ensure that your preferred email address is accurate in your OpenReview profile. We will send most emails from OpenReview (noreply@openreview.net). Such emails are sometimes accidentally marked as spam (and sometimes as updates in Gmail). Please check your spam folder regularly. If you find such an email in there, please whitelist the OpenReview email address so that you will receive future emails from OpenReview.
    • Please log into OpenReview and make sure that your profile is up to date.
    • Review COLM code of conduct.
    • Read what constitutes conflict of interest for COLM 2025.
    • In addition to the guidelines below, please familiarize yourself with the reviewer responsibilities and guidelines. You will be interacting significantly with reviewers, so please make sure you understand what is expected of them.
  2. Paper assignments:
    • We do not have AC bidding. Instead, we will use the matching algorithm and assign the papers automatically. While it is possible that this may result in less than optimal assignments, we rely on your experience and insight in handling papers from a variety of topics rather than on your specific topic expertise. Please bear in mind that we are handling an extra large number of submissions, so your flexibility is greatly appreciated.
  3. Modify reviewer assignment and add reviewers if necessary.
    • Make sure that every submission in your batch is matched with suitable reviewers whom you can trust on this submission, ideally with a diverse set of opinions.
    • It really pays off to invest some time before the reviewing process starts to ensure that your batch has expert reviewers!
    • You can continue inviting reviewers as needed after the assignment is released to reviewers. Note that because the review process would already be underway, reviewers will need to agree to review each additional paper before they can be assigned. You can follow these instructions for sending invitations to review a specific paper.
  4. Ensure all papers have quality reviews:
    • You might have to send multiple reminder emails as the review deadlines nears, but please be courteous. If a reviewer is unable to deliver a review, please find a replacement reviewer.
    • Read all reviews carefully. If a review is substandard, you should ask the reviewer to improve their review. Please remember to be polite and provide concrete guidance.
    • You may also need to get an additional review if the current reviews are particularly unhelpful, either in the existing pool or by inviting external reviewers.
    • Make sure that any questionable papers are flagged for ethics review. These papers will be assigned ethics reviewers, who will effectively join the paper's assigned program committee.
  5. Discuss with reviewers and authors.
    • As soon as the author response is entered in the system, initiate and lead a discussion via OpenReview for each submission, and make sure the reviewers engage in the discussion phase.
    • Monitor and moderate the discussion to ensure that it is respectful of everyone’s opinion. Read the submissions in your stack to steer the discussion towards the most critical aspects that need discussion.
    • Make sure your reviewers read and respond to all author responses. To minimize the chance of misunderstandings during the reviewing process, you can also initiate a rolling discussion with the authors after initial reviews and author responses are submitted. You can restrict visibility of your comment to any set of readers (authors, reviewers, senior area chair, or program chairs) as appropriate.
  6. Writing the metareviews: by Friday, Aug 30
    • This is considered the last phase of the discussions, and you will be writing the metareviews. You can elicit further comments and clarifications from the reviewers. Please help us by being on time with your metareviews, as we have only a couple of weeks before the notifications are sent to the authors.
    • Write a meta-review that explains your decision to the authors. Your comments should augment the reviews, and explain how the reviews, author response, and discussion were used to arrive at your decision. You may dismiss or ignore a review if it is clearly of low quality. If the reviewers cannot come to a consensus, you should read the paper carefully and write a detailed meta-review.
  7. Discuss accept/reject decisions with PCs
    • Discuss with the PCs your accept/reject recommendations, especially the borderline cases as well as ones where you are making a recommendation that goes against the reviewers.

This document was adapted from the NeurIPS 2024 AC Guidelines. We thank the authors.