"Good peer-review comments are worth their weight in gold."
@JeremiahCoogan
"I always try to follow a simple rule: write reviews that you would not be disappointed to receive as an author. I agree that negative ones make no one happy but, if they contain well-justified comments, they may be valuable for improving the work!"
@massi_manc
"By peer review, I just mean constructive comments to improve the validity and impact of a given work/report."
@Raamana_
Frequently Asked Questions (FAQ)
General
What is a Flying Scholar?
A Flying Scholar is someone who enjoys peer-reviewing, and who believes that reviews benefit the community most when they are widely read, shared and discussed. The mission of Flying Scholars – CS is to provide a timely and rigorous review for each and every paper for which it receives a call for review, and to share the reviews widely with the community.
Who are the Flying Scholars?
Anyone can become a Flying Scholar by volunteering to review a paper of their choice.
What papers can be reviewed by Flying Scholars?
Any paper can be reviewed, whether it is a preprint or has been published in a journal or conference.
How to call for a paper to be reviewed?
We are currently operating on twitter. To call for a review, simply reply to the tweet of a paper of interest with a call to @FlyScholarsCS. Please suggest potential reviewers by mentionning their Twitter handle.
Example: "This paper looks neat! I'd like to know what the experts on this topic think about this work, @FlyScholarsCS. I believe @ylecun, @ybengio, @ghinton and/or @jschmidhuber would be expert on this topic, but I might be wrong."
Where are reviews published?
Reviews are posted in reply to the main tweets referencing the paper, using the pastebin service https://rentry.co to accomodate the long format of a review. The reviews are also posted on the twitter account of @FlyScholarsCS and on any relevant subreddit (e.g. r/MachineLearning).
Who are the founders of Flying Scholars?
The founders prefer to remain anonymous so that the service can operate independently.
Reviewers
Are there prerequisites to become a Flying Scholar Reviewer?
Our service is open to all, and reviews are published based on their merits only.
How to get started as a Flying Scholar?
- First, identify a paper you would like to review. You can browse calls for reviews on the @FlyScholarsCS account or select a paper yourself.
- Write your review on https://rentry.co, with the URL of your choice. Check out our template here: https://rentry.co/review_template.
- Send the URL of your review by private message to @FlyScholarsCS.
- The moderators of @FlyScholarsCS will publish your review soon!
Why review with Flying Scholars – CS instead of reviewing solo?
- We connect reviewers with papers that the community called to be reviewed.
- By collecting many peer reviews on a single account, Flying Scholars – CS gives visibility to your review, and preserves your anonymity (if you choose to remain anonymous).
- If you are inexperienced and unsure of the quality of your review, our moderators can provide you with feedback (on demand).
I prefer to use my own twitter account to publish my reviews. Can I still benefit from this service?
Yes. Simply call our handle @FlyScholarsCS under your review and we will happily retweet it.
Why review at all? Seems like a waste of time.
There are many reasons to participate in peer review. Here are a few:
- Peer reviewing is fun!
- Peer reviewing is a gift to the community.
- Peer reviewing helps writing better papers, by identifying flaws and holes in the work of others.
- A reviewer can gain a reputation as a wise critic and valuable contributor to the community, by attaching their @twitter_handle to their reviews.
Are there guidelines to review on Flying Scholars – CS?
A template is provided to guide reviewers: https://rentry.co/review_template
Reviewers are free to adapt the template to their preferences.
Discourteous reviews will not be published by the moderators of Flying Scholars – CS.
Are reviewers anonymous?
Reviewers can choose to be either anonymous or identified by their twitter handle (see template https://rentry.co/review_template).
As a reviewer, can I remain anonymous to the moderators of Flying Scholars – CS?
Yes. If you wish to remain anonymous to the moderators of Flying Scholars – CS, simply use an anonymous twitter account ("burner account") to message @FlyScholarsCS.
Can I edit my review after publication?
Yes, the service https://rentry.co allows you to edit your review, as long as you remember the edit password that you provided to https://rentry.co.
How to choose a paper to review?
Choose a paper that you like something about, or that you dislike something about. Make sure to have a relevant expertise. Avoid papers for which you might have a conflict of interest (authors are friends or foes, or work in the same lab). It can be particularly useful to review papers that are under-appreciated (you believe they should be read and discussed more).
Authors
How can I get my paper reviewed by Flying Scholars – CS?
Call the handle @FlyScholarsCS in response to the paper tweet. Please suggest potential reviewers by adding their handle to the tweet.
Example: "We would love to know what the community thinks of our new work. I believe @tgebru, @mmitchell, and/or @ebender would be terrific reviewers for this work. @FlyScholarsCS"
Can I respond to a critical review of my paper?
Yes. You can either respond directly to the Review tweet (using for instance https://rentry.co for a long format), or you can send us your response by private message, and we will publish it in reply to the review.
Why would I use this service as an author?
Some authors value the feedback from community experts on their work. In addition, reviews can boost the visibility of a paper, as the community is generally interested in papers which are actively discussed.
Moderators
What is the role of a moderator at Flying Scholars – CS?
Find reviewers for papers called to be reviewed and filtering disrespectful or irrelevant reviews.
Who are the moderators of Flying Scholars – CS?
For now, there is just one anonymous moderator. A public call for moderators will be made as the service grows.
If you don't find an answer to your question, send us a private message at @FlyScholarsCS!