Archive for March, 2012

In the last month I wrote/edited 24 author responses to EC and AAAI reviews, and read 18 responses as an EC PC member and 45 responses as a AAAI SPC member, for a grand total of  87 author responses (counting three responses per paper). So you can imagine that author responses have been on my mind.

Author responses are a proud tradition in AI conferences. The theory side of the EC community seems to dislike them, and the weird result is that EC has author responses only on even years (when there is an AI PC co-chair). Debating the advantages and (nonexistent) disadvantages of author responses is a fun and popular pastime, but right now I want to do something a bit different: discuss their strategic aspects.

If you have two negative reviews then you can start studying the call for papers of your favorite second-tier conference, so let’s focus on the case where you have two positive reviews and a single negative review. The correct response of course depends on how negative the latter review is.

Case 1: the negative review is just below borderline (overall score of 6 in EC or 5 in AAAI/IJCAI). The goal is to gently nudge the negative reviewer upwards with high probability. Here I am pleased to announce that, after years of trial and error, I have identified the perfect strategy. The first step is to convince yourself (and your coauthors) that the reviewer is supremely wise. The second step is to respond by agreeing (honestly, see step 1) with everything the reviewer says and committing to a revision that addresses the reviewer’s main concern. A subtle obstacle is that conference papers should be accepted roughly as-is, hence this strategy requires a delicate balancing act between promising a revision and making the case that the revision is minor. The trick is to clearly point out exactly which paragraphs you plan to revise.

Case 2: the negative review is way below borderline. Ah, the twilight zone of author responses; here there are no perfect strategies, it’s just about playing the odds. Since we are assuming that your paper is awesome, this case implies a reviewer who completely misunderstands the paper, or an outlandish review. In AAAI/IJCAI, for example, you occasionally get the random PC member who worked on automated theorem proving in the Eighties and checked the game theory keyword because he really loves Super Mario Bros.

The first strategy is to politely point out to the reviewer why he/she totally misunderstands your paper. From my experience, this has a reasonably good chance of bumping the reviewer’s score upwards by two points. Strangely enough, two also seems to be an upper bound on the possible benefit; even after realizing the error of their ways reviewers never seem to switch from negative to positive, they get stuck at borderline. Nevertheless, this common strategy seems to work well if the reviewer’s initial overall score is 4.

The second (more desperate) strategy is to share your honest thoughts about the dubious quality of the negative review; the goal of this gambit is to get an additional review, ideally as a replacement. One possibility is to do this via a side channel to the PC chairs, but that seems a bit sneaky. I have only attempted the direct approach (via the author response) once. The outcome is best described using a useful Hebrew Idiom: “the operation succeeded but the patient died” (an additional review was given but the paper was rejected). Next time I think I’ll try the sneaky approach.

Finally, one EC SPC did not appreciate my admittedly blunt attempt to point out a perceived discrepancy between reviewers’ verbal recommendations and overall scores. In a AAAI response, my coauthor John Lai took a more diplomatic approach for the same issue: “We noticed a possible discrepancy between the verbal recommendation (…) and overall score (…) and wanted to check that this was intended.” It occurred to me later that a slight variation of this statement could provide a foolproof author response to any negative review: “We noticed a possible discrepancy between the review and the quality of our paper.” Do let me know if it works.

Read Full Post »

MIT will host the Collective Intelligence 2012 conference on April 18-20, 2012. The conference includes (but is not limited to) some of our favorite topics, e.g., social networks and crowdsourcing. The stellar list of invited speakers features familiar names such as Yiling Chen and Panos Ipeirotis, as well as quite a few non-AGT-yet-famously-eloquent speakers such as Jonathan Zittrain and Yochai Benkler.

See the conference website for more details.

Read Full Post »

EC 2012 workshops

The workshops for the EC 2012 conference have been announced:

Thursday, June 7th, 2012:

Friday, June 8th, 2012:

Read Full Post »

Postdocs post

Every so often I post announcements regarding postdoc positions.  Let me try to do the opposite:  urge faculty members and departments who are offering postdoc positions related to economics and computation / algorithmic game theory to post them as comments to this post.

Read Full Post »

The blogosphere has been abuzz lately with news of the boycott against Elsevier and its unexpectedly quick impact. Indeed, the fact that the Research Works Act was shelved is a triumph for the forces of good.

It’s hard to not to catch the revolutionary fervor, but I do want to raise several points. The boycott is led by a stellar group of mathematicians who, generally speaking, individually satisfy at least one of the two following properties:

  1. Being Tenured. Or winning the Fields Medal. Typically both.
  2. Working in fields where Elsevier journals are not considered essential. For example, the statement of purpose of the “Cost of Knowledge” initiative mentions several top mathematics journals, and says that “none of Elsevier’s mathematics journals is generally considered comparable in quality to these journals.” A similar situation exists in theoretical computer science, where as far as I know the top journal among the ones published by Elsevier is the Journal of Computer and System Sciences (JCSS).

In contrast, here is my point of view:

  1. I am untenured. Very much so. In fact, when I joined CMU half a year ago I received a formal letter from the department that so far I have 0 years of service, and therefore my tenure decision will be made in 2020. Rub it in, why don’t you?
  2. I work in fields where Elsevier journals are held in high regard. The journal Artificial Intelligence (AIJ), published since 1970, is generally considered to be the most important journal in AI (but see below regarding JAIR). In economics, Games and Economic Behavior (GEB) is the premier game theory journal, and historically has played a crucial role in facilitating the interaction between computer science and economics.

So an obvious conclusion from 1 and 2 is that the professional cost varies across different people, but this is a minor issue — point 1 is just my self-deprecating humor at work. The point I want to focus on is this: it is difficult, perhaps even morally “wrong”, to boycott Elsevier if you work in a field that has been significantly advanced by journals that happen to be published by Elsevier. It’s not only the historical aspect, it’s also the many great people who have invested their time and effort in these journals.

Therefore, I think the current initiative, by its nature, would have to exclude many fields. So what should we do? AI is an interesting case study. AIJ’s main competitor is the Journal of Artificial Intelligence Research (JAIR), published since 1993, which nowadays is comparable in prestige although perhaps still a notch below AIJ. Notably, JAIR is an open access journal. The rise of JAIR has inevitably made its mark on AIJ, which since 2008 has had a rather impressive open access policy. First, all articles become free after four years. Second, anyone can gain immediate access to all articles by signing up (for free) as an “IJCAI associate”. It seems though that someone (AAAI?) is picking up (some?) of the tab for the latter policy.

I think everyone agrees that open access will be the norm soon; the only question is how soon (two years? five? ten?), and whether it would happen “naturally” through new open access journals and initiatives. The zeitgeist is Facebook’s motto of “move fast and break things”. An interesting recent article in the New York Times contrasted this approach with Bell Labs’, whose motto might have been “move deliberately and build things”. You get the point. (OK, this paragraph reads like I’m 90.)

One example of the constructive approach (which no doubt was brought about by the boycott) is the Federal Research Public Access Act, which “proposes to make manuscripts reporting on federally funded research publicly available within six months of publication in a journal.”  A petition supporting the bill is available. Apparently if the petition gets 25,000 signatures by March 10, it will be reviewed by President Obama. After I signed it on Saturday it had 799 signatures, and now it has 898, so there is a chance of reaching the astronomical number of ONE THOUSAND signatures. Hold on… Bummer.

Read Full Post »

As the field of algorithmic economics has matured, various conferences, workshops, and symposia have sprung up to accommodate the burgeoning interest. Only one type of meeting known to science is still missing: a summer school.

Fortunately, this oversight is about to be corrected. Tim Roughgarden and I are organizing a summer school on algorithmic economics, to take place at Carnegie Mellon University on August 6-10, 2012. We are targeting current or soon-to-be graduate students (computer scientists and social scientists). We will select roughly 50 students to attend the summer school. All US-based students who are selected will receive full financial support; students based outside the US may apply provided they have their own funding. The application deadline is April 15.

For details about the summer school, including the list of confirmed speakers and the specifics of the (almost painless) application process, please see the summer school’s website.

UPDATE: It turns out that we are simply continuing a proud tradition of summer schools that started in ancient times with this one and continued with this one.

Read Full Post »