Is the conference-publication “system” serving us well today? Before we try to fix the conference publication system, we must determine whether it is worth fixing.
While I agree with the view that the conference system in CS is cracking and needs an overhaul or replacement, I feel somewhat uncomfortable with the undertones of the question. Certainly no one is opposed to conferences per-se. It seems that the real issue is “how much should we value conference publications relative to journal publications”, with an implication that this will be used in hiring or promotion decisions. Indeed Moshe points specifically to the following decision:
in 1999, the Computing Research Association published a Best Practices Memo, titled “Evaluating Computer Scientists and Engineers for Promotion and Tenure,” that legitimized conference publication as the primary means of publication in computer research.
In my view, the issue of evaluation of candidates is not the correct starting point for thinking about our conferences. Top conferences in CS started gaining higher prestige than journals simply because the average scientific quality there was better (i.e. the CRA decision was the result, not the cause, of the higher prestige of conferences). I have no doubt that we see questions about conference publications arising now simply since publication in many conferences is no longer a signal of high quality. This is being recognized by the community and will certainly be taken into account by future promotion and hiring committees. Instead our question should be how do we best use conferences (and journals, and the Internet, and anything else) to advance our field. Conferences should provide the best service that they can, and so should journals. The community will certainly come to appreciate the venues that offer the highest signal of quality, be it a conference or a journal.
I seems to me that many of the changes in CS conferences are not really driven by an attempt to advance computer science, but more by attempts t0 advance computer scientists. Oded Goldreich puts it bluntly:
My impression is that FOCS and STOC do not function any more as forums devoted to the presentation and exchange of ideas (but rather function as “weight-lifting competitions”)
I’m not sure that I agree with Oded’s point of view regarding FOCS or STOC, but the situation is certainly worse in other conferences.
Personally, I do have some preliminary ideas about what kind of changes I would like to see in the conference system: fewer conferences with proceedings, smaller PCs (unlike IJCAI’09), less accepted papers, and in general more “less”. I also don’t see any reason to limit in any way the overlap between a journal publication and a confernce publication. If needed, the EC system of allowing a confernce submission not to be publihsed in the proceedings as to allow future publication in economics journals seems good. The point of this post though is not to put forward these suggestions but rather to steer the debate away from focusing on evaluation of candidates.