In my previous post I tried to spell out the problems with the current academic journal publishing system, and pointed out to Tim Gowers’ post that suggested how it can be replaced by a combination of arXiv and a math-overflow-like online commenting&reputation system. Many comments to Tim’s post (as well as to mine, my G+, Lance’s, or Gil’s) raised objections to such a system. In many cases, I feel that the objections were to using a web-based system as things are today without incorporating the critical positive features of journals. Clearly any alternative system would have to incorporate these positive features in some way, preferably in a better way than journals do them. In this post I would like to explicitly point out the positive features that need to be duplicated by any system that seeks to replace the journal system.
That’s where the people are
The most important feature of the journal system is that leading researchers actually publish there, referee papers there, and hire and promote others according to their journal publications. Any alternative system would be a non-starter without a strategy of how to move researchers into it. I think that this is doable:. To start you need two elements: an organizer that can set up the proposed system, run it, and advance it, and a group of high-profile researchers that are willing to lend their name to the effort by serving as its “board” or so. It is important to give value immediately along-side the existing journal system. At first only a few researchers will actively use the new system, either for the added value or due to their interest in a new thing, or as a deliberate contribution to the effort. As more people join, the value goes up, more people start using it, and “reputation” earned on the site can become a minor added value in hiring decisions (comparable to the “bonus points” one may get for being a good expositor) maybe via letters people write. Here we hope that a positive feedback loop emerges gradually pulling all the community in. This can happen: see how CS switched to a conference culture from its previous journal one.
Peers must review peers’ work
The main service that journals still provide is to have someone read your paper, check it, point out possible mistakes, suggestions for improvement, and even typos. This is indispensable: someone qualified needs to spend the time doing so. However, the idea that these someones are arbitrarily chosen by an editor and then provide anonymous feedback bundled with a reject/accept decision (and in case of reject, everything is repeated again with another journal) is not ordained from heaven. Any web-based system must provide the mechanisms by which peers read each others’ work, check it, and suggest improvements. While setting the incentive structure right for this is not trivial, I believe that it is totally solvable, especially compared to the non-existent incentives for refereeing well for journals. There are just so many ways by which it possible to provide more useful information to authors, save duplication of review effort, reward refereeing, and reduce arbitrariness.
Journals get their authority from their editorial board that should be compose of well-respected researchers (although the legal journals offer an intriguing alternative where the best law journal are student-edited). When one thinks of web-based reputation systems, one shudders at the thought of papers being evaluated by popular vote rather than by trusted experts. This is another critical element that every web-based system replacing journals must include: any “decisions”, rankings, or scores must take into account the identity of the recommender/voter/referee/commenter, giving more weight to those with higher reputation. This includes implicit rankings like the order and prominence that papers or comments are presented to the user. Good systems will do this in a flexible way where each user may choose his preferred reputation metrics and be presented information ranked according to his taste. This choice may be implicit e.g. choosing the ranking system suggested by virtual-journal X. Math-overflow-type systems have gone a step in this direction, but we will likely need more sophisticated systems, maybe along the lines of page-rank or personalize-page-rank. The bar set by the journal system is not very high.