Where is it best for an AGT/E researcher to publish his research? How do we evaluate a CV in the AGT/E area?
As the number of young researchers in the AGT/E area increases, and as lower level hiring/promotion/tenure/grant committees need to “justify” their decisions to higher-up committees, it may be helpful to have some explicit discussion of the matter. I am talking about the “domain-specific” knowledge of what carries a signal of excellence in AGT/E rather than the usual debates about the importance and pitfalls of various bibliometrics, rankings, recommendation letters, committees, etc, which obviously apply to AGT/E as well.
I would like to open this up for discussion, but let me start with my own impressions.
- AGT/E is interdisciplinary with most work currently coming from the following scientific communities (a) CS theory (b) AI (c) Networking (d) game theory (e) economic theory (f) operations research. The best publishing venues can either belong to any one of these disciplines or can be in interdisciplinary forums.
- Judging disciplinary publishing venues is according to the standards in that field. E.g. in economics the highest quality signal is publication in one of the “top 5” journals (of which for AGT/E probably means in Econometrica). Similarly, publications in top OR journals (or even general management science ones), or in the top GT journal, GEB, carry a signal of significance. In the CS disciplines, conferences count most (FOCS/STOC and then SODA in CS theory, AAAI/IJCAI in AI, and SIGCOM and then INFOCOM in networking.)
- The top interdisciplinary forum in AGT/E is the ACM EC conference. The top “web” conference WWWC may be considered another top interdisciplinary venue. There is no serious AGT/E per-se journal yet. Other recognized well-regarded (but not top) truly AGT/E conferences are WINE and SAGT.
- In general, publication in a “top venue” in any one of the possible disciplines is a strong signal. Still, not all these signals are of the same strength. While it is hard to compare between the top forums in the different disciplines, at the risk (and hope) of drawing some anger among my readers, let me try my own (subjective) assessment of the signal strength from the top venues (that I’m more or less familiar with) in the different types of disciplines: Econometrica > STOC/FOCS > EC > WWWC > GEB > AAAI/IJCAI.
- Publishing in a top conference and then the journal version in a top venue of another discipline is a plus. An invited paper to a special journal issue collecting papers from a conference is a plus.
- Citation counts are more significant than the venue of publications. Currently Google scholar is the citation count to use.
Comments?
Turing's Invisible Hand 
[…] This post was mentioned on Twitter by Chris Carnduff, TCS blog aggregator. TCS blog aggregator said: AGT/E publication venues « Algorithmic Game-Theory/Economics: http://bit.ly/br8lwQ […]
It’s an interesting topic — I’m not as brave as Noam to post a ranking of these different venues 🙂 but a few comments:
1) The ranking is of course likely to depend on whom you ask. I imagine that microeconomic theorists, even the ones familiar with EC, would be very surprised to see GEB ranked below EC. Generally, people on the X side of things will tend to rank X venues higher (where X can be microeconomic theorists, theoretical computer scientists, AI/multiagent systems researchers, networking researchers, etc.).
2) Some other venues with papers in this general area: Journal of Economic Theory, International Journal of Game Theory, Social Choice and Welfare, Math of OR, Operations Research, Management Science, numerous TCS and AI journals, AAMAS, AMMA, SODA, COMSOC (workshop), NetEcon (workshop), ADT, TARK/LOFT, and SIGecom Exchanges (newsletter).
3) Related to the first point, different venues have different readerships and different criteria by which they evaluate papers, so that, for example, clearly not every EC paper makes for a good GEB paper even though it may be a very strong paper. Also my impression is that WWW is interested only in a specific subarea of our area. (transitioning into a plug…) As announced at EC, we are in the early stages of starting a new journal on Economics and Computation that will be devoted to the intersection of those two fields.
Noam, without endorsing your ranking :-), it would be interesting to know how you would fit other venues that you’ve published in, such as MOR and SODA, into this ranking (esp. MOR vs. STOC/FOCS and SODA vs. EC).
Continuing my exercise in silliness (and please do take it this way…), I’ll probably put MOR at the same level of GEB, and SODA lower than them and above AAAI/IJCAI. While we’re at it, from all the venues mentioned by Vincent, the only ones that I’d put in as strong signals of quality are the JET, MS, and OR journals, which I’d insert as: Econometrica > MS > OR > JET > FOCS/STOC. Of all CS journals, the only one that I’d put in the list is JACM, which I’d put, say, between MS and OR.
So I guess it logically follows that you don’t consider MOR, GEB, and SODA to be strong signals of quality? What about EC? (Of course I understand that you could place the cutoff for “strong” anywhere in your ranking.)
It’s a fun exercise, although the Kemeny distance from my ranking is significant. In any case, now the ranking seems to be irrational: “OR journals” > STOC/FOCS >> GEB = MOR, and because MOR is probably the most highly regarded theoretical OR journal it follows that MOR > STOC/FOCS >> MOR.
I consider all the list of “top venues”, from Econometrica to AAAI/IJCAI as strong signals of quality. (Vincent, I missed SODA in your list.) This is the main real point of my post. The internal ranking between the top venues in the different disciplines can not be taken too seriously…
Ariel: I didn’t rank “OR journals” above anything, but rather the journal called “OR” which I understand is the top one in the field which is ahead of MOR since it is more general. Thus you did not uncover an irrationality in my ranking, although I don’t really doubt that such can be found…
By the way, here is an objective way of ranking conferences according to signal strength (which also has some problems). Given two conferences X and Y, X >= Y iff there exist papers that were rejected from X and then accepted to Y. For example, I know of quite a few papers that were rejected from SODA and then accepted to EC, but I don’t know of any papers that were rejected from EC and then accepted to SODA (without major improvements), so I conclude that SODA > EC.
I disagree with this as a general method. One can argue that using this method would result in you ranking conferences outside your own area higher, because papers that you are familiar with are probably in your own area and have an easier time getting into conferences in your own area.
As for SODA vs. EC specifically, my personal sense right now is that SODA gives a stronger signal of technical prowess, but EC gives a stronger signal of a well-motivated problem. So a 1-dimensional ranking may not be ideal.
Never in a million years would I consider EC > GEB. My ranking is GEB >> EC. I guess that categorize me to be a micro-economist rather than a computer scientist.
From CS journals, I would also add SICOMP to the list (certainly a number of excellent papers on equilibrium computation have appeared there). Otherwise, I would not expand on Noam’s extended list given in reply to Ariel and Vincent (with SODA included). If forced to rank them, I would have GEB > EC.
Vince, it is interesting to hear about the new journal on Economics and Computation. No doubt you will, please keep us posted…
hi, in economics it is most likely to determine if economics system will suceed is by game theory.an example is creating an computer algorthim in a basket ball court.So, if you put a bunch of 0,1 on the floor for binary position with each player having a micro chip you could get the winning algorthim for the losser and winner.Then get the winning lotto ticket and compare algorthims to the basketball games new algorthim that gave the difference in winning and losing.After this simulation has been done in a main frame data base one could apply it to co-operate ceo`s in meet conventions,or marketers on the floor in the stock market.Also by creating an algorthim field at a place of employees, not to sound belittling,but people would be like ants in a algorthim system to determine the stockmarket function for the week and adjust accordingly.calgary ab 18 falmere way
[…] rate, rather in terms of the quality of accepted papers. On the theory side, many people now prefer EC over SODA as an outlet for papers on algorithmic economics (or whatever you want to call it). I know several […]
[…] AGT/E publication venues « Turing’s Invisible Hand https://agtb.wordpress.com/2010/08/30/agte-publication-venues/#comment-2373 […]
[…] on Turing’s Invisible Hand we make it our mission to address the most profound questions of algorithmic economics. This post fearlessly tackles one of those timeless questions: […]