I've been debating how the criteria should work; I've also been thinking of doing a points-based system, but with results unique to each voter. The way I figure, this is a completely ridiculous thing to be doing anyway, so we might as well make the voting procedure ridiculous as well.
Scoring will be something like:
5 points if a song contains any non-traditional instrument (i.e., horn/string sections for bands that don't ordinarily feature those instruments)*
5 bonus points for each decade in the list
5 points for each different genre represented
10 points if two consecutive songs are in the same key
-5 points for showtunes
5 points for non-English lyrics (although "ba ba ba" doesn't count)
5 points for instrumentals
5 points for a song you (the voter, not the submitter) own
10 points for a song you like and don't own
-5 points for song you don't like
The idea this time around is to have as varied a list as possible, and to make the voting process somewhat personal. If your iPod only features Pitchfork's best new albums for the last 2 years, you will probably lose.
* This is potentially tricky, but also could be cool. For example, Genesis and Jethro Tull's use of flute does not count, as Peter Gabriel played flute a lot and Jethro Tull wouldn't exist without it. Likewise, any James Brown song wouldn't get extra points for having a horn section. Sufjan Stevens, however, renders this argument completely moot, so maybe it's not such a great idea. If you can find a better way to incorporate the concept of extra credit for having non-traditional instrumentation, be my guest.
If you have comments or ideas, feel free to send 'em; I'll unscreen those comments, obviously, so everyone can discuss.