[Trilite] Questionable tie breaker

Say you have two identical "perfect" scripts, and one that makes random moves. Regardless of how you use moves as a tie breaker (sum(all) or sum(losing)-sum(winning) or whatever), the outcome of this tournament will be random (except that the random script will be last).

This is obviously true - the outcome between the identical scripts will be identical, and the number of moves required to play the random script will obviously be random.

I say that time is much better. With time you can at least win by being a better coder, not by getting lucky...

I somehow agree... sounds like an interesting argument. However Matt has already stated that he will not change the rules unless absolutely necessary, so I, personally, doubt we'll see any changes to the tie-breaker at this point...

I also somehow agree, but since my judge script is 98% complete or something like that, and I am going away for some time (unknown duration of stay) and wont be able to alter my judge script in this period I also hope this rule wont change....

But what about this rule cange instead :
"RANDOM SCRIPTS" - NOT ALLOWED !!

That won't help - any non-perfect script is somewhat random - most probably it will lose against a perfect script, but the number of moves will most probably not be the same always (i.e. random)

I see what you mean...

But there will probably be a bunch of non-perfect scripts (mine included (hope not))....

So to make it fare to everyone the Top10 scripts of all can compete against eachother. Snd then the Top5 of those to determin the final official winner....

This would probably make everyone glad...
And random and bad scripts would not affect the final outcome too much...

This would be an advantage for perfect scripts that don’t try to win if they don't begin. In my opinion scripts should fight to the bitter end, and if they don’t..... they deserve to be losers ;)

I agree to mcoders statement that the result will be somewhat random. My script has a sort of scoring system. And if more moves get a equal score it will select a random move out of those scores. I don’t think you can completely block luck out in this game.

to display mcoder's point (and see for myself the impact) I have run my judge script 3 times

Script 1 is a copy of Script 4

and 2 and 3 are some testscripts to attack my scripts in a non perfect but advanced way ;)

Judge run 1:
Script 1
[wins] => 25
[total_rounds] => 187

Script 2
[wins] => 7
[total_rounds] => 227

Script 3
[wins] => 3
[total_rounds] => 219

Script 4
[wins] => 25
[total_rounds] => 187


Judge run 2:
Script 1
[wins] => 25
[total_rounds] => 203

Script 2
[wins] => 5
[total_rounds] => 248

Script 3
[wins] => 5
[total_rounds] => 231

Script 4
[wins] => 25
[total_rounds] => 188


Judge run 3:
Script 1
[wins] => 25
[total_rounds] => 183

Script 2
[wins] => 5
[total_rounds] => 218

Script 3
[wins] => 5
[total_rounds] => 204

Script 4
[wins] => 25
[total_rounds] => 193

Well, first of all, I disagree because we shouldn't penalize a recursive-searching algorithm. As my thing is right now, it goes it less than .001 seconds except for the very first move as the second player.

I think the best thing, in an ideal weighting system, is to assign points to each contestant depending on how many wins they make. Then, if you beat a "hard" opponent, you get more points. But that would still lead to ties.

I see the concern now about the random player... but we're assuming perfect players even going second, so let's hope no more than 1 person has a perfect player while going as second player.

I just found how to improve the tie breaker !
First, we count the number of wins (this won't penalize for random entries, since they are likely to lose !).

Then, all the entries that have the same amount of wins will be judged by the number of LOST rounds (this number must be minimal), instead of TOTAL rounds.

This way, the number of rounds for winning or drawing is not taken into account (and the competition remains fair, IMHO).

I don't want to be judged by time, since I wrote a recursive search (no table saved !).

JC

If you are participating in this contest, please send an email to contest@codewalkers.com with your choice of the following:

1. Leave tie breaker alone.
2. Change tie breaker to total script run time.
3. Change tie breaker to total number of rounds in games lost (lowest number wins).

Please respond by 4/25/2003 23:59 GMT.

Should
3. Change tie breaker to total number of rounds in games lost (lowest number wins).
Not be
3. Change tie breaker to total number of rounds in games lost (highest number wins).

To Mcoder:
I still think the number of losses = total rounds - number of wins in almost all cases. So if 2 scripts have the same number of wins there is a huge chance that they have the same number of losses to.

Self started (winning) games for perfect scripts against non perfect scripts will be between 3 to 7 rounds max

Self started (winning) games for perfect scripts against perfect scripts will be 7 rounds each time

Not self started games for perfect scripts against non perfect scripts are the most interesting ones.

Again winning in as little rounds as possible is admirable

Losing in as little rounds as possible is NOT admirable

So a fair judgment would still be in my opinion

First sort on winning games... most wins is first ;) and in case of a tie
winning rounds - loosing rounds lowest score wins.

I think time will be an extremely difficult thing to measure. If you use the max_execution_time directive under Linux it will use system counters and if a buffer cleanup in the kernel takes place during the run of your script it will not be counted as time for your script. If you measure time outside your php script by using file('http://localhost') you also time apache. Was this script in the buffer of this specific apache script? Compile time. Other processes running at the same time (like cleaning up php memory allocations of your opponent). If you don’t run over localhost it could even be other network traffic slowing it down

I doubt so !
What does make you think that ?



I count that in moves, instead of rounds, so yes, it seems that 13 moves is the longest win, but I found certain positions where the win is at a depth of 18 moves !
Suppose your program is confronted to an imperfect program which can play the first perfect moves...



I agree that losing in a small number of rounds is not a pride, but it forces your script to evaluate the strength of your opponent.
Suppose an opponent doesn't play ALWAYS perfectly:
you cannot let it win in 5 moves, since you'll lose a precious point if you can find an error in his defence. Also, trying to make the game as long as possible is very easy, since it's the same algorithm as winning.
I would like to add that all games are harder when you have to play to the losing variant !



I don't agree with that, this works only if you are confronted to yourself !

Now, Matt let us choose what we prefer.
Using the time as the tie breaker will reduce a lot my efforts on this contest, since I'll just have to save the explored tree (for information, I currently wrote my player in C, and it explores the full tree to 20th move in exactly 45685 nodes, not bad, eh ?).

Let's vote !

JC

But how can you? Most games will be over in 13 moves or less (6.5 rounds). The only way to determine whether a script is good is to go through with the game. Perhaps he just got lucky with the first few moves. And since there's no guaranteed order of matches, that information doesn't help you in later rounds, because you never know who you're playing.


Will it? What if I generate the same tree in half your time? What if I store all the information in 50kbytes as opposed to your 1mbyte (making it load faster). What if in the last 5 moves, loading info is slower than recreating the subtree? The point is, of course, that this are all coding challenges, as opposed to guesswork dealing with reducing the number of moves...


At one point or another, everyone has to make a recursive search (unless I completely missed something about this game). It only forces you to not do it each turn (which is good for everyone, otherwise we'll wait 2 weeks for the results ).

Suppose that every contestant submits a perfect player that computes the full tree at first move(it's pretty simple, since there are less than 20,000 nodes).
Except the first move, all moves will take the same amount of time, so your criterion is simply to bench the creation of the tree.
Compared to analyzing the behaviour of the opponent, this is much easier, but also I doubt there will be a huge difference in timings between the best entries, or am I missing something ?

JC

Recently, I asked everyone to vote concerning the tie-breaker for Trilite. Well, it was very close, but the decision has been made to leave things as they are. I understand that this may disappoint some, but now is not the time to worry about what could have been or would would have been. Now is the time to finish those scripts up and get them in. Some that wrote me voting to leave the tie breaker alone seem to believe that they have solutions that will fair very well under the current conditions. It is now your job to see if you too can figure out what they believe is true.