[appleeaters]Benchmarking

Hello Everybody..
I dont know about you guys, but my algorithm is comming along nicely and I was wondering if anybody is up for some benchmarking? I know that I would personally like to see what I am up against.

I have generated three different maps and posted my AI scores next to them. They are all 500x500 maps and I will try and make one that has lots of robots and few apples, one with lots of apples and few robots, and one with about an even mix.

Map 1: Apples: 1886, Robots: 535, Random moves available: 4
http://www.stellarcore.com/phpcontest/benchmark1.txt
My Score: The player ate 11 apples, killed 51 robots, and made 84 moves. Score: 1442

Map 2: Apples: 516, Robots: 1383, Random moves available: 1
http://www.stellarcore.com/phpcontest/benchmark2.txt
My Score: The player ate 1 apples, killed 40 robots, and made 28 moves. Score: 444

Map 3: Apples: 2075, Robots: 2128, Random moves available: 2
http://www.stellarcore.com/phpcontest/benchmark3.txt
My Score: The player ate 1 apples, killed 81 robots, and made 27 moves. Score: 856

Since map3 has so many of both robots and apples, heres another map that has ~500 of each.

Map 4: Apples: 482, Robots: 513, Random moves available: 2
http://www.stellarcore.com/phpcontest/benchmark4.txt
My Score: The player ate 4 apples, killed 57 robots, and made 79 moves. Score: 812

I have two robots that I'm working on. Their scores are

Robot 3 test 1: 10 apples, 119 robots, 116 moves = 1958
Robot 3 test 2: 2 apples, 61 robots, 45 moves = 720
Robot 3 test 3: 0 apples, 57 robots, 21 moves = 528
Robot 3 test 4: 2 apples, 10 robots, 31 moves = 238

Robot 4 test 1: 12 apples, 141 robots, 149 moves = 2312
Robot 4 test 2: 1 apple, 39 robots, 24 moves = 442
Robot 4 test 3: 1 apple, 145 robots, 35 moves = 1480
Robot 4 test 4: 2 apples, 11 robots, 31 moves = 248

I modified them to have static teleport locations (so my robotslib.php would repeat the exact same score for every run I did with the same game and same robot). All this and more at my dorkiest of web sites http://rumkin.com/reference/php_contest/ (only up while the contest runs).

Great! Later today sometime I will download your updated robotslib and retest my robot again (I have made some updates, so its working a bit better).

Here are my latest benchmarks using my latest version of my robot and your new robotslib (same teleport points):

Map 1: 8 apples, 136 robots, 130 moves => 1900 points
Map 2: 1 apples, 025 robots, 027 moves => 0296 points
Map 3: 1 apples, 099 robots, 029 moves => 1032 points
Map 4: 5 apples, 054 robots, 127 moves => 0784 points

more updates to come...

Just checking, but are you using the modified benchmark game files that are on my server too? They have the data required to make the teleport move not use random coordinates (the 'randommoves' lines at the end).

My best scores for each (using the current official lib, NOT Tyler's -- so teleport locations will differ):

Map 1 -
The player ate 9 apples, killed 73 robots, and made 86 moves
Score: 1458

Map 2 -
The player ate 2 apples, killed 28 robots, and made 25 moves
Score: 430

Map 3 -
The player ate 2 apples, killed 178 robots, and made 50 moves
Score: 1880

Map 4 -
The player ate 4 apples, killed 129 robots, and made 117 moves
Score: 1456

Chris B

Fidian: yes

Hi. I've rerun my script through thewe four maps, but this time using Tyler's robotslib so that the teleports are to the same place as your guys'. Better to compare that way.

Map 1 -
The player ate 9 apples, killed 86 robots, and made 113 moves
Score: 1534

Map 2 -
The player ate 1 apples, killed 27 robots, and made 25 moves
Score: 320

Map 3 -
The player ate 2 apples, killed 111 robots, and made 35 moves
Score: 1240

Map 4 -
The player ate 7 apples, killed 33 robots, and made 114 moves
Score: 802

And, thanks to Tyler's MakeMap script, watch my AI play Map 4: http://mretc.net/php_contest/

Is all this benchmarking being done on the same machine?

> Is all this benchmarking being done on the same
> machine?

No, but if no one is timing out on any of these maps, it doesn't matter.

My results are:
The player ate 9 apple(s), killed 93 robot(s), and made 113 move(s).
Score: 1604.
The player ate 1 apple(s), killed 28 robot(s), and made 25 move(s).
Score: 330.
The player ate 1 apple(s), killed 108 robot(s), and made 33 move(s).
Score: 1114.
The player ate 7 apple(s), killed 37 robot(s), and made 114 move(s).
Score: 842.

All results using Tyler's lib.

Hey Tyler,

Hey Tyler,
(sorry about last message.. accidentally hit submit)

I think that it would be beneficial if we could modify your perl testing program (perhaps a php version?) so that it uses the same 100 maps so that we can all test with maps that are to spec. If I have time, I will write one up.

I just finished one. Still need to test it more thoroughly. What I have is on my web site: http://rumkin.com/reference/php_contest/

I'll also be posting my scores there.

FYI: I decreased the number of random games to only 100. My statistics friend told me that 30 games would be a minimum for good statistics, and the 250 games that I was running was way excessive. If you downloaded the benchmark.php file from my web site, just change the 250 at the top of the file.