Page 2 - [appleeaters]Benchmarking

Will your benchmarking program generate the same games on your computer as it does on mine?

It really should. If your appleeater and robotavoider scripts don't give you the same numbers as they give me, then I need to do something with the random number generator. As it is, I seed the prng with a specific value and generate random numbers based on that. Running it multiple times on my machine will generate the same 100 games every time. Because my robotslib.php uses randommoves and randomseed, even the random robot will (should? haven't tested this one) come up with the same score every time.

Just a quick comment on your benchmarking script, the output files (game.***) are offset by one. For instance, game.58 is really trial 59. Kinda threw me off at first. =) he he he.

Hey Tyler,

Just curious, but what did your bot(s) score on map 59 (or 58 by the game output files)? My bot times out after about 84 moves (while trying to commit suicide..) If I disable all time limits and suicide timers, it gets 23 apples, 1103 robots, 129 moves and 13072 score (impressive score that makes me curse the timeout limit! he he). Just wondering if yours had problems with that map as well (15265 apples is nuts).

59: 1 apples, 148 robots, 27 moves = 1526 points in 58.2 seconds

I guess that my robot must commit suicide before being able to gobble up all of the apples. Only 27 moves? Dang, that map must be huge! ... Yep. 949x810 with 15263 apples and 6370 robots. Lots and lots of processing.

Dang. Got automatically logged out when I made that last post. It was me. Trust me. :-)

Edit 1: One of the annoying things is that all of my 27 moves were made in less than 0.6 seconds. 27 * 0.6 = about 16 seconds for my part. Since my script finished in about 58 seconds, that means 42 seconds were consumed by robotslib. Eeeeek.

Edit 2: With no time limit, I get 10 apples, 864 robots, 100 moves, 9440 moves.

There have been a couple maps that mine has problems on like that. The *best* I have had so far (with no time limit) was one where I killed ~ 1200 robots (all but one) and went around collecting apples until suicide time and it gave me a score of around 22,000.

Just out of curiosity, do you mind if I ask you what your suicide strategy is? Mine currently will wait until there is 5 secconds left and then move to the nearest dead bot or the nearest robot (whichevers closest). But I think that I have to find a better way to call my suicide method. Right now, it kills it self with 5 secconds left regardless of how big the map is or how many robots there are, but in some maps, it could take like 10 secconds to kill myself, while in others only 2 secconds, so if I set it to 10 then I am wasting 8 secconds in other maps.

With map 59, It *would* get to killing it self but it takes more than 5 secconds to get the move back from the computer, so it times out before the suicide has a chance to be called.

I haven't implemented a suicide function yet, but one way to go about it might be to time each move and keep a running average so that you know how much time you'll need to get to the closest robot.

My "suicide" function is to essentially become appleeater. Let the bots come to me. Not the best strategy, especially if it is a map that takes a lot of cycles to compute the robot movements...

Mind if I ask what your (Cypher and Chris B.) scores are for your robots? My web page is looking quite lonely with my bot, appleeater, and robotavoider.

Yah.. I will post soon. I am somewhat unsatisfied with the scores I am getting. My robot is split into several "personalities" and each personality works perfectly, I just have to figure out some ingenious way to know *when* to do them. Biggest dilema is probably which apple to go after, currently I have it just going to the closest apple that it can reach before a robot, but thats not the best because it could be passing up a patch of apples for a single apple thats in the other direction.

Shoot. I had the times reversed. My code took 3 to 4 seconds per move. The library only took the 0.6 seconds.

Ok, here are my results with your benchmarking program Tyler.

Sum: 484 apl, 20827 rob, 6420 mov = 243839 pts, 0 fail
Avg: 4.8 apl, 208.3 rob, 64.2 mov = 2438.3 pts, 0% fail
StD: 9.14 apl, 231.73 rob, 96.67 mov = 2544.7 pts

Wow.. our scores are *very* close. I will have to do just a bit more work to get mine a *little* better. I would still like to see some other peoples scores... also, I wouldnt mind knowing how many people are planning to compete. Judging from the forum activity so far, I would say only about a dozen or so...

note: for whatever reason, your benchmarking program wasnt able to collect the times for mine, but none of them timed out, so just put n/a next to the time column and 0% in the fail column.

Are you using the updated robotslib.php? It now prints out the time per move and the total time. The benchmark.php script needs that information for figuring out how long the script ran for.

Ok, this is strange.. I ran the benchmark again with your newest library and the same robot, and I am getting completely different scores for alot. I ran the same benchmark on appleeater and robotavoider to make sure that they match, and they seem to match up, so here is my updated scores:

Sum: 452 apl, 22916 rob, 6035 mov = 262290 pts, 850.1 sec, 0 fail
Avg: 4.5 apl, 229.2 rob, 60.4 mov = 2622.9 pts, 8.5 sec, 0% fail
StD: 8.48 apl, 370.14 rob, 78.55 mov = 3831.26 pts, 12.5 sec

Wow, I am a bit more satisfied with that score, but even more impressingly, the speed change between the old and new libraries is magnificant. Although I dont know the *exact* average from the first library, I can assure that my robot is now at least double the speed.