[appleeaters]Critisism for this contest

Let me start by saying that this was the first coding competition i have ever attended and i must say it was real fun. I even took a good position (6th or 7th), so don't take my words the wrong way.

I have a couple of complains about the way the competition was designed:

My first disappointment comes from the fact that this competition could not pick out the best algorithm, since randomness played such a big role in the results. Take for example the results of a script running the first map 10 times:

1: 1 apples, 20 robots, 9 moves = 282 points in 1 seconds
2: 3 apples, 74 robots, 26 moves = 988 points in 3.2 seconds
3: 1 apples, 59 robots, 25 moves = 640 points in 2.5 seconds
4: 4 apples, 37 robots, 22 moves = 726 points in 2.5 seconds
5: 1 apples, 44 robots, 23 moves = 494 points in 3 seconds
6: 1 apples, 51 robots, 23 moves = 564 points in 2.5 seconds
7: 2 apples, 55 robots, 30 moves = 690 points in 3.3 seconds
8: 6 apples, 111 robots, 35 moves = 1640 points in 4.2 seconds
9: 5 apples, 78 robots, 34 moves = 1212 points in 4.1 seconds
10: 2 apples, 50 robots, 23 moves = 654 points in 2.7 seconds

As you can see it scores from 282 to 1640 points! With zero teleports it always scores 272 points. I wish future competitions will be more deterministic, eliminating randomness problems.

My second complaint is the admission of multiple entries. I won't go into details, since Matt has already said that future competitions will only allow one script per developer, but it was especially bad for this competition where randomness played a big role (i repeat, i don't want to whine, i submitted multiple entries myself).

So that's what i had to say and i really hope no one gets offended or anything like that... (especially Matt, Zack and anyone else who donates his free time so we can have fun).

we already know all that.. do you have anything new to add?

Don't mind zombie...he is just zombie .

Anyway, thanks for the feedback. Yes, we are aware of both of those and they will be corrected in the future..

how exactly is this different than what i said?

how exactly is this valuable feedback?

we already know all that. we already said that we will fix it..

he knows that we know that, and he even knows that we said we will fix it..


so, let me ask again, how exactly is this valuable feedback?


(and i wasn't even rude or anything this time)

And i think i wasn't rude too. And yes i know you know about the multiple submissions (i say it myself), but i saw no comments from you, Matt or Zack (or anyone "official") about the randomness. If i missed the thread, then accept my apologies.

zombie...the difference between our posts is that I started it with "thanks for the feedback.". See that makes someone feel good. I know you were not trying to be rude, but it was just one of those things that the came out in a 'zombie' way and could have been taken as a rude comment. ;)

I also had not seen any complaints about the randomness, though I could have missed it as well.

(btw zombie....you know you are my favorite admin )

brown noser... ;)

I have two things to add to the list of complaints. Before I do that, I should add that the reason I submitted a billion robots is because of the randomness factor. The algorithm testing really didn't do much, so I figured that by submitting a bazillion robots, I would have a better chance. I'm quite pleased that the first place person only submitted 1 robot (unless I'm mistaken). I like that.

Anyway, it would be nice if there was a new contest every month or so, even if the old one isn't done judging. I found it taxing to wait day after day for another coding challenge to commence. You had more time to judge it than we had to write them. Granted, you could come back with "we had to run them all", but you only had to run them all once. In our testing, we ran each of our robots numerous times, then we needed to code the AI for the bots. You needed to proofread the entries, find a judging script (I had 2 available for download), and do your customizations that you deemed necessary ... then hit go and collect the results. Granted, there could have been a death in the family or other unforseen trouble, but months of waiting seems a bit long.

Secondly, now that this contest is officially over, where is the new contest? If the deadline for contesting the results was Friday, how long do I need to wait to get a new problem to solve?

I'm totally in it for the challenge and not the prizes. I can't wait for another challenge. I was thinking that if you have trouble coming up with one, you could limit the program by size (1024 bytes or smaller, etc.) or perhaps by the number of vowels. That would be interesting.

Thanks for having the contest. I really can't wait for the next challenge.

hehe.. :-P there is a first time for everything, even for matt to "kiss up" (is there a polite way to say this in english?) to someone here, and not the other way around (like with notepad or postalcow)



no, this is not about you.. you did nothing bad (maybe just fruitless/redundant, but not bad)

this is about me, and about the fact that everyone thinks that i mean something bad when i don't sugar-coat it, or when i don't say "thank you"..

if i said smth like "you are so stupid, we already know that", that would be rude, but i didn't (because i don't mean it), so why is everyone trying to find in my words something that i didn't say (i usually say what i mean, no need to "read between the lines" with me (and i consider that a quality in another person).

i genuinely wanted to ask if there is anything else that you have to say/suggest (while you are in this criticism mood).

about the randomness.. it seems that nothing was actually said out loud about it (even though i could have sworn that it was), but i thought it was understood when mat agreed with this comment on the other thread..



see, a small amount of randomness is not that bad (even in life, you need to have a bit of luck in order to do good).

also, multiple entries by the same person are not *that* bad, if the person can write several different (good) algorithms..

but the combination of (great deal of) randomness and multiple entries is a killer! when luck is a great factor, then number of attempts (entries) plays a very big role in determining the winner (even if you submit absolutely the same script 16 times).


for the next contest, i vote for maybe just a bit of randomness, and for allowing multiple entries (maybe up to 3, or something), but, the score for that person would be calculated as the average of all his entries.

reasoning: i might think of two very different but equally good algorithms. first might be good for one half of the situations, and the second for the other half. i don't know what situations (test cases) matt will use to judge our scripts, so i send them both.

the point is that i maid them both, and that both algorithms represent my skill level, so the average makes sense to me..

And that's why i pinpointed it in a new thread... so we can discuss it, get an official decision and get over with it.



I would agree if this was a short stories or beauty competition where human judges are needed... but in a coding competition i think it's more right to always have the same results for the same dataset/entries (it is what computers do afterall). If i feel lucky i can always use randomness in my script, but i see no need for the judge-script to shuffle the results.



I agree, average scoring of multiple entries means only good/different approaches will be submitted, which is a nice thing.

Yes, exactly! I agree 100%

submitting multiple entries when using an average will actually significantly decrease your chances of winning ifluck is a factor in the scores.

Over any number of entries you're going to move towards a "mean" score. This will never be as good as the outlier on the right or as bad as the outlier on the left but in any event you almost guarantee yourself to be in the middle. This might be a good plan in a multi round contest where the lowest scorers were eliminated but it's destined for failure where one single "lucky" highscore takes the prize.

For the next contest, we will put nanowires in everyone's brain to see who could have written the best script without actually writing it.

In retrospect, it's obvious that the random teleports had too much of an impact on script performance. Luck should not be one of the deciding factors in a contest like this (one designed to recruit top-notch coders for clandestine programming missions - I'll try not to give too much away).

I also agree that in the future, there should be a limit on the number of entries each person can submit, and that the average of a person's entries should determine his or her score.

in this game changing the range of the teleports to something like 5-10 instead of random would greatly reduce the luck factor