2008 BotPrize contest
The first BotPrize contest was held in Perth, on 17 December 2008, as part of the 2008 IEEE Symposium on Computational Intelligence and Games. The aim of the contest was to see if a computer game playing bot could convince a panel of expert judges that it was actually a human player.
The Judging Panel
- David Fogel - President of IEEE Computational Intelligence Society
- Robert (RJ) Spencer - COO of Interzone Entertainment
- Penny Sweetser - senior game designer at 2K Australia and an AI expert
- Cam Atkinson - a l33t game player
- John Wiese - senior project manager in Special Operations Command Support Systems at Thales Australia
The Humans
- Andrew Smith
- Roderick Baker
- Byron Pogson
- Keith Johnson
- Seb Davidson
The Teams
Team |
Affiliation |
Members |
Amis |
Charles University in Prague |
Michal Stolba |
Jakub Gemrot |
Juraj Simlovic |
|
ICE-UT@RITS |
Ritsumeikan University, Japan |
Daichi Hirono |
Yuna Dei |
Ruck Thawonmas |
|
Intelligent Systems Centre |
Nanyang Technological University |
Budhitama Subagdja |
Ah Hwee Tan |
Di Wang |
|
Underdog |
University of Western Australia |
|
University of Texas at Austin |
University of Texas at Austin |
Jacob Schrum |
Igor Karpov |
Risto Miikulainen |
|
* The team of Jon Wright, David Nichol and Russel Thom from Glasgow Caledonian
University made the finals but was unable to attend.
Results
The prize winners were:
- Best bot: AMIS (Michal Stolba, Jakub Gemrot, Juraj Simlovic)
- Best human: Roderick Baker
- Best Judge: John Wiese
Here is a summary of the results in the form of a table of bots and human confederates, sorted by human-ness, as rated by the judges.
Identity |
Mean rating |
#Judges convinced |
Byron (human) |
4 |
5 |
Andrew (human) |
3.8 |
4 |
Roderick (human) |
3.8 |
4 |
Keith (human) |
3 |
2 |
Seb (human) |
2.6 |
2 |
AMIS (bot) |
2.4 |
2 |
ICE (bot) |
2.2 |
1 |
ISC (bot) |
2 |
2 |
UTexas (bot) |
0.8 |
0 |
Underdog (bot) |
0.4 |
0 |
All the humans were rated as more human than all the bots.
Complete round by round results can be found here.
The table below contains videos made from the games in the final, from the judge's viewpoint.
Within each video, all players appear identical, but can be distinguished by name (e.g. Player123), which appears when the player is targeted by the judge. The table entries give the player names used in each game. You might like to try your hand at picking the bots! Can you do better than the judges?
Each video is between 30MB and 50MB. Warning: these videos depict simulated violence.
|
Judge 1 |
Judge 2 |
Judge 3 |
Judge 4 |
Judge 5 |
Round 1 |
video |
Player144=Judge |
Player443=Human |
Player932=Bot |
|
video |
Player502=Judge |
Player212=Human |
Player773=Bot |
|
video |
Player830=Judge |
Player136=Human |
Player415=Bot |
|
video |
Player875=Judge |
Player433=Human |
Player441=Bot |
|
video |
Player388=Judge |
Player471=Human |
Player639=Bot |
|
Round 2 |
video |
Player787=Judge |
Player859=Human |
Player470=Bot |
|
video |
Player752=Judge |
Player739=Human |
Player148=Bot |
|
video |
Player952=Judge |
Player760=Human |
Player884=Bot |
|
video |
Player173=Judge |
Player408=Human |
Player389=Bot |
|
video |
Player640=Judge |
Player961=Human |
Player178=Bot |
|
Round 3 |
video |
Player673=Judge |
Player367=Human |
Player833=Bot |
|
video |
Player711=Judge |
Player431=Human |
Player189=Bot |
|
video |
Player231=Judge |
Player247=Human |
Player703=Bot |
|
video |
Player845=Judge |
Player525=Human |
Player352=Bot |
|
video |
Player568=Judge |
Player275=Human |
Player794=Bot |
|
Round 4 |
video |
Player325=Judge |
Player959=Human |
Player228=Bot |
|
video |
Player165=Judge |
Player707=Human |
Player818=Bot |
|
video |
Player720=Judge |
Player160=Human |
Player822=Bot |
|
video |
Player475=Judge |
Player859=Human |
Player739=Bot |
|
video |
Player467=Judge |
Player755=Human |
Player886=Bot |
|
Round 5 |
video |
Player261=Judge |
Player357=Human |
Player217=Bot |
|
video |
Player237=Judge |
Player980=Human |
Player157=Bot |
|
video |
Player635=Judge |
Player955=Human |
Player494=Bot |
|
video |
Player373=Judge |
Player642=Human |
Player189=Bot |
|
video |
Player683=Judge |
Player178=Human |
Player578=Bot |
|