Loebner_small.gifBob Norris, administrator of the Bot Central forum, has renewed a monthly competition for producers of chatbots: The World Chatterbot Competition.  This is the first monthly competition for conversational agents (that Im aware of ), but surely not the first competition for chatbots or even other forms of elit.  

In 1991, Hugh Loebner offered $100,000 to the first person who could answer Turings challenge and in doing initiated the Loebner Prize competition.  Although no one one has successfully answered the challenge, botmasters continue to vy annually for the smaller partial prize ($2000).  In effect a whole field of practitioners was inspired by this competition.  Rollo Carpenters Jabberwacky (discussed here) won the most recent round.  (Didnt Turings initial [con]Test initiate these contests, albeit without the structure of the rules and reward?)

While some prizes (i.e. Pulitzer, Nobel) reward general behavior or behavior independent of the competition, others (i.e., the X-Prize Cup, the Olympics), spawn fields of competitors who might not otherwise be working  within the parameters of the competition.  We imagine Ghandi did not act nobly just to please Nobel. (Oh, wait, he never won it. If he had only been a bit more virtuous, his life could have really paid off.)  Does anyone shot put just for relaxation?

Of course, much of academic work revolves around contests and tournaments, whether a CFP for publication or an application for a grant.  Often, they will narrow the field (e.g. panel on chatbots that teach algebra) or even establish design criteria.  However, those dont have the same project-fostering power of an annual competition based on consistent rules. Continuous projects offer entrants a reason to continue to develop new versions in subsequent years.

The contest can artificially create new grand (or minor) challenges, to which artists can apply their sustained energies.  Norris, who also sells his Cyber Operator chatbot-based customer support, explains, The WCBC was created to give Novice Botmasters a fun learning tool.  As they compete they will find areas to improve their bots.
The current Users Choice awards allows registered users to rate bots in a variety of categories, including: Best Interface, Smartest Bot, Info Bot, Funnest Bot, and Your Favorite Bot.

Winners of previous months contests include: Mathetes (5/05) (mentioned here), Joan (6/05), and Gherkins (9/05). The April contest also features ALICE (discussed 1, 2, 3, 4) and my own Joe from Barthes Bachelorette.  Although the contest differentiates between custom (from scratch) and template bots (programmed by commercial bot company but trained by the user), all 10 currently are competing in the same category.

Other competitions for chatbots include Verbots Chatterbox Challenge: Started in 200, this annual contest features a variety of bots including many developed through Personality Forge.

Of course, many of the same names appear on these contests: ALICE, Jabberwacky, Jabberwock, and Talk-bot, as well as a sea of other competitors, including adapted clones of the winners. 

Chatbots are not the only eliterature object encouraged by contest.  Most notably, the widereaching  Annual IF Competition has kept newbies and artists developing.  Founded by Kevin Wilson the rec.arts.int-fiction Usenet group in 1995, the competition has grown in size and scope from a mere 12 in the first competition. 

Other contest include:

Coding Fun:
Perl Poetry
International Obfuscated C Code Contest


Electronic Literature fiction contest (only in 2001?)
Drunken Bots Panliterary Awards

Numerous literary magazines (online and otherwise) have contests.

Text and Image:
Grouchhouse has a monthly contest of the best mash-up picture using its collage system.

Personal/family favorite:
The Bulwer-Lytton Dark and Stromy Night contest for worst first line.

While contests create tournaments for artists, beginners, and programmers, they do not always seem to produce answers to the larger questions.  Sometimes, programmers begin to write to the test.  This has happened with the Loebner Prize (as people write articles on tricking the judges) and perhaps has happened in the development of chatbots in general.  Critics (see Stuart Shieber, for example) of the Loebner Prize argue that it is not spurring creativity but wasting time in the pursuit of these larger questions.  Of course, such objections completely trivialize and denigrate the accomplishments of the programmers and artist who innovate while working within the constraints of the contest. Kurt Heintz of About.com further warns against fake contests.

When legit and well-run, these competitions create fun regular tournaments for developers and newcomers alike.  I also suspect they can be lodestones, attracting developers to form communities around a central challenge.

Perhaps WRT will hold a contest for the best online contest.  Please submit ones weve forgotten.

(Oh, and Vote for Joe!!!)

0 Responses to “Chatbot Idol--Contesting Innovation”

  1. No Comments

Leave a Reply

thesis writing service