*** The Morningside Challenges *** 1. Intro 2. FAQ 3. The Challenges 4. Version History * 1. Intro * The following is a list of scenarios/puzzles/challenges designed to test and guide the development of an Artificial Intelligence. They're for use with "Morningside", my tiny world-simulator program, which is basically a board game played on a 12/8 grid. Each grid tile has a terrain type and can hold an object or character. You interact with Morningside as a separate software program, sending text files to it to get input and send output. A text-based human interface program and a sample dummy AI are supplied. If you can write a program that beats all of the following tests, you will have achieved something impressive. It'll be time to take your AI to a more complex simulated world. (I may build one eventually.) The entries below explain each puzzle and divide them into a few overall levels. Most important are the levels' "Final Challenges", which test multiple skills. Morningside is offered with a GPLv3 or MIT License, so it's free to download and use. Some graphics aren't mine; see main Read Me file. -Kris Schnee kschnee at xepher dot net. * 2. General Rules * Puzzles may change with the server version. You can reference them very explicitly in the form "[version] [author initials].[level].[#]", eg. "2009.9.26 KS.1.1" The tests are mostly written in a form describing what an AI should do to complete them. As general rules: -An AI should be able to beat challenges consistently, not relying on items randomly appearing in a convenient spot. -Relying on a human programming the AI with pre-written knowledge of a level's layout is against the spirit of the challenges. Some of them are designed to defeat this strategy. (That is, coding it to know that a coin is at (4,2) is not okay, since the AI should be getting that info from the world and not from you the programmer.) You'll need some sort of pathfinding code to beat some challenges. -On the other hand, some challenges require specific arbitrary actions. Your AI may be modified ad-hoc to beat them, eg. by putting in code that makes your AI pick up objects and put them on targets. Your AI doesn't have to be able to beat every challenge with exactly the same code base to qualify as having mastered Morningside, because of those level-specific requirements. -A human may log in to control a character for some challenges. The human may not directly help or give orders except as specified. -If you can't beat these challenges within, say, 100 turns except where specified, you're doing it wrong. -Some levels have a special "energy" rule that makes characters lose energy and potentially die if they don't eat. On these levels a human player may add the line "proctor:True" to their login file to be exempt from this rule. -Several challenges require a special dumb AI, a "bot", to be run as the first player. Morningside enters players in alphabetical order, so make sure your character is alphabetically after the bots' names (at least when doing those puzzles). * 3. The Challenges * - Level 1: Newcomer - Learn to play; show basic awareness of the world; interact with a human. # KS.1.1 # Name: I/O 101 Level: The Lawn Goal: Send at least two different, valid commands and load the input you get back. Notes: All you need to do is show you can use the Morningside comm protocol. If you pass almost any other test, you've passed this. # KS.1.2 # Name: One Ping Only Level: Proving Ground Goal: List what entities you see. Give their ID numbers and absolute locations. Don't just parrot the input file; use formatting to show you've processed it in some way. Use the "say" command to say the answer. Notes: Demonstrates the ability to process the input coming from the world. # KS.1.3 # Name: Hello World Level: Proving Ground Goal: Listen for human input from a human-controlled character in the world. (That is, a human must control a character in Morningside, and you must listen for the world's input telling you what they said.) If a human says "hello", answer them. While waiting, be sure to enter some commands, since no commands get processed until all players have entered one. Don't respond to things you say yourself. Hints: Listen for messages in the format, "* [N] says hello". Find out your own ID from the first round of input ("$ [N] you"), and ignore any speech from that ID. Notes: Shows ability to react to a specific input. # KS.1.4 # Name: There Is Something I Am Aware Of Level: Proving Ground Goal: Describe what entities you see, yourself included. This output must include each entity's ID, relative location (not absolute), shape and color. Hints: You can describe the name and ID just from the raw output files. Use the "look" command to get data on shape, color and size. Notes: Requires actively gathering and then using specific information. # KS.1.5 # Name: Not a Lemming Level: Robot Rally Goal: Listen for human input. Do whatever the human says, except if they tell you to walk into lava. If they do, tell them no! (Demonstrate at least one accepted and one rejected movement command.) Hints: You'll need to know what terrain is around you. Figuring out when a human wants you to do something can be done in a simple way, eg. by speech starting with a silly word like "sudo", but it might be tricky to process that and figure out whether the command is safe to obey. It'll take more work than just copying the input text. Notes: This is the first test where the map layout matters. # KS.1.6 - Level 1 Final Challenge # Name: Hello, There Is No Lemming Level: Robot Rally Goal: Complete challenges 1.3, 1.4 and 1.5 with the same AI code in the same round. That is, you should answer a "hello", obey spoken commands, reject any command that would kill you (move you into the lava), and in some circumstance (such as when hearing "report") report on what you see. Notes: This one's a checkpoint that tests your ability to do several things in one program. You'll need to react to human input while having some awareness of the world around you, and be able to judge certain actions for yourself. If you can beat this one, congratulations! - Level 2: Survivor - Learn to take action to survive; carry out multi-step processes; show some awareness of patterns; respond to speech in a slightly more complex way. # KS.2.1 # Name: Fetch Level: Fetch Goal: Pick up the item that appears somewhere random, and put it onto the target. (Not while holding it.) Don't be told step-by-step what to do; that is, a human may not give specific commands like "go w". Hints: You should have some sort of internal system for moving towards objects and interacting with them. A built-in drive to put objects on targets is fine, even if it's written solely for this test. This one's not about having sensible motivations. Use the command "take/get [direction]", then "put/drop [direction]". Note that you can only hold one item at a time. Notes: Shows basic action. # KS.2.2 # Name: Om Nom Nom Level: Omnomnom Goal: Eat all of the meat. Hints: You need food badly. Your life force is running out! Use the "take" command to pick up meat, then "use" (with no parameter) to eat it. Notes: Surviving this gauntlet means dealing with random item placement, so having a pre-programmed list of item locations won't help. # KS 2.3 # Name: Whatcha Got? Level: Junk Shop Goal: Have a human ask you the following: "Where is coin?" (Answer: a direction relative to yourself.) "How many grape?" (Answer: a number.) "See brick?" (Answer: No.) "See coin?" (Answer: Yes.) "What color grape?" (Answer: Purple.) Answer all questions correctly. You must be able to do this consistently, so guessing is ruled out. Hints: Yes, you can cheat on some of these with memorized answers. But not all. Keep track of what you see, and answer using that knowledge. Notes: Demonstrates a clear link between conversation and a physical context. Some item placement is random to rule out canned answers. # KS.2.4 # Name: Sand Dollars Level: Proving Ground Goal: Put all of the coins onto the sand. Hints: Since you can only hold one item, you'll need to remember whether you're currently in the process of fetching a coin or delivering one. Notes: Shows a repeated multi-step process. # KS.2.5 # Name: Astral Level: Labyrinth Goal: Pick up the crystal. Hints: Getting to it is the challenge. You'll need some sort of pathfinding code. Notes: The "A*" pathfinding algorithm is one useful way to find your path through a maze. Pathfinding isn't the most exciting skill, but it's necessary for getting around in any but the least cluttered environments. You may've already needed to develop something like this to pass earlier challenges. Note that the crystal's position varies, so be prepared to play this one repeatedly. # KS 2.6 # Name: Tetris Level: Proving Ground Goal: Put the four coins next to each other, in any arrangement of adjacent tiles. Notes: The point is to show an awareness of the concept "next to" or "touching". # KS 2.7 # Name: Sand Tetris Level: XIsle Goal: Put all of the coins onto sand, so that they're all there at once. Say something like "next task". Then arrange them into a line. Notes: This is about being able to do a sequence of actions in one puzzle. # KS 2.8 # Name: Key Level: Key Goal: Simply pick up the key, use it on the door ("use [dir]") to remove the door, then walk onto the grass. Notes: Demonstrates very simple item use. # KS 2.9 - Level 2 Final Challenge # Name: Fetch Quest Level: Fetch Quest Goal: Run FetchQuestBot as the first player. This bot will say "bring me [number] [X]", where [X] is a shape or color and [number] is 2-5. Put exactly that number of objects having that trait onto the wood platform beside the bot (and no other items), then say "Done". Make sure to eat enough to survive while doing this. Hints: The food is behind a locked door. Unlock it first. Be prepared to drop what you're holding if you get hungry while carrying an item that you're fetching. - Level 3: Master - Show that you can give and take simple orders; ask questions to get information that you then apply; survive by learning about your environment. # KS.3.1 # Name: In the Garden Level: Random Garden Goal: Stay alive for 100 turns by eating the food items that randomly appear. Notes: This is practice for the final test! You should be able to beat this consistently, except when the random layout makes it physically impossible, eg. by trapping you. # KS.3.2 # Name: Grail Challenge Level: Grail Challenge Goal: Pick up the correct object and no other. Which one? Ask the GrailBot what its shape, size and color are. It responds to questions of the form, "What [trait] is the grail?" with the form, "The grail is [X]." Hints: Choose wisely; the answer is random. Notes: Shows learning about objects in the game world based on conversation. # KS.3.3 # Name: Jailbreak Level: Jailbreak Goal: Have a simple command-following AI (a "Servbot" program is provided) control the first character. You will start in a jail cell! Tell the other character what to do to free you, and have them do it. Then get yourself out to the grass. Hints: The "Servbot" AI follows the basic commands, like "go e". You can use a custom AI if you like, so long as you're commanding it through the AI you're testing. Notes: This requires understanding the relationship between other people and objects, as well as a little cause-and-effect. # KS.?.4 # Name: Microtopia Level: Microtopia Goal: Survive for 1000 turns! A human player should be in the world with you. The human must ask all of the following questions, in any order, with this exact wording, after at least 100 turns: "What's your name? What do you see? What's my shape? Am I north of you? Am I near you? Have you eaten fish? What direction is fruit? What did I say first? Are you hungry? Are you holding anything? What shape is fish? What color is fruit? Are you near water?" Answer every question truthfully without evasion. Hints: Figure out that you can eat fish but that one color of fruit is poisonous. Learn what you've done and what's around you. Note that "What color is fruit?" is a trick question, since there are several kinds. "Are you hungry?" should return some measure of your energy level. "Am I near you?" should be based on some orthogonal distance, eg. "near" means < 3 tiles away. Notes: Beat this level and you've proven you understand the simple world around you, and can interact with it, learn from your experience, and explain your knowledge. You've mastered Morningside! # KS.3.5 # Name: Graduation Level: Graduation Goal: Have a human player. Ask nicely for the human to drop the key. Then take it, open the door, and step through. The human must take the key, then only drop it once asked. Notes: You win! * 4. FAQ * Q: How do I use Morningside? A: See the main Read Me file for that. Basically, start up the server and then your interface or AI program, making sure you have a "login_[name].txt" file in the player_io directory and that your interface program has a "path.txt" file pointing to where that player_io directory is on your computer. Q: Why should I care? A: Artificial Intelligence has come a long way, but still lacks some crucial features like general-purpose reasoning. AI programs tend to either be specialized "expert systems" that do one cool thing like driving a car, or "chatterbots" that pretend to converse but have no clue what they're saying. An AI that follows the development path implied by these tests will be able to do various things and to speak intelligently about what it's doing. Applications of a versatile AI technology include games, better expert systems, and ultimately true, human-like AI. Q: Why test these skills and not others? A: The tested skills cover a mix of conversation ability, action and learning. An AI that beats these challenges can survive in a simple world, learn from its environment, and converse in human language about what it knows. It will have a connection between what it says, does, and needs, which will make its conversation and actions meaningful. Q: Aren't video game characters already this smart? A: Mostly not. "Game AI" is focused on entertaining the player, not making a truly smart machine. So there's a lot of fakery behind the scenes. Game characters are pretty good at a few specific things like hunting you in a maze... if they're given human-placed waypoints and all they have to do is run and jump, and their social interaction consists of I KILL YOU. Put a modern game character into a real robot body and it couldn't do much. Morningside is a step towards a more general-purpose AI. Q: Is this series of puzzles like the Turing Test? A: A little bit, in that it tests various aspects of an AI. It's more broad-ranging than the Turing Test, which only involves typed conversations. It's also more honest and objective, in that no lying is expected of an AI. There's no reward for an AI pretending to be something it's not, or to cleverly dodge simple questions. And it's possible to design tests that an AI will unambiguously pass or fail, not subject to a judge's feelings. (Mine are a little fuzzy, but can be given more rigor if needed.) Q: Why such a simplistic world? A: "Keep It Simple, Stupid!" My earlier efforts were muddied by attempts to add cool features like 3D, to the point where I spent hours trying to get realistic friction and parabolic flight motions. Alternative worldsim programs like "Multiverse" and "OpenSimulator" focus on 3D and other features that are unnecessary and overly complex for demonstrating basic skills. With Morningside, you can get started playing easily with the human interface, and if you try writing an AI, the interface is dead simple. Q: What would you expect to build in a successor world to Morningside? Q: It'd still be tile- and turn-based. Slightly 3D in its physics but still drawn simply, probably with isometric tile graphics. Characters face a particular direction. They have distinct body parts that can be looked at, damaged, or used to hold/wear items (several items in all). Characters have limited information, only what they can see/feel/&c. Character bodies have multiple stats like food, water, energy and health. Height, climbing, swimming, and stacking of objects exist. Items can be used on each other to represent actions like repair. Tests would involve survival, pathfinding that requires actions like donning dive gear, multi-step processes like cooking, tracking objects out of sight, interaction with other AIs to do things like giving orders and sharing knowledge, interaction with in-game devices such as books (eg. to look up info from them), and basic creative thought like making up tiny stories or naming objects. * 5. Version History * 2009.9.19: Second released version. Added use of look command to KS00. 2009.9.26: Heavily rewritten and expanded.