Thursday, September 30, 2010

The Ambiguity of Defining Task Environments

Consider a taxi driving agent. This agent is clearly in a multi-agent environment because of the other drivers on the road. They typically prevent the taxi-driver from doing what would be optimal for it (especially when they create a traffic jam). But how about a puzzle solving agent that gets distracted by a bird singing his song? We could argue for both single or multi-agent environments depending on how much the bird irritates the puzzle solving agent.

Another example. Consider a poker playing agent. Poker involves picking cards from a deck. Since these cards are randomly shuffled there seems to be a random element to the game. So is this environment stochastic? It depends. If you model the deck of cards as part of the state space, then the state + the action deterministically determine the next state. However, if the state definition does not involve the order of the deck of cards, then this can be viewed as a random draw which would then argue for a stochastic environment.

Or this example. A part picking robot confronted with a new problem every time can be classified as operating in an episodic environment. However, it may wear in the process or use up battery power in which case it really is sequential.

Chess has a discrete number of states. Or not? If we define a state to be the exact position of every piece on the board then it would be continuous.

Bottom-line: the task environment crucially depends on one's definition of state, action and utility function.