During my work towards my university degree I completed a thesis on development of an Artificial Intelligence agent to play an open source version of Mario. The agent was based on an A* implementation and my research also went into developing an agent based on Monte Carlo methods.
The video below shows the actual game on the left and the simulator on the right. The agent simulates the world ahead of time based on current observations. The agent gets no more information that a human wouldn't get playing the game on a TV. Using this data the simulator steps ahead in time allowing for an appropriate path to be planned. This plan is updated whenever new information is received and replanning is required.