Google is always interested in finding solutions of complex computing problems, and what’s more complex than artificial intelligence. Probably, nothing. There are already tons of companies working on AI, from small startups to giant organizations. The topic has been trending on the internet, especially after Elon Musk’s tweet mentioning “Mark Zuckerberg’s knowledge about AI is limited”. A couple of days later, a false news that ‘Facebook shut down AI‘ gone viral.
Bygones be bygones! The search giant company, Google is now teaching artificial intelligence to predict the future. Google’s AI research division, Deepmind is developing AI algorithms that can “imagine” and predict what will happen in the next move.
Deepmind introduces the architectures that provide effective ways for agents to learn and build moves in order to get the best outcome. These complex and imperfect models can adopt flexible approach to exploit their imagination.
Imagination Augmented Agents
The agents introduced by Deepmind are based on “Imagination encoder” – a neural network that learns to extract data useful for future decision, and ignores things that are not relevant. These are the following features of agents –
Interpret Internal Simulation: Agents capture the environmental dynamics (including imperfect dynamics) by interpreting their internal simulation.
Use Imagination Efficiently: Agents adapt some imagined trajectories to suit the problem. These trajectories contain helpful clues, even if they don’t certainly result in perfect outcome.
Learn Different Strategies: Agents are capable to learn several methods to build plans. They do this either by restarting from scratch or contributing a current imagined trajectory. Also, they can adapt different imagination models, with different computational costs and accuracies.
This is much different than AlphaGo – an AI agent that can plan for future moves under a confined set of rules. The new agents will apply the lessons (learned from dynamic environment) to the real world.
Testing the Architectures
The proposed algorithm is tested on multiple tasks, including a spaceship navigation game and Sokoban puzzle game. These games provide a perfect environment for testing new agents as both games require forward planning as well as reasoning.
To stop agents trying different strategies (before testing them in real surrounding) developers limited trial-and-error in a manner that agents can only try it once.
The above video shows an agent playing Sokoban from a pixel representation. The agent is unaware of game’s rule, and based on five possible futures, it decides what action to take
The algorithm is applied to spaceship game – green and blue line indicate imagined trajectories while red depicts trajectories executed in the environment.
With the addition of “manager” entity that helps to build a plan, the agents learn to solve problems more efficiently, in fewer steps. For example, in spaceship game, it can differentiate between situations where the gravitational force is weak or strong.
Conclusion and Future
In both tasks, agents outperform the imagination-less baselines. This shows that agents can learn with less experience, deal with imperfections in modeling the environment and can solve problems with fewer imagination steps as compared to traditional search methods such as Monte Carlo tree search.
According to the Deepmind, a lot of further analysis would be required to provide scalable solutions to other problems, and develop agents that can use their imagination to plan for the future in different scenarios.