Last week I gave an AI agent a baseball bat, and tried to teach it to hit a ball. At first it behaved rather randomly. That was mostly due to the fact that the observations and decisions were not made frequently enough. The agent was reacting to an observation which was too outdated. Imagine trying to hit a ball while blinking very slowly. I retrained the agent with a higher observation frequency. Even with the same number of training steps, the new agent was doing much better. With 10 million more training steps, it was able to slam the ball twice as far as the original.