You need to have Unity installed, and all python modules listed in the Requirements section below.

The Donkey Car simulator is created with Unity. There are 3 Unity scenes available (created by Tawn Kramer) for training now, which are generated roads, warehouse, and Sparkfun AVC. Before we run the RL training script, we have to either build the Donkey Car Unity environment ourselves (need to install Unity) or download the pre-built environment executables below:

Linux: donkey.x86_64 | MacOS: donkey.app
For windows, the executables are already present inside the src folder.

Then place the executable inside the ReinforcementLearning-UnityCar/src folder.

You need python 3.4 or higher, 64 bit. You can create a virtual env if you like:

virtualenv -p python3 env
source env/bin/activate

Install required dependencies:

pip install -r requirements.txt

This will install Donkey Gym and Donkey Car packages from source.

If you have an cuda supported GPU - probably NVidia

pip install tensorflow-gpu

• We have provided the DDQN algorithm in both jupyyter notebook(DonkeyCar.ipynb) and Python(ddqn.py). They will save the trained model in save_models folder.
• aws.h5 model is trained for 10,000 episodes on AWS. save_model.h5 is trained only for 100 episodes.
• Open the unity simulator while training/testing to see the progress run simulated on the unity simulator.
• If the script runs successfully, you should see some printouts in the command prompt with the training statistics (e.g. episode number, action, reward, etc).
• Notice that by default a Unity GUI will be launched where you can see the Donkey car being trained. If you want to train in headless mode (i.e. no GUI), you can edit ReinforcementLearning-UnityCar/src/donkey_gym/donkey_gym/envs/donkey_env.py and set headless flag to True.

1. Start the prediction server with the pre-trained model.

cd ReinforcementLearning-UnityCar/src
python predict_server.py --model ../save_model/aws.h5

2. Load the Unity project sdsandbox/sdsim in Unity. Double click on Assets/Scenes/main to open that scene.

3. Hit the start button to launch. Then the "Use NN Steering".
   You will see the output in unity like below: