JetBot User Guide

Collision-avoidance Document

The collision-avoidance demo of the NVIDIA JetBot open source project is used here. The autonomous obstacle avoidance function of JetBot is divided into three steps: data collection, model training, and autonomous obstacle avoidance.

Open the "data_collection.ipynb" file under the directory of Notebooks/collision_avoidance/.

Create a camera connection, display the camera screen on the web page, and be careful not to modify the resolution settings of the camera.

Create a dataset directory to store the images we collect next. Create two folders in the dataset directory, blocked and free. The blocked folder will be used to store the picture of the obstacle avoidance scene, and the free file will be used to store the picture of the unblocked scene. A try/except structure is used here to avoid file overwriting. If these two folders already exist in the path, the program will report an error that the folder already exists, and will not overwrite the old directory.

Create two buttons and two display boxes. The buttons are used to add obstacle avoidance scene pictures and smooth scene pictures. The display boxes respectively display the number of currently stored pictures. If you are running the script for the first time, the number of pictures should be 0. Don't press the button hard, the button here is just for demonstration and has no actual effect.

Write the button associated function. Among them, save_snapshot(directory), this function is used to save the picture, it will be called by the key-triggered program, the function saves the current camera image to the corresponding path; the save_free() function, corresponding to the add free green button above, the function Call the save image function in , save the current camera image to the free folder, and update the number of free images; the save_blocked() function saves the current image to the blocked folder and updates the number of images.

Now, the obstacle avoidance scene and the clear scene graph can be collected. When the car needs to turn when encountering an obstacle, you can press the add blocked red button, and press the add free green button in the unblocked scene; collect at least 100 obstacle avoidance maps and unblocked pictures for each.

Open Notebooks/collision_avoidance/train_model.ipynb.
Import the PyTorch library. PyTorch is a common deep learning library. If you are interested in it, you can learn about it on Baidu or Google.

Decompress the packet. Note: If you are doing model training on the same car as me, please skip this step, because your data is already saved in the current directory, if you run the program below, it will keep stuck; If you are training on a different car, you need to copy the data package you collected and packaged before to the current directory, and then run the program below to decompress it.

Divide the data packets into two groups, namely the training group and the test group. The test group is used to verify the accuracy of the model.

Define a neural network. In transfer learning, we can use pre-trained models for new trainning tasks, so that we can directly use some of the learned functions in the pre-trained models, and then train new tasks, which will save a lot of effort. To sum up, it is like teaching a middle school student high school knowledge, which is easier and faster than teaching a child who has no primary school knowledge.

Use the downloaded model to judge the results: avoid obstacles and go straight. That is, two labels are required.
Convert the model to run on the GPU.

Model training. 30 sets of training are performed on the model, and a model file of best_model.pth will be generated after the training. Wait for the training to finish.

After obtaining the model through the above training, we start the obstacle avoidance test.
Open the live_demo.ipynb file in the Notebooks/collision_avoidance path. Initialize the PyTorch model.

Load the model into CPU memory and prepare to pass it into the GPU device for calculation.

Display the camera image on the web page, and use a slider to display the obstacle avoidance value of the current image.

According to the obstacle avoidance value, operate the car to go straight or turn (obstacle avoidance). If the speed of the car is too fast when avoiding obstacles, you can reduce the parameter value in robot.forward() to reduce the forward speed of the car, and reduce the parameter value in robot.left() to reduce the turning range of the car.

If your car is not running properly, or you want to stop the car, you can continue to run the following program.