= Self Driving Vehicular Project = **Team:** Michelle Gutwein and Christopher Lee [[BR]] **Advisors:** Ivan Seskar and Jennifer Shane[[BR]] **Project Description & Goals:**[[BR]] Build and train miniature autonomous cars to drive in a miniature city.[[BR]] **Technologies:** ROS (Robot Operating System), Pytorch ---- == Week 1 Read through the [https://docs.google.com/document/d/1KWIyvGMnB2nboosv068qp-X2BbNm9y4tUnWLzKbz6bI/edit?tab=t.0#heading=h.gz0otq9rwrdy manual] provided to us. [[BR]] \\ [https://docs.google.com/presentation/d/1ZJatipqqiaRQ06gbw_t5Jbho-dZOFxXZS6fXx_FxIIc/edit?slide=id.p#slide=id.p Week 1 slides]// == Week 2 Learned what ROS is and how to use it[[BR]] Also got to know the car, RASCAL[[BR]] \\ [https://docs.google.com/presentation/d/1VsER6V9A0wsgOowDd7vA26k3LNe6V8tMvsMXYWiOBVo/edit?slide=id.p#slide=id.p Week 2 slides]// == Week 3 Read through all of the code in the [https://gitlab.orbit-lab.org/self-driving-car-2023/upcar/-/tree/main/catkin_ws/src/rascal/src?ref_type=heads Gitlab] [[BR]] Learned how to record and edit data on the web display[[BR]] \\ [https://docs.google.com/presentation/d/1NL-Xl_2XD7dQo6XJMo8_wsapmv6N-sxQl58-IZ-0_L0/edit?slide=id.p#slide=id.p Week 3 slides]// == Week 4 Learned how to train a model[[BR]] Trained and tested the model on simple paths[[BR]] \\ [https://docs.google.com/presentation/d/10KDlyMXHnCqYHTe5A6nhevLICf3jup91HIhf3qXCt6U/edit?slide=id.g369586f805a_0_10#slide=id.g369586f805a_0_10 Week 4 slides]// == Week 5 Trained model on making right and left turns in the city environment[[BR]] Started training on turns at the intersection[[BR]] Created a new model called gps which will allow users to give the car commands on the direction to turn at an intersection before running the car[[BR]] \\ [https://docs.google.com/presentation/d/1_Utb2f8qbcjIbTonJ2XXRmijsZU6iXQUkLB3nwDeOGs/edit?slide=id.p#slide=id.p Week 5 slides]// == Week 6 Collected new data on making wider right and left turns in the city environment[[BR]] Continued training on turns at the intersection[[BR]] Updated the documentation to be more clear [[BR]] \\ [https://docs.google.com/presentation/d/1HPqGgHc9MtS-kRgKqUCfaMNVU0m56_TEW2BSjwRlzpk/edit?slide=id.p#slide=id.p Week 6 slides]// == Connecting to Rascal Through ORBIT: [[BR]] 1. Make a reservation on your orbit account under the **intersection** bed [[BR]] 2. Open a new terminal and type **ssh @console.intersection.orbit-lab.org** [[BR]] 3. You should see the 'welcome to ORBIT-LAB' sign [[BR]] 4. Enter ** ** for the rascal 5. In a separate terminal, enter **ssh rascaltrain@srv1** for the rascal server [[BR]] [[BR]] Through Zerotier: [[BR]] 1. Sign into Zerotier under the account **rascalstonesdc-at-gmail.com** password: **si2022sdc**[[BR]] 2. Connect to the network: rascalstonesdc's 1st network **(ID: d3ecf5726d1a9fcd)**[[BR]] 3. Now you can open a terminal and connect to the IP address of the rascal using the command: **ssh rascal@10.244.114.238[[BR]]** 4. Next, run the command: **roslaunch rascal pure_pursuit.launch** to launch all the nodes[[BR]] 5. In a web page type **10.244.114.238:5000** to open the web display where you can record and edit data for training[[BR]] [[BR]] 6. In a separate terminal, enter **ssh rascaltrain@10.244.176.240** to connect to the rascal server [[BR]] == Collecting Data To record data:[[BR]] 1. Press **bagrecorder/enable** on the web display to start recording [[BR]] 2. Use the joystick or keys on the keyboard (wasd) to move the car around[[BR]] 3. Press **bagrecorder** again to stop recording [[BR]] 4. Open data **opt/upcar/dataProcessing** and type **python bag2AllData.py** in the terminal to access all recorded sessions[[BR]] 5. Select the number that you want to download (should be the most recent one)[[BR]] 6. After it finishes downloading you can type the command: **roslaunch rascal sim.launch** which will upload that session to the web display[[BR]] 7. In the web display you can play the video to see what the camera recorded (you might need to refresh the page to see the session come up)[[BR]] 8. You can now edit the data that will be used for training[[BR]] 9. To save data, run the **data_loader/smoothen** command == Deleting Bad Data Segments When portions of the recorded data are messy or irrelevant for training, they can be excluded to improve model performance [[BR]] 1. In the web display you can click **replayer/pause-play** to run through all of the data segments [[BR]] 2. To jump between different segments that the web display automatically divides the data into, click **replayer/nextSeg** or **replayer/prevSeg** 3. When you get to a segment you want to delete, pause the run through and hit **replayer/editMode** [[BR]] 4. Select the data you want to delete by using the **replayer/jumpEnd** and **replayer/jumpStart** buttons [[BR]] 5. Select **replayer/editMode** again to highlight the part that's going to be deleted [[BR]] 6. Click **replayer/deleteSelection** to delete the segment [[BR]] == Training Rascal We used the fisheye model to train on [[BR]] 1. Open a new terminal and connect to the rascal server's IP address by using this command: **ssh rascaltrain@10.244.176.240** (or use ORBIT) [[BR]] 2. Open **/upcar/dataProcessing** and type **./copyFromCar.sh** which will get the files from the car and bring them over to the server [[BR]] 3. It will ask you to select a host. Choose **1) rascal@172.24.114.238** which is the car's IP address [[BR]] 4. If it asks for a password it's always **si2022sdc** [[BR]] 5. It will ask you which sessions you want to copy over (pick the session version of the files you want that says smooth) [[BR]] 6. If you go to **/sessions** and type **ls** your chosen files should be listed there [[BR]] 7. Go back to **/upcar** and move your data to /rascaltraining/data: **mv dataProcessing/sessions/ rascaltraining/data [[BR]] 8. If you go into **/rascaltraining/data** you should see your file listed [[BR]] 9. Go back to **/rascaltraining** and type the command: **python trainModel.py -m fisheye --use_labels 1.0 -s labels** [[BR]] 10. Now your model is being trained [[BR]] 11. To stop training press enter [[BR]] 12. Remember to stop training when the validation loss doesn't seem to be decreasing by a significant amount anymore to avoid overfitting [[BR]] 13. To check what number model you just created, go to **/savedModels** and type **ls** (yours will be the most recent number) 14. To push your newly made model to gitlab, type **git add -f ** [[BR]] 15. Next type the command **git commit -m ""** [[BR]] 16. Next type the command **git push origin main** [[BR]] 17. Finally, back in the car's terminal go to **/savedModels** and type **git pull origin main** [[BR]] You're now ready to start testing! [[BR]] == Testing Rascal To test rascal: [[BR]] 1. Go to /upcar in the car's terminal (not the server) [[BR]] 2. Enter **./launch_with_model.sh pure_pursuit** [[BR]] 3. Open up the web display [[BR]] 4. Run these commands in the web display: [[BR]] b. sim_camera/set_enable (set to true) [[BR]] c. ml/imgDisp (set to true) [[BR]] d. ml/0/enable (click enable) [[BR]] Watch the car move on it's own! [[BR]] Make sure it doesn't crash!