Link list: Autonomous Driving¶

(Literature / Blogs / Code)

General Introduction to autonomous driving:¶

• http://www.computervisionblog.com/2015/03/mobileyes-quest-to-put-deep-learning.html
• http://spectrum.ieee.org/automaton/robotics/artificial-intelligence/how-google-self-driving-car-works

• https://www.youtube.com/watch?v=Ywh0votSJxk

• Proposal of a Basic Architecture on slide 18 of http://www.umtri.umich.edu/sites/default/files/Ryan.Eustice.UM_.Engineering.IT_.2015B.pdf

• DeepDriving: Learning Affordance for Direct Perception in Autonomous Driving http://deepdriving.cs.princeton.edu/

• Amnon Shashua CVPR 2016 keynote: Autonomous Driving, Computer Vision and Machine Learning: https://www.youtube.com/watch?v=n8T7A3wqH3Q (min 38: deep reinforcement learning)

• http://bdd.berkeley.edu/

• https://www.youtube.com/watch?v=1L0TKZQcUtA&feature=youtu.be&list=PLrAXtmErZgOeiKm4sgNOknGvNjby9efdf

• High level explaination of commercial companies:

  • http://360.here.com/tag/3-steps-to-automation/
  • https://waymo.com/tech/

• https://www.linkedin.com/pulse/deep-learning-neural-networks-self-driving-cars-claudio-hernandez/

Infrastructure: Robot Operating System (ROS)¶

• used for the Udacity Car
• ROS for autonomous driving at BMW: https://www.youtube.com/watch?v=z9WzHXfYqFI
  • Why ROS? starts at 7:50
  • Using Matlab Simulink with ROS at 14:50 see also https://www.mathworks.com/products/robotics.html
• http://www.ros.org/
• https://github.com/udacity/self-driving-car/tree/master/steering-models/steering-node
• http://wiki.ros.org/gazebo

• Deep Reinforcement Learning for Simulated Autonomous Driving

Supervised Learning / Imitation Learning for Steering (Problem: Distribution mismatch):¶

• https://medium.com/udacity/teaching-a-machine-to-steer-a-car-d73217f2492c#.40lmh92ps
• https://devblogs.nvidia.com/parallelforall/deep-learning-self-driving-cars/
• https://arxiv.org/pdf/1604.07316v1.pdf

• https://www.youtube.com/watch?v=kl_G95uKTHw&index=2&list=PLkFD6_40KJIwTmSbCv9OVJB3YaO4sFwkX

• Explaining How a Deep Neural Network Trained with End-to-End Learning Steers a Car

• Deep Steering: Learning End-to-End Driving Model from Spatial and Temporal Visual Cues

Some examples of Deep Reinforcement Learning for autonomous driving:¶

• https://yanpanlau.github.io/2016/10/11/Torcs-Keras.html
• https://www.youtube.com/watch?v=0xo1Ldx3L5Q&feature=youtu.be
• https://www.youtube.com/watch?v=QK0LxA8FWq4
• https://www.youtube.com/watch?v=QK0LxA8FWq4&t=127s
• https://arxiv.org/pdf/1602.01783.pdf
• https://www.youtube.com/watch?v=FovLsAFiIJU
• Following a trail: http://people.idsia.ch/~giusti/forest/web/
• https://arxiv.org/pdf/1704.02532.pdf

Virtual Environments:¶

• Udacity's Self-Driving Car Simulator: https://github.com/udacity/self-driving-car-sim
• Torcs:
  • http://torcs.sourceforge.net/
  • J. Koutnık, G. Cuccu, J. Schmidhuber, and F. J. Gomez. Evolving large-scale neural networks for vision-based torcs. In FDG, pages 206–212, 2013
  • stochastic policy reinforcment learning with keras and torcs: https://yanpanlau.github.io/2016/10/11/Torcs-Keras.html

• Vdrift http://wiki.vdrift.net/index.php?title=About_the_project

• Learning a Driving Simulator, https://arxiv.org/pdf/1608.01230.pdf

---

Datasets:¶

• http://www.cvlibs.net/datasets/kitti/ - a lot of datasets for
• http://research.comma.ai/
• https://www.udacity.com/self-driving-car
• https://github.com/udacity/self-driving-car/
• http://www.vision.caltech.edu/malaa/datasets/caltech-lanes/

Hardware:¶

• Computing:

  • Nvidia Drive PX: http://www.nvidia.de/object/drive-px-de.html
  • Nvidia Jetson TX1: http://www.nvidia.com/object/jetson-tx1-dev-kit.html

• Sensors:

  • Cameras:

  • Logitech C920 see page 42 of https://www.dropbox.com/s/q34bi7t0udms01x/lecture3.pdf?dl=1 https://www.youtube.com/watch?v=U1toUkZw6VI&index=3&list=PLrAXtmErZgOeiKm4sgNOknGvNjby9efdf

  • Lidar:

  • http://quanergy.com/

  • http://www.golem.de/news/autonomes-fahren-laserscanner-fuer-den-massenmarkt-kommen-1701-125585.html

  • Example Sensors of Udacity Car: https://medium.com/udacity/were-building-an-open-source-self-driving-car-ac3e973cd163#.ql2e0qkpk

Moocs / Books etc.:¶

• Deep Learning for Self Driving Cars
• https://de.udacity.com/course/artificial-intelligence-for-robotics--cs373/
• http://www.probabilistic-robotics.org/
• Convolutional Neural Networks for Visual Recognition: http://cs231n.stanford.edu/syllabus.html
• https://www.fun-mooc.fr/courses/inria/41005S02/session02/about
• https://de.udacity.com/course/self-driving-car-engineer-nanodegree--nd013/
• Reinforcement Learning:
  • http://www0.cs.ucl.ac.uk/staff/d.silver/web/Teaching.html
  • https://webdocs.cs.ualberta.ca/~sutton/book/the-book.html
  • RL with Python and openAI (draft -in progress): http://christianherta.de/lehre/dataScience/machineLearning/reinforcementLearning/Reinforcement_Learning.php
• https://medium.com/self-driving-cars/term-1-in-depth-on-udacitys-self-driving-car-curriculum-ffcf46af0c08#.cie3lrv5k

Other Reading and Link Lists:¶

• https://handong1587.github.io/deep_learning/2015/10/09/dl-and-autonomous-driving.html
• Paper List of http://www.cs.toronto.edu/~urtasun/courses/CSC2541/CSC2541_Winter16.html

Possible subtasks of autonomous driving:¶

• Sensing / Perception:

  • Lane detection and Tracking:
  • https://arxiv.org/pdf/1602.03409.pdf
  • http://www.cv-foundation.org/openaccess/content_cvpr_2016_workshops/w3/papers/Gurghian_DeepLanes_End-To-End_Lane_CVPR_2016_paper.pdf
  • Object Detection and Tracking:
  • Obstacle (Object) Detection
  • Sign recognition

  • YOLO/ROLO: http://pjreddie.com/darknet/yolo/ http://guanghan.info/projects/ROLO/

  • Semantic Segmentation:

    • http://mi.eng.cam.ac.uk/projects/segnet/#publication
    • https://people.eecs.berkeley.edu/~jonlong/long_shelhamer_fcn.pdf
    • https://github.com/tkuanlun350/Tensorflow-SegNet

  • Off-Road Obstacle Avoidance through End-to-End Learning: http://yann.lecun.com/exdb/publis/pdf/lecun-dave-05.pdf

• Motion Planning:

  • http://www.sciencedirect.com/science/article/pii/S0968090X15003447
  • http://rkala.in/autonomousvehiclesvideos.php
  • https://www.youtube.com/watch?v=itiViN8LJFY
  • https://arxiv.org/pdf/1604.07446.pdf

• Simultaneous Localisation and Mapping / Odometry / Dead Reckoning / Visual Odometry / Ego Motion

  • Bayesian Filter: Kalman Filter, Particle Filter, Probabilistic Graphical Models
  • Ego-Motion: www.mobileye.com/wp-content/uploads/2011/09/EgoMotion.pdf
  • SLAM: Simultaneous Localisation and Mapping
    • ORB-SLAM2: https://github.com/raulmur/ORB_SLAM2
    • http://www.computervisionblog.com/2016/01/why-slam-matters-future-of-real-time.html
    • https://wp.doc.ic.ac.uk/sleutene/2016/02/04/release-of-okvis-open-keyframe-based-visual-inertial-slam/
    • http://www.dis.uniroma1.it/~labrococo/tutorial_icra_2016/
    • FastSlam: http://robots.stanford.edu/papers/montemerlo.fastslam-tr.pdf
      • Kitt, Bernd Manfred, et al. "Monocular visual odometry using a planar road model to solve scale ambiguity." (2011)
      • Konda, Kishore, and Roland Memisevic. "Learning visual odometry with a convolutional network." International Conference on Computer Vision Theory and Applications. 2015.

• Sensor fusion:

  • http://www.kdnuggets.com/2015/10/data-science-iot-sensor-fusion-kalman-filters-part1.html

• Environment models / Scene Understanding:

  • 3D Traffic Scene Understanding from Movable Platforms: http://ttic.uchicago.edu/~rurtasun/publications/geiger_et_al_pami13.pdf
  • Creating an environment model -> planing where is't save to drive: https://arxiv.org/abs/1608.01230
  • 3D Perception and Planning for Self-Driving and Cooperative Automobiles: http://www.mrt.kit.edu/z/publ/download/2012/StillerZiegler2012SSD.pdf

Teaching and student projects at other universities:¶

• https://www.viktoria.se/career-opportunities/opportunity-for-master-thesis-reinforcement-learning-for-end-to-end-autonomous-driving
• CARMA: https://web.stanford.edu/~anayebi/projects/CS_239_Final_Project_Writeup.pdf
• http://islab.hh.se/mediawiki/Reinforcement_learning_for_End-to-End_autonomous_driving
• http://cs231n.stanford.edu/reports2016/123_Report.pdf

Code¶

• https://github.com/udacity/self-driving-car
• https://yanpanlau.github.io/2016/10/11/Torcs-Keras.html
• https://github.com/nivwusquorum/tensorflow-deepq
• https://github.com/commaai/openpilot
• https://github.com/commaai/research
• https://github.com/dat-ai/steering-model