Instructors: Anastasios Lekkas (NTNU) and Francesco Scibilia (Equinor/NTNU)

Updated Info: The first lecture will take place on Monday 5 October between 12:15-14:00 at S4 (Sentralbygg 1).

Prerequisites: For the academic part, it's useful to have some past knowledge on how to train neural networks.

This specialization course will present an introduction to autonomous robots from both the academic and industrial viewpoints. For the academic part, emphasis will be given to recent advances in deep reinforcement learning, which combines deep neural networks with reinforcement learning to provide a framework for discovering suitable control actions (policies) and addressing complex tasks without explicit programming. For the industry‐focused lectures, aspects of artificial intelligence and autonomous robotics systems will be considered from industrial domain perspectives as inspection and maintenance.

The lecture plan below is from last year. For the 2020 lectures, it will be enhanced with additional algorithms and details regarding their implementation.

Lecture 1 (October 5, 2020, 12:15 - 14:00, S4 (Sentralbygg 1))

• Course introduction: Objectives, overview of lectures, assignments, exam info.
• Autonomy in the scope of the course. An introduction to reinforcement learning and its connection with dynamic programming.
• Markov Decision Processes and problem formulation

Lecture 2 (October 12, 2020, 12:15 - 14:00, S4 (Sentralbygg 1))

• Optimal policies in discrete environments (Part 1): Value iteration and policy iteration
• Optimal policies in discrete environments (Part 2): Q-learning and SARSA

Lecture 3 (October 19, 2020, 12:15 - 14:00, S4 (Sentralbygg 1))

• Deep learning and deep reinforcement learning.
• The DQN algorithm.
• Introduction to policy gradient algorithms for continuous action and state spaces. REINFORCE algorithm

Lecture 4 (October 26, 2020, 12:15 - 14:00, S4 (Sentralbygg 1))

• Deep deterministic policy gradients (DDPG)
• Proximal policy optimization (PPO)

Lecture 5 (November 2, 2020, 12:15 - 14:00, S4 (Sentralbygg 1))

Artificial intelligence in autonomous robotics systems: what is an actionable definition in an industrial setting. Different levels of autonomy. Hierarchical architecture (Sense‐plan‐act and behaviorbased substrates) and autonomy layers. Data and connectivity aspects for autonomy.

Lecture 6 (November 9, 2020, 12:15 - 14:00, TBD)

AI Robotics, market value chain considerations. Operational considerations on implementing autonomous systems in industrial applications as: business models, system integration, solutions fit to existing customer infrastructure and systems, emerging industrial information standards aspects.

Lecture 7 (November 16, 2020, 11:45 - 14:00, online(details to come) )

• 11:45-12:00 Connecting and Intro – Francesco
• 12:00-12:50 Guest lecture + QA – Steffan Sørenes, Leading Advisor IT Architecture at Equinor
• 12:50-13:10 Break
• 13:10-14:00 Guest lecture + QA – Fakhri Landolsi, Manager Data Science at Equinor

November 25-26.