Reinforcement Learning (EIM)

Thu, 01 Sep 2022 00:00:00 +0000

Reinforcement Learning with Application to Autonomous Systems (EIM)

Learning Objectives

Students

gain insight into the theory and applications of reinforcement learning
learn to analyze the challenges in a reinforcement learning application and to identify promising learning approaches
are able to assess for which problems reinforcement learning is particularly well suited and which disadvantages exist with regard to this.
understand, explain, and classify relevant basic concepts.

More information as well as lecture noted are available in the .

Lecture

#	Name	Summary
1	Markov Decision Process	Markov Processes, Markov Reward Processes, Markov Decision Processes, Bellman Expectation Equations, Bellman Optimality Equations
2	Dynamic Programming	Policy Evaluation, Policy Iteration, Optimal Policy, Generalized Policy Iteration, Value Iteration
3	Monte Carlo Methods	Monte Carlo Prediction, Monte Carlo Policy Evaluation, Monte Carlo Control, Exploration-Exploitation Tradeoff
4	Temporal Difference Learning	On-policy TD Control, Off-policy TD Control, SARSA, Q-Learning
5	Function Approximation	Incremental Methods, Gradient Descent, Prediction Algorithms for the Linear Case, Control Algorithms for the Linear Case, Batch Methods
6	Policy Gradient Methods	Policy-based Methods, Deterministic vs. Stochastic Policies, Gradient-based Estimator, Monte-Carlo REINFORCE, Actor-Critic Architectures
7	Introduction to Deep Learning	Components of Deep Learning Architectures, Activation Functions, Output Functions, Typical Applications, Image Classification, Object Segmentation, Object Detection
8	Beyond DQN	Double DQN, Dueling DQN, Rainbow DQN, Trust Region Policy Approximation, Soft Actor-Critic

Lab Work

For the majority of lab session, we will use the OpenAI Gym environment, click for more information. Besides implementing deep reinforcement learning agents on your own, the deep reinforcement learning library will be used.

#	Name	Summary
1	Tic-Tac-Toe	Tabular Solution for the Tic-Tac-Toe Game using a simplified temporal difference solution.
2	Frozen Lake	We solve the frozen lake environment with Policy as well as Value Iteration
3	Blackjack	Monte Carlo Methods are reviewed with the Blackjack environment
4	Taxi	Homework assignment to implement SARSA to solve the Taxi environment
5	Lunar Lander	First implementation of function approximation with a simple neural network
6	Racecar	More advanced deep learning networks in combination with more advanced strategies, e.g. Dueling DQN, are being evaluated.
7	DonkeyCar	DonkeyCar offers a more realistic simulation of a racecar using the unity engine. We use the simulator to train a Reinforcement learning agent to drive autonomously.
8	RC Car	We use the DonkeyCar-trained agent to evaluate the performance in a real remote-controlled car.

Wireless Communications (EIM)

Thu, 01 Sep 2022 00:00:00 +0000

Wireless Communications (EIM)

Subject-specific Competencies

Students

have acquired a fundamental knowledge about key aspects of the physical layer of contemporary wireless communication systems.
design and analyse the signal processing applied in modern digital radio communication systems under different constraints

Methodological Compentencies

Students

will be able to understand the meaning or relevant features and aspects of mobile and wireless communication standards and make informed judgments about their suitability for different communication tasks.
will be able to evaluate overall system performance of communication systems.

Interdisciplinary Competencies

Students

can evaluate solutions in highly complex environments
are able to acquire detailed knowledge for solving specific problems on their own
learn to discuss and derive solutions within a team

Course Content

Introduction: History of wireless communications. From classical radio to modern cellular communication systems
Propagation, Antennas & Wireless Channels: From single antenna channels to multipath multi-antenna channel models, propagation models, path loss, multipath effects
Digital Modulation in the Baseband: Introduction of basic concepts. N-QAM and N-PSK modulation. Metrics and Limits of Modulation schemes
Multi-antenna Techniques (MIMO): Receive and transmit diversity. Beamforming, Detection
Multi-user Systems: OFDMA, CDMA, TDMA, SDMA
Cellular Systems: 5G and beyond, 5G key technologies
Current research topics: Terahertz signal transmission, AI core networking and network management

More information as well as lecture noted are available in the .

Lecture

#	Name	Summary
1	Fading Channels	Single-antenna systems, path loss, shadowing
2	Multipath Channel Models	Doppler Shift, Rayleigh and Rician fading, Jakes model, WINNER Multipath channel models
3	Modulation	Common modulation formats e.g. ASK, PSK, QAM, average error probability, coherent and differential detection
4	Diversity	Multi-antenna setups, Diversity Combining, V-BLAST
5	Multicarrier Modulation	OFDM basics, OFDM transmitter and receiver, adaptive rate and power allocation
6	Channel Coding	Block Codes, convolutional codes
7	MIMO Systems and Detection	Maximum Likelihood detection, Linear Receivers, ZF and MMSE
9	MIMO Beamforming	Analog and digital beamforming, max-SNR MMSE beamforming
10	Multiuser Systems	Introduction of various multiplexing schemes such as CMDA, TDMA, SDMA, OFDMA
11	Cellular Systems	Introduction to cellular systems
12	5G and Beyond	5G key technologies, Massive MIMO, AI for Wireless Communications

Lab Work

For the majority of lab session, we will use Matlab and corresponding Matlab toolboxes.

#	Name	Summary
1	Channel Models	Impact of time-dispersive frequency-selective channels
2	Modulation and BER	Monte-Carlo simulation with higher-order modulation
3	Diversity and OFDM	MIMO-OFDM transmitter and receiver
4	Coded MIMO-OFDM	Convolutional channel code in combination with MIMO-OFDM

Master | Christopher Knievel

Reinforcement Learning (EIM)

Learning Objectives

Lecture

Lab Work

Wireless Communications (EIM)

Subject-specific Competencies

Methodological Compentencies

Interdisciplinary Competencies

Course Content

Lecture

Lab Work