Connect Four Robot: Reinforcement Learning on Embedded Hardware

Mon, 16 Oct 2023 11:13:50 +0100

The Connect Four robot is a physical demonstrator for reinforcement learning. It was originally developed as a master’s project in the EIM program (Electrical Engineering and Information Technology) by the students Alexander Bernhardt, Christian Bernhardt, Raphael Carli and Manuel Reichelt.

In addition to a DQN agent, a Monte Carlo Tree Search (MCTS) agent and AlphaZero were implemented. All algorithms were parameterized to run in real time on a Raspberry Pi 4. Different difficulty levels can be selected via a touch display. At the highest level, winning against the AI is nearly impossible.

Alongside the software, an open-source robot arm was built with a 3D printer, a sensor system was set up on a real game board, and a magazine was designed from which the robot arm picks up the game pieces.

The robot attracted the interest of local media, who visited the project at HTWG in autumn 2023. A Regio TV report about the project is available in the media library: (in German)

An article was also published in the Südkurier: (in German)

Current work

The setup is currently being extended with a Dobot Magician E6, a six-axis collaborative robot arm.

Safety Assistance Systems for Bicycles

Wed, 15 Dec 2021 11:13:50 +0100

With the increasing spread of e-bikes, the number of accidents is also rising. An increase of more than 50% compared to last year, to over 2900 accidents, can be observed . Many accidents occur due to excessive speed and/or riding into oncoming traffic (oncoming cyclists). A similar development was observed with the introduction of the automobile. Step by step, more and more active safety features (ESP, Emergency Brake Warning, ACC, …) were introduced in addition to passive ones (seat belts, airbags). In the bicycle sector, besides passive safety functions such as safety vests and helmets, active safety functions are not yet widespread.

Overview

In this project, a warning system for cyclists shall be developed. The system shall consist of affordable components, exteroceptive sensors (e.g. camera, ultrasonic sensors) as well as proprioceptive sensors (e.g. accelerometer, gyroscope), be energy-efficient and compact in design so that it can be mounted unobtrusively on an e-bike.

Project shall be extended to various usecases such as

Scene interpretation with respect to other cyclists and/or pedestrians
Classification of unsafe/unstable driving behavior
Recognition of potential hazards in the environment, e.g. potholes, high curbs,…

This project is kindly supported by Kettler Alu-Rad GmbH.

Completed thesis

(click titles for more information)

Details

Details to be added.

Details

The aim of this thesis is to implement different deep learning methods and evaluate if they can be used to accurately estimate the 6-D pose of a bicycle, based on IMU measurements of a 6-DOF IMU.

The following requirements must be met by the algorithm:

Accounting for sensor noise and biases
Fusing of measurements data from multiple IMUs
Capable of real-time computation

A video-SLAM algorithm has been implemented to serve as a state-of-the-art comparison and showed a good performance.

Details

This thesis focused on the development of a neural network-based classification of cyclists, pedestrians, cars, and trucks. Two popular datasets have been used for the training, namely the KITTI Vision Benchmark Suite and the Tsinghua-Daimler Cyclist Detection Benchmark Dataset. While the accuracy of the classification of cars was as expected, the accuracy to detect cyclist was unexpectedly poor. A deeper analysis of the datasets revealed that the number of cyclists labels are insufficient. Nevertheless, the potential of detection performance could be highlighted with self-recorded videos.

Details

In this thesis, an efficient object detection algorithm has been implemented to detect cyclist and a criticality metric has been investigated for its real-time capability using a monocular camera running on a Raspberry Pi 3b. Specifically, a machine learning algorithm had to identify relevant features per cyclist and tracked these features over time via the optical flow.

As both, detected cyclist and moving, the consistent identification and tracking of relevant features is a challenge, especially in scenarios with high background noise. As a result, a feature outlier detection is of importance. Various algorithms have been investigated and analyzed for the real-time capability running on a Raspberry Pi 3b.

Based on the scaling of the detected features, a time-to-collision (TTC) can be well estimated. As the results show, the TTC is accurate up to 3 seconds, which is sufficient for the system to warn the cyclist of an upcoming critical situation.

More information can be found in the .

Embedded AI | Christopher Knievel

Connect Four Robot: Reinforcement Learning on Embedded Hardware

Current work

Safety Assistance Systems for Bicycles

Overview

Completed thesis