Predestrian Classifier for Intelligent Vehicles

Overview

This 3-month project, developed at TU Delft, explores advanced deep learning-based pedestrian detection and motion planning for intelligent vehicle systems. The goal was to design and implement a high-accuracy pedestrian classifier capable of real-time object detection, tracking, and trajectory estimation in dynamic urban environments. The project incorporated multiple disciplines, including DEEP LEARNING, COMPUTER VISION, SENSOR FUSION, and MOTION PLANNING to create a robust system.

The classification system relied on Convolutional Neural Networks (CNNs) for pedestrian detection, enhanced by traditional feature extraction techniques such as Histogram of Oriented Gradients (HOG) and Support Vector Machines (SVMs).

Additionally, advanced Kalman Filters and Particle Filters were employed to track detected objects.

The project also investigated the 2D-to-3D projection of pedestrian bounding boxes, converting 2D image detections into real-world spatial coordinates. The scope was to go from 2d_bboxes to 3d_bboxes in order to plot them in 3D using k3d library.

Beyond perception, the study explored path planning and trajectory optimization strategies, integrating steering profiles with splines for automated vehicle maneuvers. A Best-First Search algorithm was implemented for optimal motion planning in constrained spaces such as urban intersections and parking lots.

Deep Learning & Object Classification

Key methodologies applied:
✔️   Feature Engineering:   CNN-based pedestrian classification, HOG feature extraction, SVM for fine-grained classification.
✔️   Tested Neural Network Architectures:   YOLOv9 based NNs, self-crafted CNNs, and MobileNet.
✔️   Data Augmentation:   Pedestrian dataset created by the Intelligent Vehicle Group at TU Delft.
✔️   Occlusion Handling:   Multi-frame object aggregation and Kalman filtering for occlusion recovery.

Implemented Performance Evaluators for the Pedestrian Classifier

✔️   mAP (mean Average Precision) over the whole dataset of frames.
✔️   Confusion Matrices to deal with FPs and FNs.
✔️   HOTA Matrices.
✔️   ROC curves.
✔️   F1 scores.