Integrated passenger flow analysis and street-level classification for public transport management using deep learning and IoT

In recent years, the rapid development of digital technologies and computer vision systems has profoundly transformed the way cities observe and manage urban mobility. Within this evolving context, public transportation serves as a key data source for understanding travel patterns and improving service quality. However, the real-time acquisition and processing of reliable information about passengers and the surrounding urban environment remain a complex challenge, often constrained by traditional sensor-based or indirect counting systems. This thesis introduces an integrated approach for real-time passenger counting and street-level monitoring in public transport vehicles, combining computer vision, artificial intelligence, and geospatial analysis. Unlike conventional methodologies, the proposed system leverages pre-trained deep convolutional neural networks, specifically the YOLO model, to simultaneously analyze video streams captured both inside and outside the vehicles. Inside the vehicles, cameras are used to perform accurate passenger counting after door closure and flow analysis during stops, substantially improving precision compared to flow-only approaches. Externally, additional cameras monitor the surrounding environment to detect relevant urban elements—such as potholes, and bicycles—thus providing a comprehensive view of street conditions and infrastructure quality. All detections are georeferenced using GPS data and transmitted to a cloud platform, where they are processed to generate informative maps and indicators that support urban maintenance and planning. A further contribution of this research lies in the integration of passenger flow data within public transport network analysis tools, enabling the identification of usage patterns, bottlenecks, and operational inefficiencies. The system was validated through a large-scale deployment carried out in the city of Bari, Italy, involving 50 buses operating across 30 lines of the local public transport network. The experimental results demonstrated high accuracy, robustness, and real-world applicability, confirming the potential of the proposed solution to support intelligent mobility management and the transition toward more sustainable and data-driven urban systems. This thesis is organized into six chapters. Chapter 1 introduces the problem of visionbased monitoring in public transport, motivates the use of onboard perception as a source of actionable mobility intelligence, and frames the main research contributions through a review of state-of-the-art passenger counting and monitoring approaches. Chapter 2 provides the theoretical background required by the proposed framework, covering the core principles of neural networks as well as the optical, geometric, and calibration aspects that influence the quality and reliability of camera-based measurements in real operating conditions. Chapter 3 focuses on the object-detection backbone adopted in this work, namely YOLOv5: after presenting the model fundamentals and representative application domains, it motivates the choice of YOLOv5 for both in-cabin passenger analytics and outdoor urban-scene monitoring under embedded, real-time constraints. Chapter 4 describes the end-to-end system architecture and operating workflow, detailing the onboard hardware and networking setup, the dual internal/external processing pipeline, the training procedures, and the cloud-oriented data collection strategy that enables fleet-level scalability. Chapter 5 reports the experimental evaluation and results, including passenger counting validation, cloudbased analytics for real-time fleet monitoring and network-level insights, and the analysis of geo-referenced external detections for infrastructure and urban-environment assessment. Finally, Chapter 6 summarizes the main findings and limitations, and outlines future research directions and deployment perspectives; it also reflects on the dissemination outcomes of the work, including the formative experience.