Soilse: A decentralized approach to optimization of fluctuating urban traffic using Reinforcement Learning

Abstract

Increasing traffic congestion is a major problem in urban areas, which incurs heavy economic and environmental costs in both developing and developed countries. Efficient urban traffic control (UTC) can help reduce traffic congestion. However, the increasing volume and the dynamic nature of urban traffic pose particular challenges to UTC. Reinforcement Learning (RL) has been shown to be a promising approach to efficient UTC. However, most existing work on RL-based UTC does not adequately address the fluctuating nature of urban traffic. This paper presents Soilse1, a decentralized RL-based UTC optimization scheme that includes a nonparametric pattern change detection mechanism to identify local traffic pattern changes that adversely affect an RL agent's performance. Hence, Soilse is adaptive as agents learn to optimize for different traffic patterns and responsive as agents can detect genuine traffic pattern changes and trigger relearning. We compare the performance of Soilse to two baselines, a fixed-time approach and a saturation balancing algorithm that emulates SCATS, a well-known UTC system. The comparison was performed based on a simulation of traffic in Dublin's inner city centre. Results from using our scheme show an approximate 35%–43% and 40%–54% better performance in terms of average vehicle waiting time and average number of vehicle stops respectively against the best baseline performance in our simulation.