Robust Learning-Based Trajectory Planning for Emerging Mobility SystemsBehdad Chalaki, Andreas A. Malikopoulos
In this paper, we extend a framework that we developed earlier for coordination of connected and automated vehicles (CAVs) at a signal-free intersection to incorporate uncertainty.
We formulate a deterministic motion planning problem the solution of which guarantees that none of the constraints in the system becomes active. Considering the uncertainty in the CAVs’ trajectories, we reformulate the motion planning problem as a probabilistic problem, the solution of which guarantees that constraints in the system are satisfied within a threshold. Using the noisy observations of actual time trajectories and leveraging the Gaussian process regression, we obtain bounded confidence intervals for deviations from the nominal trajectories of CAVs. Incorporating these confidence intervals, each CAV aims at solving the probabilistic motion planning problem in a robust approach. We demonstrate the effectiveness of our proposed framework through a numerical simulation.
A Hysteretic Q-learning Coordination Framework for Emerging Mobility Systems in Smart CitiesBehdad Chalaki, Andreas A. Malikopoulos
Connected and automated vehicles (CAVs) can alleviate traffic congestion, air pollution, and improve safety. In this paper, we provide a decentralized coordination framework for CAVs at a signal-free intersection to minimize travel time and improve fuel efficiency. We employ a simple yet powerful reinforcement learning approach, an off-policy temporal difference learning called Q-learning, enhanced with a coordination mechanism to address this problem. Then, we integrate a first-in-first-out queuing policy to improve the performance of our system. We demonstrate the efficacy of our proposed approach through simulation and comparison with the classical optimal control method based on Pontryagin’s minimum principle.
Optimal Control of Connected and Automated Vehicles at Multiple Adjacent IntersectionsBehdad Chalaki, Andreas A. Malikopoulos
In this paper, we establish a decentralized optimal control framework for connected and automated vehicles (CAVs) crossing multiple adjacent, multi-lane intersections to minimize energy consumption and improve traffic throughput. Our framework consists of two layers of planning. In the upper-level planning, each CAV computes its optimal arrival time at each intersection recursively along with the optimal lane to improve the traffic throughput.
In the low-level planning, we formulate an energy-optimal control problem with interior-point constraints, the solution of which yields the optimal control input (acceleration/deceleration) of each CAV to cross the intersections at the time specified by the upper-level planning. Moreover, we extend the results of the proposed bi-level framework to include a bounded steady-state error in tracking the optimal position of the CAVs. Finally, we demonstrate the effectiveness of the proposed framework through simulation and comparison with traditional signalized intersections.
Experimental Validation of a Real-Time Optimal Controller for Coordination of CAVs in a Multi-Lane RoundaboutBehdad Chalaki, Logan E. Beaver, Andreas A. Malikopoulos
Roundabouts in conjunction with other traffic scenarios, e.g., intersections, merging roadways, speed reduction zones, can induce congestion in a transportation network due to driver responses to various disturbances. Research efforts have shown that smoothing traffic flow and eliminating stop-and-go driving can both improve fuel efficiency of the vehicles and the throughput of a roundabout. In this paper, we validate an optimal control framework developed earlier in a multi-lane roundabout scenario using the University of Delaware’s scaled smart city (UDSSC).
We first provide conditions where the solution is optimal. Then, we demonstrate the feasibility of the solution using experiments at UDSSC, and show that the optimal solution completely eliminates stop-and-go driving while preserving safety.
Time-Optimal Coordination for Connected and Automated Vehicles at Adjacent IntersectionsB. Chalaki, A. A. Malikopoulos
In this paper, we present a decentralized optimal control framework for connected and automated vehicles (CAVs) crossing two adjacent intersections. The framework consists of an upper-level scheduling problem and a low-level optimal control problem. The solution of the upper-level problem designates the optimal time of each CAV aimed at minimizing its travel time to cross the intersections. The outcome of the upper-level scheduling problem becomes the input of the low-level problem, the solution of which yields the optimal control input (acceleration/deceleration) of each CAV to exit the intersections at the time specified in the upper-level scheduling problem. We demonstrate the effectiveness of the proposed framework through simulation and comparison with a signalized intersection.
An Optimal Coordination Framework for Connected and Automated Vehicles in two Interconnected IntersectionsB. Chalaki, A. A. Malikopoulos
In this paper, we provide a decentralized optimal control framework for coordinating connected and automated vehicles (CAVs) in two interconnected intersections. We formulate a control problem and provide a solution that can be implemented in real time. The solution yields the optimal acceleration/deceleration of each CAV under the safety constraint at “conflict zones,” where there is a chance of potential collision. Our objective is to minimize travel time for each CAV. If no such solution exists, then each CAV solves an energy-optimal control problem.
We evaluate the effectiveness of the efficiency of the proposed framework through simulation.
Demonstration of a Time-Efficient Mobility System Using a Scaled Smart CityL. E. Beaver, B. Chalaki, A. M. Mahbub, L. Zhao, R. Zayas, A. A. Malikopoulos
The implementation of connected and automated vehicle (CAV) technologies enables a novel computational framework to deliver real-time control actions that optimize travel time, energy, and safety. Hardware is an integral part of any practical implementation of CAVs, and as such, it should be incorporated in any validation method. However, high costs associated with full scale, field testing of CAVs have proven to be a significant barrier. In this paper, we present the implementation of a decentralized control framework, which was developed previously, in a scaled-city using robotic CAVs, and discuss the implications of CAVs on travel time.
Simulation to Scaled City: Zero-Shot Policy Transfer for Traffic Control via Autonomous VehiclesJang, K., Vinitsky, E., B. Chalaki, Remer, B., Beaver, L. E., Malikopoulos, A.A., and Bayen, A.
Using deep reinforcement learning, we successfully train a set of two autonomous vehicles to lead a fleet of vehicles onto a roundabout and then transfer this policy from simulation to a scaled city without fine-tuning.
Zero-Shot Autonomous Vehicle Policy Transfer: From Simulation to Real-World via Adversarial LearningB. Chalaki, Beaver, L., Remer, B., Jang, K., Vinitsky, E., Bayen, A., & Malikopoulos, A. A.
We demonstrate a zero-shot transfer of an autonomous driving policy directly from an autonomous driving simulator to the University of Delaware Scaled Smart City under stochastic disturbances. Using adversarial multi-agent reinforcement learning, we train autonomous vehicles to coordinate with each other crossing a roundabout. We demonstrate that the addition of adversarial training considerably improves the stability and robustness of policies being transferred to the real world.
Vehicle Routing for the Last-Mile Logistics Problem Vehicle Routing for the Last-Mile Logistics ProblemDesai, H., Remer, B. Chalaki, B., Zhao, L., Malikopoulos, A. A & Rios-Torres, J.
Energy consumption is the major contributor associated with large and growing transportation cost in logistics. Optimal vehicle routing approaches can provide solutions to reduce their operating costs and address implications on energy. This paper outlines a solution to the single-depot capacitated vehicle routing problem with the objective of minimizing daily operation cost with a homogeneous fleet of delivery vehicles. The problem is solved using Simulated Annealing, to provide optimal routes for the vehicles traveling between the depot and destinations. Simulation results demonstrate that the proposed approach is effective to recommend an optimal route and reduce operation cost.