I enjoy making things. Here are a selection of projects that I have worked on over the years.
The project "MRTAgent" is a self-play-driven bi-level reinforcement learning framework, designed to address the growing complexity of task allocation and robot selection in dynamic warehouse environments. As warehouse automation becomes increasingly prevalent, optimizing the assignment of tasks to robots while accounting for practical constraints such as energy consumption, battery life, and avoiding collisions becomes crucial. MRTAgent was developed to minimize inefficiencies such as unnecessary travel distances and delays in task completion. The framework utilizes a dual-agent approach, where one agent focuses on task selection and the other on robot selection. By separating these tasks and having one agent in evaluation mode while the other trains, MRTAgent ensures that real-time decisions can be made efficiently. This structure allows for continuous learning and adaptation in environments with constantly changing conditions. To support safe navigation, the system integrates a modified Linear Quadratic Regulator (LQR) for collision-free robot movement, accommodating physical constraints present in real-world scenarios. MRTAgent was rigorously tested across multiple datasets with varying environmental conditions, demonstrating substantial improvements in reducing operational costs and delays compared to traditional methods. This innovative framework enables more intelligent task management, helping warehouses run more smoothly while ensuring safety and efficiency.
In the fast-paced world of e-commerce, minimizing product delivery costs—often referred to as the cost-to-serve (C2S) is a significant challenge. This project introduced an integrated algorithmic framework designed to tackle this issue by efficiently managing the high volume of spatio-temporally diverse orders that must be fulfilled from multiple warehouses using a fleet of vehicles. The project focuses on two critical decision-making levels: (i) selecting the optimal fulfillment node for each order, with the option of deferring orders to future time slots, and (ii) routing vehicles that carry multiple orders from the same warehouse. The proposed solution leverages graph neural networks and reinforcement learning to train agents for both node selection and vehicle routing, ensuring that real-world constraints such as warehouse inventory capacity, vehicle characteristics, and customer delivery time windows are fully considered. The complexity of this problem is amplified by the interdependence of fulfillment node selection and vehicle routing, both of which influence the final outcomes. Our experiments demonstrate that this AI-driven approach significantly outperforms traditional heuristic methods, providing a more efficient and scalable solution to e-commerce delivery logistics.
Power grids transport electricity across states, countries, and even continents. They are the backbone of global societies and economies, playing a pivotal economic and societal role by supplying reliable power to industry, business, and domestic consumers. Their importance is even more critical today as we transition toward a sustainable world within a carbon-free economy. Issues within the power grid can range from transient stability problems and localized blackouts to complete system or country-wide blackouts, which can cause significant economic and social disruptions. Grid operators are responsible for ensuring a secure supply of electricity at all times and designing systems to be both reliable and resilient. With the advent of renewable energy, electric mobility, and limitations on new grid infrastructure projects, the task of controlling existing grids is becoming increasingly difficult, forcing operators to do ‘more with less.’ We have developed RL-based solutions to address this important real-world problem by alleviating congestion in power networks through topological reconfigurations.
The energy sector is undergoing a massive transformation that includes key aspects such as integrating renewables, improving operational efficiency, leveraging smart grid infrastructure, and handling the dynamics of transactive energy.
