Valiollah Ghaffari

In recent years, Unmanned Surface Vehicles (USVs) have emerged as essential tools for ocean exploration, environmental monitoring, and maritime operations. Among USVs, unmanned surface sailing (USS) vehicles, which utilize wind energy for propulsion, operate autonomously for extended periods compared to engine-based vehicles dependent on fuels. This thesis explores the application of Reinforcement Learning (RL) in the formation control of multi-agent leader-following USS systems. The target is developing a control strategy enabling USS vehicles to maintain a predefined formation while navigating dynamic and uncertain marine environments. Traditional control methods often struggle with nonlinear dynamics, environmental variations, and complicated sailing techniques in the formation of USS robots in Multi-Agent Systems (MAS). The agents' heading angle and speed must be controlled in the Multi-Input Multi-Output (MIMO) system of the USS robots to perform the formation control. This research addresses these challenges by integrating RL with Deep Deterministic Policy Gradient (DDPG) agents to achieve smooth navigation and adaptive formation control. The proposed method is validated through different simulations, which demonstrate the adequate performance of controllers in various environmental conditions and formation topologies.