For many complex perception problems, such as human or animal MoCap, it is hard to model a reliable objective function that models the perception uncertainty. These models can not only be difficult to derive but also highly non-convex. We therefore take a deep RL approach to solve the perception-driven formation control problem. The key idea is to model the problem as a sequential decision making problem and learn optimal policies for each robot/agent in the team that maximizes the joint MoCap accuracy -- embedded as rewards only during training. For human MoCap, we learned navigation policies for multiple aerial robots purely in simulation and deploy on real robots. Here observations were joint detections on 2D images obtained by the robots. Ongoing work involves end-to-end DRL methods that overcome the need for joint detections, and DRL approach for animal MoCap. Figure: An illustration of an aerial MoCap system where aerial robot agents learn formation control policies based on MoCap performance rewards.

In this context we develop various RL and DRL-based methods ranging from airship control, autonomous landing of multi-rotors on moving platforms, autonomous soaring of gliders, and collision avoidance in large teams of robots. For airship control, we have developed a DRL method that combines a classical PID with learning and leverages PID's stability properties. For autonomous landing, our RL approach learns an optimal policy for 2D horizontal motion of the robot, with a special focus on interpretable hyper-parameter derivation. For autonomous soaring, we develop DRL based approaches that maximize the exploitation of updrafts and balance it with a simultaneous waypoint navigation task. In the context of collision avoidance, we explore attention networks that allow scalability to a very large number of robots and interpretability of the RL policy.  Figure: Reinforcement learning based autonomous landing by our aerial robot on a moving platform.

Through the development of various synthetic generation procedures, we have successfully overcome the challenges of collecting real-world ground truth data of dynamic environments and challenging scenarios. In our work in aerial human MoCap project, we utilized Unreal Engine and RenderPeople assets to create a dataset for human pose and shape (HPS) estimation using the AirSim toolkit. By doing so, we were able to solve several issues, including HPS from aerial views, multi-view ground truth information, and diverse pose generation. Separately, our work on GRADE, which is related to our active SLAM project, was developed using ISAAC Sim and the NVIDIA Omniverse suite. Through a novel method to automatically generate data of dynamic environments with minimal effort, we overcame issues of state-of-the-art robotics simulator like Gazebo, such as lack of visual realism and limited controllability of the simulation environment. As a result, we were able to demonstrate that current state-of-the-art methods that perform Dynamic SLAM are over-optimized for currently available datasets and do not generalize well. Furthermore, the generated data is realistic enough for both humans and zebras, allowing for detectors using only synthetic data with performances that are as good as or better than pre-trained models. We have released code and data related to all these works. Figure: A scene from our synthetic data generation work.