Brain-Swarm Interaction and Control Interfaces

Abstract: This project focuses on understanding how multi-agent robotic systems can be controlled and coordinated by one human supervisor. More specifically, we focus on understanding how human supervisors perceive and plan high-level functions for a swarm of semi-autonomous agents. We define basic principles of communication and utilize them in building the language of supervised autonomy between the human operator and the multi-agent system. The overarching goal of the research is to unveil brain mechanisms that perceive multi-agent systems information, and model them in order to define methods for extracting centralized and/or de-centralized control commands for a multi-agent system.

Results: We have identified specific brain areas and patterns of activations related to collective behaviors of swarms of robots. Swarm cohesions seems to be represented in the brain of human supervisors (see paper 1 below). We have built a system that transforms ElectroEncephaloGraphic activity of the human brain to collective behaviors of robotic swarms. The system can output continuous variables for the control of the robots. Moreover, the system is combined with a joystick controller, to allow for a hybrid brain-machine interface between the humans and a robotic swarm. The system has been demonstrated via real-time control of a swarm of quadrotors using brain activity and joystick control.

Funding: This work is supported by the U.S. Defense Advanced Research Projects Agency (DARPA) grant D14AP00068 (2014-2017) and U.S. Air Force Office of Scientific Research (AFOSR) award FA9550-14-1-0149 (2014-2017) and FA9550-18-1-0221 (2018-2021), as well as the DURIP award #: FA9550-18-1-0464.

Publications: (see Publications page for a more recent list of papers)

Ajitesh Singh and Panagiotis Artemiadis, “Automatic Identification of the Leader in a Swarm using an Optimized Clustering and Probabilistic Approach,” in the 2021 IEEE International Conference on Multisensor Fusion and Integration (MFI 2021), 2021. Best Student Paper Award 2nd Runner Up [pdf]

Chuong H. Nguyen, George K Karavas and Panagiotis Artemiadis, “Adaptive multi-degree of freedom Brain Computer Interface using online feedback: towards novel methods and metrics of mutual adaptation between humans and machines for BCI,” PloS one 14, no. 3 (2019): e0212620. [link to pdf]

Chuong H Nguyen and Panagiotis Artemiadis, “EEG Feature Descriptors and Discriminant Analysis under Riemannian Manifold perspective,” Neurocomputing, 275, pp. 1871-1883, 2018. [pdf]

Chuong H Nguyen, George K Karavas and Panagiotis Artemiadis, “Inferring imagined speech using EEG signals: a new approach using Riemannian manifold features,” Journal of Neural Engineering, 15.1, 016002, 2018. [pdf]

George Karavas, Daniel T. Larsson and Panagiotis Artemiadis, “A hybrid brain-machine interface for control of robotic swarms: Preliminary results”, in 2017 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), pp. 5065 – 5075, 2017. [pdf]

Panagiotis Artemiadis, “Brain-Swarm Control Interfaces: The Transition from Controlling One Robot to a Swarm of Robots,” Adv Robot Autom 6: e127, 2017 (editorial article) [link to pdf]

George K. Karavas and Panagiotis Artemiadis “On the Effect of Swarm Collective Behavior on Human Perception: Towards Brain-Swarm Interfaces,” IEEE Conference on Multisensor Fusion and Integration (MFI), pp. 172-177, 2015. [pdf]

Patents

Panagiotis Artemiadis and Daniel Larsson, “Systems and methods for dynamics, modeling, simulation and control of mid-flight coupling of quadrotors”, U.S. Patent No. 10,642,285. 5 May. 2020.

Panagiotis Artemiadis and Georgios Konstantinos Karavas, “Systems and methods for hybrid brain interface for robotic swarms using EEG signals and joystick inputs”, U.S. Patent Office, 2016.