Annie S. Wu’s research while affiliated with University of Central Florida and other places

This research explores the use of Cosmic Microwave Background (CMB) radiation as a reference signal for Initial Orbit Determination (IOD). By leveraging the unique properties of CMB, this study introduces a novel method for estimating spacecraft velocity and position with minimal reliance on pre-existing environmental data, offering significant advantages for space missions independent of Earth-specific conditions. Using Machine Learning (ML) regression models, this approach demonstrates the capability to determine velocity from CMB signals and subsequently determine the satellite's position. The results indicate that CMB has the potential to enhance the autonomy and flexibility of spacecraft operations.

This study explores the use of Reinforcement Learning (RL) to develop autonomous agents for Beyond Visual Range (BVR) air combat, addressing the challenges of dynamic and uncertain adversarial scenarios. We propose a novel approach that introduces a task-based layer, leveraging domain expertise to optimize decision-making and training efficiency. By integrating multi-head attention mechanisms into the policy model and employing an improved DQN algorithm, agents dynamically select context-aware tasks, enabling the learning of efficient emergent behaviors for variable engagement conditions. Evaluations in single- and multi-agent BVR scenarios against adversaries with diverse tactical characteristics demonstrate superior training efficiency and enhanced agent capabilities compared to leading RL algorithms commonly applied in similar domains, including PPO, DDPG, and SAC. A robustness study underscores the critical role of diverse enemy selection in the RL process, showing that adversaries with variable tactical behaviors are essential for developing robust agents. This work advances RL methodologies for autonomous BVR air combat and provides insights applicable to other problems with challenging adversarial scenarios.

This study investigates high-performance models for predicting the Weapon Engagement Zone (WEZ) in beyond-visual-range (BVR) air combat scenarios. Accurate WEZ predictions are crucial for decision-making in air combat, and high-performance solutions are essential for developing and deploying autonomous systems. To optimize the model training process, we introduce novel feature engineering and data augmentation strategies, achieving a 70% improvement in the Mean Absolute Error (MAE) of WEZ predictions. A comparison of various regression methods highlights the potential of polynomial-based alternatives when fully utilized. In our evaluations, Polynomial Regression (PR) with higher interaction degrees outperforms more complex machine learning models in prediction accuracy and computational efficiency. For instance, Lasso regression, a PR method with regularization, achieves results that are 33% better and 2.1 times faster than the best artificial neural network-based solution. Our results challenge common assumptions in the literature about the complexity and feasibility of higher-order PR solutions, suggesting that they can be a compelling alternative for various challenges across domains. This study also provides a new open dataset to facilitate further research and advancements in this field.

This paper introduces B-ACE (Beyond Visual Range (BVR)-Air Combat Environment), an open-source simulation framework leveraging the Godot game engine to evaluate Multi-Agent Reinforcement Learning (MARL) for military research and development. Traditional military simulations are often restricted, limiting research discussions and comparisons among different groups. B-ACE addresses this by providing an open, accessible environment that can be easily shared and extended within the research community, ensuring reproducibility and flexibility for further studies. Our approach capitalizes on Godot's high performance and script-based development, offering a cost-effective and customizable solution for creating air combat scenarios. This integration allows rapid prototyping and evaluation of autonomous agent behaviors using existing reinforcement learning frameworks. In the developed scenario, agents should learn to engage in BVR Air Combat, defend itself and a position against enemy aircraft. Using integration with state-of-the-art MARL algorithms, we explore advanced techniques in autonomous agent development within complex Beyond Visual Range (BVR) air combat scenarios. The environment simulates key aspects of air combat, including radar detection, weapons engagement, and tactical maneuvers. While not overly realistic, B-ACE provides a valuable testbed for prototyping and evaluating AI development approaches. Through three study cases, we demonstrate B-ACE's capability to support research in BVR air combat scenarios, including the generation of weapon efficiency models, optimizing baseline behaviors, and training agents using MARL. ABOUT THE AUTHORS Andre R. Kuroswiski, M.S., is a Lieutenant Colonel in the Brazilian Air Force. As a fighter pilot and electrical engineer, he became a researcher at the Institute for Advanced Studies (IEAv) in 2018. Since then, he has been working on simulation development to support autonomous agents research and military scenario analysis. Currently, he is a visiting researcher at the University of Central Florida, focusing on enhancing deep reinforcement learning techniques to develop cooperative air combat agents.

In this paper, we introduce an alternative approach to enhancing Multi-Agent Reinforcement Learning (MARL) through the integration of domain knowledge and attention-based policy mechanisms. Our methodology focuses on the incorporation of domain-specific expertise into the learning process, which simplifies the development of collaborative behaviors. This approach aims to reduce the complexity and learning overhead typically associated with MARL by enabling agents to concentrate on essential aspects of complex tasks, thus optimizing the learning curve. The utilization of attention mechanisms plays a key role in our model. It allows for the effective processing of dynamic context data and nuanced agent interactions, leading to more refined decision-making. Applied in standard MARL scenarios, such as the Stanford Intelligent Systems Laboratory (SISL) Pursuit and Multi-Particle Environments (MPE) Simple Spread, our method has been shown to improve both learning efficiency and the effectiveness of collaborative behaviors. The results indicate that our attention-based approach can be a viable approach for improving the efficiency of MARL training process, integrating domain-specific knowledge at the action level.

We compare the performance of machine learning methods for building predictive models to estimate the expected characteristics of hypoglycemic or low blood glucose events in type 1 diabetes patients. We hypothesize that the rate of change of blood glucose ahead of a hypoglycemic event may affect the severity and duration of the event and investigate the utility of machine learning methods on using blood glucose rate of change, in combination with other physiological and demographic factors, to predict the minimum glucose value and the duration of a hypoglycemic event. This work compares the performance of six state-of-the-art methods on prediction accuracy and feature selection. Results find that XGBoost delivers the best performance in all cases. Examination of the XGBoost feature importance scores show that glucose rate of change is the most used feature in the models generated by XGBoost.

This paper investigates the robustness of a genetic algorithm (GA) in feature selection across a dataset with increasing imputed missing values. Feature selection can be beneficial in predictive modeling to reduce computational costs and potentially improve performance. Beyond these benefits, it also enables a clearer understanding of the algorithm's decision-making processes. In the context of real-world datasets that can contain missing values, feature selection becomes more challenging.A robust feature selection algorithm should be able to identify the key features despite missing data values. We investigate the effectiveness of this approach against two other feature selection algorithms on a dataset with increasingly imputed values to determine whether it can sustain good performance with only the selected features.Our results reveal that compared to the other two methods, the features selected by GA resulted in better classification performance across different imputation rates and methods.

Internet of Things (IoT) devices are common in today’s computer networks. These devices can be computationally powerful, yet prone to cybersecurity exploitation. To remedy these growing security weaknesses, this work proposes a new artificial intelligence method that makes these IoT networks safer through the use of autonomous, swarm-based cybersecurity penetration testing. In this work, the introduced Particle Swarm Optimization (PSO) penetration testing technique is compared against traditional linear and queue-based approaches to find vulnerabilities in smart homes and IoT networks. To evaluate the effectiveness of the PSO approach, a network simulator is used to simulate smart home networks of two scales: a small, home network and a large, commercial-sized network. These experiments demonstrate that the swarm-based algorithms detect vulnerabilities significantly faster than the linear algorithms. The presented findings support the case that autonomous and swarm-based penetration testing in a network could be used to render more secure IoT networks in the future. This approach can affect private households with smart home networks, settings within the Industrial Internet of Things (IIoT), and military environments.