Bogdan Burlacu’s research while affiliated with University of Applied Sciences Upper Austria and other places

The increasing complexity of machine learning models has motivated the need to ensure that the results are understandable and transparent, enabling trust and accountability. This work provides an extensive overview of methods to measure the explainability and interpretability of machine learning results. This work addresses the challenges posed by closed-box models, which lack transparency in their decision-making processes and evaluates techniques used to make these models more understandable. Through the application of open-box models, like symbolic regression, we demonstrate that it is possible to achieve high interpretability without sacrificing model performance. Additionally, we highlight the necessity for robust and unified evaluation metrics for explainability and interpretability, evaluating complexity and fidelity scores as a comprehensive measure. A variety of closed-box models have been trained and evaluated in terms of performance, and explainability including, artificial neural networks, support vector regression, and random forest regression. Additionally, we trained symbolic regression models with different levels of complexity, which were evaluated regarding their performance and interpretability. Our results underscore the importance of developing methodologies that balance complexity and interpretability, advocating for further research into explainable artificial intelligence frameworks, particularly those incorporating genetic programming. This work aims to contribute to the advancement of responsible and transparent artificial intelligence systems.

Gradient descent-based local search can dramatically improve solution performance in symbolic regression tasks, at the cost of significantly higher runtime as well as increased risks of overfitting. In this paper, we investigate exactly what amount of local search is really needed within the GP population. We show that low intensity local search is sufficient to boost the fitness of the entire population, provided that local search information in the form of optimized numerical parameters is written back into the genotype at least some of the time. Our results suggest that spontaneous adaptations (in the Lamarckian sense) act as evolutionary fuel for the Baldwin effect in genetic programming, and that in the absence of the former, the latter does not occur and evolution is hindered. The Lamarckian model works particularly well in symbolic regression, as local search only affects model coefficients and does not affect the inheritance of useful building blocks contained in the model structure.

In the past, field-programmable gate arrays (FPGAs) have had some notable successes when employed for Boolean and fixed-point genetic programming (GP) systems, but the more common floating-point representations were largely off limits, due to a general lack of efficient device support. However, recent work suggests that for both the training and inference phases of floating-point-based GP, contemporary FPGA technologies may enable significant performance and energy improvements—potentially multiple orders of magnitude—when compared to general-purpose CPU/GPU devices. In this chapter, we highlight the potential advantages and challenges of using FPGAs for GP systems, and we showcase how novel algorithmic considerations likely need to be made in order to extract the most benefits from specialized hardware. Primarily, we consider tree-based GP, although we include suggestions for other program representations. Overall, we conclude that the GP community should earnestly revisit the use of FPGA devices, especially the tailoring of state-of-the-art algorithms to FPGAs, since valuable enhancements may be realized. Most notably, FPGAs may allow for faster and/or less costly GP runs, in which case it may also be possible for better solutions to be found when allowing an FPGA to consume the same amount of runtime/energy as another platform.

This paper establishes the potential of accelerating the evaluation phase of tree-based genetic programming through contemporary field-programmable gate array (FPGA) technology. This exploration stems from the fact that FPGAs can sometimes leverage increased levels of both data and function parallelism, as well as superior power/energy efficiency, when compared to general-purpose CPU/GPU systems. In this investigation, we introduce a fixed-depth, tree-based architecture that can fully parallelize tree evaluation for type-consistent primitives that are unrolled and pipelined. We show that our accelerator on a 14nm FPGA achieves an average speedup of 43$\times$ when compared to a recent open-source GPU solution, TensorGP, implemented on 8nm process-node technology, and an average speedup of 4,902$\times$ when compared to a popular baseline GP software tool, DEAP, running parallelized across all cores of a 2-socket, 28-core (56-thread), 14nm CPU server. Despite our single-FPGA accelerator being 2.4$\times$ slower on average when compared to the recent state-of-the-art Operon tool executing on the same 2-processor, 28-core CPU system, we show that this single-FPGA system is 1.4$\times$ better than Operon in terms of performance-per-watt. Importantly, we also describe six future extensions that could provide at least a 64–192$\times$ speedup over our current design. Therefore, our initial results provide considerable motivation for the continued exploration of FPGA-based GP systems. Overall, any success in significantly improving runtime and energy efficiency could potentially enable novel research efforts through faster and/or less costly GP runs, similar to how GPUs unlocked the power of deep learning during the past fifteen years.