Paul Watters’s research while affiliated with Simplot Australia Pty. Ltd. and other places

This comprehensive survey explored the evolving landscape of generative Artificial Intelligence (AI), with a specific focus on the recent technological breakthroughs and the gathering advancements toward possible Artificial General Intelligence (AGI). It critically examined the current state and future trajectory of generative AI, exploring how innovations in developing actionable and multimodal AI agents with the ability scale their “thinking” in solving complex reasoning tasks are reshaping research priorities and applications across various domains, while the survey also offers an impact analysis on the generative AI research taxonomy. This work has assessed the computational challenges, scalability, and real-world implications of these technologies while highlighting their potential in driving significant progress in fields like healthcare, finance, and education. Our study also addressed the emerging academic challenges posed by the proliferation of both AI-themed and AI-generated preprints, examining their impact on the peer-review process and scholarly communication. The study highlighted the importance of incorporating ethical and human-centric methods in AI development, ensuring alignment with societal norms and welfare, and outlined a strategy for future AI research that focuses on a balanced and conscientious use of generative AI as its capabilities continue to scale.

The rapid rise in popularity of Large Language Models (LLMs) with emerging capabilities has spurred public curiosity to evaluate and compare different LLMs, leading many researchers to propose their own LLM benchmarks. Noticing preliminary inadequacies in those benchmarks, we embarked on a study to critically assess 23 state-of-the-art LLM benchmarks, using our novel unified evaluation framework through the lenses of people, process, and technology, under the pillars of benchmark functionality and integrity. Our research uncovered significant limitations, including biases, difficulties in measuring genuine reasoning, adaptability, implementation inconsistencies, prompt engineering complexity, evaluator diversity, and the overlooking of cultural and ideological norms in one comprehensive assessment. Our discussions emphasized the urgent need for standardized methodologies, regulatory certainties, and ethical guidelines in light of Artificial Intelligence (AI) advancements, including advocating for an evolution from static benchmarks to dynamic behavioral profiling to accurately capture LLMs’ complex behaviors and potential risks. Our study highlighted the necessity for a paradigm shift in LLM evaluation methodologies, underlining the importance of collaborative efforts for the development of universally accepted benchmarks and the enhancement of AI systems’ integration into society.

Ransomware has grown to be a dominant cybersecurity threat, by exiltrating, encrypting or destroying valuable user data, and causing numerous disruptions to victims. The severity of the ransomware endemic has generated research interest from both the academia and the industry. However, many studies held stereotypical assumptions about ransomware, used unveriied, outdated and limited self-collected ransomware samples, and did not consider government strategies, industry guidelines or cyber intelligence. We observed that ransomware no longer exists simply as an executable ile or limits to encrypting iles (data loss); data exiltration (data breach) is the new norm, espionage is an emerging theme, and the industry is shifting focus from technical advancements to cyber governance and resilience. We created a ransomware innovation adoption curve, critically evaluated 212 academic studies published during 2020 and 2023, and cross-veriied them against various government strategies, industry reports and cyber intelligence on ransomware. We concluded that many studies were becoming irrelevant to the contemporary ransomware reality, and called for the redirection of ransomware research to align with the continuous ransomware evolution in the industry. We proposed to address data exiltration as priority over data encryption, to consider ransomware in a business-practical manner, and recommended research collaboration with the industry.

Online child sexual abuse material (CSAM) is a growing problem. Prevention charities, such as Stop It Now! UK, use online messaging to dissuade users from viewing CSAM and to encourage them to consider anonymous therapeutic interventions. This experiment used a honeypot website that purported to contain barely legal pornography, which we treated as a proxy for CSAM. We examined whether warnings would dissuade males (18-30 years) from visiting the website. Participants (n = 474) who attempted to access the site were randomly allocated to one of four conditions. The control group went straight to the landing page (control; n = 100). The experimental groups encountered different warning messages: deterrence-themed with an image (D3; n = 117); therapeutic-themed (T1; n = 120); and therapeutic-themed with an image (T3; n = 137). We measured the click through to the site. Three quarters of the control group attempted to enter the pornography site, compared with 35 % to 47 % of the experimental groups. All messages were effective: D3 (odds ratio [OR] = 5.02), T1 (OR = 4.06) and T2 (OR = 3.05). Images did not enhance warning effectiveness. We argue that therapeutic and deterrent warnings are useful for CSAM-prevention.

In this study, we explored the progression trajectories of artificial intelligence (AI) systems through the lens of complexity theory. We challenged the conventional linear and exponential projections of AI advancement toward Artificial General Intelligence (AGI) underpinned by transformer-based architectures, and posited the existence of critical points, akin to phase transitions in complex systems, where AI performance might plateau or regress into instability upon exceeding a critical complexity threshold. We employed agent-based modelling (ABM) to simulate hypothetical scenarios of AI systems' evolution under specific assumptions, using benchmark performance as a proxy for capability and complexity. Our simulations demonstrated how increasing the complexity of the AI system could exceed an upper criticality threshold, leading to unpredictable performance behaviours. Additionally, we developed a practical methodology for detecting these critical thresholds using simulation data and stochastic gradient descent to fine-tune detection thresholds. This research offers a novel perspective on AI advancement that has a particular relevance to Large Language Models (LLMs), emphasising the need for a tempered approach to extrapolating AI's growth potential and underscoring the importance of developing more robust and comprehensive AI performance benchmarks.

This study investigated the integration readiness of four predominant cybersecurity Governance, Risk and Compliance (GRC) frameworks – NIST CSF 2.0, COBIT 2019, ISO 27001:2022, and the latest ISO 42001:2023 – for the opportunities, risks, and regulatory compliance when adopting Large Language Models (LLMs), using qualitative content analysis and expert validation. Our analysis, with both LLMs and human experts in the loop, uncovered potential for LLM integration together with inadequacies in LLM risk oversight of those frameworks. Comparative gap analysis has highlighted that the new ISO 42001:2023, specifically designed for Artificial Intelligence (AI) management systems, provided most comprehensive facilitation for LLM opportunities, whereas COBIT 2019 aligned most closely with the European Union AI Act. Nonetheless, our findings suggested that all evaluated frameworks would benefit from enhancements to more effectively and more comprehensively address the multifaceted risks associated with LLMs, indicating a critical and time-sensitive need for their continuous evolution. We propose integrating human-expert-in-the-loop validation processes as crucial for enhancing cybersecurity frameworks to support secure and compliant LLM integration, and discuss implications for the continuous evolution of cybersecurity GRC frameworks to support the secure integration of LLMs.