Building Adaptative and Transparent Cyber Agents with Local Language Models

Building Adaptative and Transparent Cyber Agents with Local Language Models

Autonomous intelligent agents offer a transformative approach to cyber defense by operating independently in complex and dynamic environments. While much research has focused on defensive systems, offensive agents are equally important for testing system resilience and improving defenses through realistic adversarial interaction. Recent advances demonstrate that large language models can automate penetration testing effectively, rivaling traditional reinforcement learning methods, but their reliance on cloud-based services introduces significant concerns around privacy and reproducibility.

Out of the Cage: How Stochastic Parrots Win in Cyber Security Environments

Out of the Cage: How Stochastic Parrots Win in Cyber Security Environments

Large Language Models (LLMs) have gained widespread popularity across diverse domains involving text generation, summarization, and various natural language processing tasks. Despite their inherent limitations, LLM-based designs have shown promising capabilities in planning and navigating open-world scenarios. This paper introduces a novel application of pre-trained LLMs as agents within cybersecurity network environments, focusing on their utility for sequential decision-making processes.

AlquistCoder: A Synthetic Data Approach to Training Compact Secure Coding Assistants and Building Security Benchmarks

AlquistCoder: A Synthetic Data Approach to Training Compact Secure Coding Assistants and Building Security Benchmarks

Large language models are increasingly used as programming assistants, but their security behavior remains uneven: they may generate code with vulnerable patterns, and they may provide actionable help for malicious requests. This paper introduces AlquistCoder, a compact 3.8B‐parameter coding assistant designed to address both risks through targeted synthetic‐data alignment.

Building Adaptative and Transparent Cyber Agents with Local Language Models

Building Adaptative and Transparent Cyber Agents with Local Language Models

Autonomous intelligent agents offer a transformative approach to cyber defense by operating independently in complex and dynamic environments. While much research has focused on defensive systems, offensive agents are equally important for testing system resilience and improving defenses through realistic adversarial interaction. Recent advances demonstrate that large language models can automate penetration testing effectively, rivaling traditional reinforcement learning methods, but their reliance on cloud-based services introduces significant concerns around privacy and reproducibility.

VelLMes: A High-Interaction AI-Based Deception Framework

VelLMes: A High-Interaction AI-Based Deception Framework

There are very few SotA deception systems based on Large Language Models. The existing ones are limited only to simulating one type of service, mainly SSH shells. These systems - but also the deception technologies not based on LLMs - lack an extensive evaluation that includes human attackers. Generative AI has recently become a valuable asset for cybersecurity researchers and practitioners, and the field of cyber-deception is no exception. Researchers have demonstrated how LLMs can be leveraged to create realistic-looking honeytokens, fake users, and even simulated systems that can be used as honeypots.

CTU Hornet 65 Niner: A Network Dataset of Geographically Distributed Low-Interaction Honeypots

CTU Hornet 65 Niner: A Network Dataset of Geographically Distributed Low-Interaction Honeypots

This data article introduces a new network dataset created to help understand how geographical location impacts the quality, type, and amount of incoming network attacks received by honeypots. The dataset consists of 12.4 million network flows collected from nine low-interaction honeypots in nine cities across the world for 65 days, from April 29th to July 1st, 2024. Each low-interaction honeypot was identically configured to capture incoming attacks using a state-of-the-art network flow collector, Zeek. Honeypots were distributed in nine cities: Amsterdam, Bangalore, Frankfurt, London, New York, San Francisco, Singapore, Toronto, and Sydney. The dataset is in JSON format and contains all types of Zeek network flow files, including protocol-specific logs.

DeepRed: A Deep Learning–Powered Command and Control Framework for Multi-Stage Red Teaming Against ML-based Network Intrusion Detection Systems

DeepRed: A Deep Learning–Powered Command and Control Framework for Multi-Stage Red Teaming Against ML-based Network Intrusion Detection Systems

Emerging studies demonstrate that machine learning (ML) has the potential to improve the detection capabilities of network intrusion detection systems (NIDS) against evolving cyber threats. However, recent adversarial ML (AML) studies have revealed critical ML vulnerabilities. This paper presents innovative multistage red-teaming techniques to evaluate the robustness of ML-NIDS in real-world adversarial settings.

Fine-Tuning Large Language Models for DGA and DNS Exfiltration Detection

Fine-Tuning Large Language Models for DGA and DNS Exfiltration Detection

Domain Generation Algorithms (DGAs) are malicious techniques used by malware to dynamically generate seemingly random domain names for communication with Command & Control (C&C) servers. Due to the fast and simple generation of DGA domains, detection methods must be highly efficient and precise to be effective. Large Language Models (LLMs) have demonstrated their proficiency in real-time detection tasks, making them ideal candidates for detecting DGAs. Our work validates the effectiveness of fine-tuned LLMs for detecting DGAs and DNS exfiltration attacks. We developed LLM models and conducted comprehensive evaluation using a diverse dataset comprising 59 distinct real-world DGA malware families and normal domain data. Our LLM model significantly outperformed traditional natural language processing techniques, especially in detecting unknown DGAs. We also evaluated its performance on DNS exfiltration datasets, demonstrating its effectiveness in enhancing cybersecurity measures. To the best of our knowledge, this is the first work that empirically applies LLMs for DGA and DNS exfiltration detection.

Towards Better Understanding of Cybercrime: The Role of Fine-Tuned LLMs in Translation

Towards Better Understanding of Cybercrime: The Role of Fine-Tuned LLMs in Translation

Understanding cybercrime communications is paramount for cybersecurity defence. This often involves translating communications into English for processing, interpreting, and generating timely intelligence. The problem is that translation is hard. Human translation is slow, expensive, and scarce. Machine translation is inaccurate and biased. We propose using fine-tuned Large Language Models (LLM) to generate translations that can accurately capture the nuances of cybercrime language. We apply our technique to public chats from the NoName057(16) Russian-speaking hacktivist group. Our results show that our fine-tuned LLM model is better, faster, more accurate, and able to capture nuances of the language. Our method shows it is possible to achieve high-fidelity translations and significantly reduce costs by a factor ranging from 430 to 23,000 compared to a human translator.

LLM in the Shell: Generative Honeypots

LLM in the Shell: Generative Honeypots

Honeypots are essential tools in cybersecurity for early detection, threat intelligence gathering, and analysis of attacker's behavior. However, most of them lack the required realism to engage and fool human attackers long-term. Being easy to distinguish honeypots strongly hinders their effectiveness. This can happen because they are too deterministic, lack adaptability, or lack deepness. This work introduces shelLM, a dynamic and realistic software honeypot based on Large Language Models that generates Linux-like shell output. We designed and implemented shelLM using cloud-based LLMs. We evaluated if shelLM can generate output as expected from a real Linux shell. The evaluation was done by asking cybersecurity researchers to use the honeypot and give feedback if each answer from the honeypot was the expected one from a Linux shell. Results indicate that shelLM can create credible and dynamic answers capable of addressing the limitations of current honeypots. ShelLM reached a TNR of 0.90, convincing humans it was consistent with a real Linux shell. The source code and prompts for replicating the experiments have been publicly available.

Conti Inc.: Understanding the Internal Discussions of a Large Ransomware-as-a-Service Operator with Machine Learning

Conti Inc.: Understanding the Internal Discussions of a Large Ransomware-as-a-Service Operator with Machine Learning

This paper analyzes the main discussion topics in the Conti chat leak using machine learning techniques such as Natural Language Processing (NLP) and Latent Dirichlet Allocation (LDA), as well as visualization strategies. Five discussion topics are found: (1) Business, (2) Technical, (3) Internal tasking/Management, (4) Malware, and (5) Customer Service/Problem Solving. Moreover, the distribution of topics among Conti members shows that only 4% of individuals have specialized discussions while almost all individuals (96%) are all-rounders, meaning that their discussions revolve around the five topics.

Trust in Shapley: A Cooperative Quest for Global Trust in P2P Network

Trust in Shapley: A Cooperative Quest for Global Trust in P2P Network

The domain of digital networking has witnessed a lot of advancements, one of which is peer-to-peer (P2P) networks for file sharing. These networks gained significant traction with the success of Napster, BitTorrent, Gnutella, and Kazaa [13]. P2P networks offer enhanced scalability and resilience by eliminating the need to go through a server [24]. While the P2P network has transformed the file-sharing landscape, the main challenge these networks face is the ability to determine their peers’ trustworthiness accurately.

Prompt. Exploit. Repeat: Automating Network Security Testing with LLMs

Prompt. Exploit. Repeat: Automating Network Security Testing with LLMs

Large Language Models (LLMs) have demonstrated remarkable capabilities in various tasks, but their potential in cybersecurity remains largely unexplored. Despite their inherent limitations, LLM-based designs have shown promising ability in planning and navigating open-world scenarios. This paper investigates the application of pre-trained LLMs as agents in network security environments, a domain traditionally dominated by reinforcement learning (RL) approaches. We introduce a novel method that leverages LLMs for sequential decision-making in cybersecurity scenarios, that is flexible and does not need re-training to adapt to new scenarios.

Bridging the Explanation Gap in AI Security: A Task-Driven Approach to XAI Methods Evaluation

Bridging the Explanation Gap in AI Security: A Task-Driven Approach to XAI Methods Evaluation

Deciding which XAI technique is best depends not only on the domain, but also on the given task, the dataset used, the model being explained, and the target goal of that model. We argue that the evaluation of XAI methods has not been thoroughly analyzed in the network security domain, which presents a unique type of challenge. While there are XAI methods applied in network security there is still a large gap between the needs of security stakeholders and the selection of the optimal method. We propose to approach the problem by first defining the stack-holders in security and their prototypical tasks. Each task defines inputs and specific needs for explanations. Based on these explanation needs (eg understanding the performance, or stealing a model), we created five XAI evaluation techniques that are used to compare and select which XAI method is best for each task (dataset, model, and goal). Our proposed approach was evaluated by running experiments for different security stakeholders, machine learning models, and XAI methods. Results were compared with the AutoXAI technique and random selection. Results show that our proposal to evaluate and select XAI methods for network security is well-grounded and that it can help AI security practitioners find better explanations for their given tasks.

A Survey of Privacy Attacks in Machine Learning

A Survey of Privacy Attacks in Machine Learning

As machine learning becomes more widely used, the need to study its implications in security and privacy becomes more urgent. Although the body of work in privacy has been steadily growing over the past few years, research on the privacy aspects of machine learning has received less focus than the security aspects. Our contribution in this research is an analysis of more than 45 papers related to privacy attacks against machine learning that have been published during the past seven years. We propose an attack taxonomy, together with a threat model that allows the categorization of different attacks based on the adversarial knowledge, and the assets under attack. An initial exploration of the causes of privacy leaks is presented, as well as a detailed analysis of the different attacks. Finally, we present an overview of the most commonly proposed defenses and a discussion of the open problems and future directions identified during our analysis.

The Power of MEME: Adversarial Malware Creation with Model-Based Reinforcement Learning

The Power of MEME: Adversarial Malware Creation with Model-Based Reinforcement Learning

Due to the proliferation of malware, defenders are increasingly turning to automation and machine learning as part of the malware detection toolchain. However, machine learning models are susceptible to adversarial attacks, requiring the testing of model and product robustness. Meanwhile, attackers also seek to automate malware generation and evasion of antivirus systems, and defenders try to gain insight into their methods. This work proposes a new algorithm that combines Malware Evasion and Model Extraction (MEME) attacks. MEME uses model-based reinforcement learning to adversarially modify Windows executable binary samples while simultaneously training a surrogate model with a high agreement with the target model to evade. To evaluate this method, we compare it with two state-of-the-art attacks in adversarial malware creation, using three well-known published models and one antivirus product as targets. Results show that MEME outperforms the state-of-the-art methods in terms of evasion capabilities in almost all cases, producing evasive malware with an evasion rate in the range of 32–73%. It also produces surrogate models with a prediction label agreement with the respective target models between 97–99%. The surrogate could be used to fine-tune and improve the evasion rate in the future.

Stealing and Evading Malware Classifiers and Antivirus at Low False Positive Conditions

Stealing and Evading Malware Classifiers and Antivirus at Low False Positive Conditions

Model stealing attacks have been successfully used in many machine learning domains, but there is little understanding of how these attacks work against models that perform malware detection. Malware detection and, in general, security domains have unique conditions. In particular, there are very strong requirements for low false positive rates (FPR). Antivirus products (AVs) that use machine learning are very complex systems to steal, malware binaries continually change, and the whole environment is adversarial by nature. This study evaluates active learning model stealing attacks against publicly available stand-alone machine learning malware classifiers and also against antivirus products.

On the Dynamics Behind Profit-Driven Cybercrime: From Contextual Factors to Perceived Group Structures, and the Workforce at the Periphery

On the Dynamics Behind Profit-Driven Cybercrime: From Contextual Factors to Perceived Group Structures, and the Workforce at the Periphery

Through an inductive thematic analysis of semi-structured interviews with experts, this study corroborates key findings on contextual and organisational dynamics behind profit-driven cybercrime. The findings pinpoint three contextual factors influencing individuals to participate in profit-driven cybercrime: lack of legal economic opportunities, lack of deterrents, and drifting means. The findings also highlight how experts perceive group structures of those behind profit-driven cybercrime: as organised, enterprise-like, loose networks, or communities. Experts’ narratives, moreover, emphasise the presence of a workforce at the periphery of cybercrime groups. Such a workforce is not actively involved in developing criminal schemes, yet it helps their orchestration by achieving necessary tasks such as writing texts or developing software. The study results confirm key insights on crime participation related to both cyber and non-cybercrime literature while also raising new research avenues, including questions concerning to what extent those forming the peripheral workforce are willing to participate in cybercrime.

On the Motivations and Challenges of Affiliates Involved in Cybercrime

On the Motivations and Challenges of Affiliates Involved in Cybercrime

The cybercrime industry is characterised by work specialisation to the point that it has become a volume industry with various “as-a-service” offerings. One well-established “as-a-service” business model is blackmarket pay-per-install (PPI) services, which outsource the spread of malicious programmes to affiliates. Such a business model represents the archetype of specialisation in the cybercrime industry: a mass of individuals, known as affiliates, specialise in spreading malware on behalf of a service. Extant literature has focused on understanding the scope of such a service and its functioning. However, despite the large number and aggregate effect of affiliates on cybercrime, little research has been done on understanding why and how affiliates participate in such models. This study depicts the motivations and challenges of affiliates spreading Android banking Trojan applications through a blackmarket PPI service. We conducted a thematic analysis of over 6,000 of their private chat messages. The findings highlight affiliates’ labour-intensive work and precarious working conditions along with their limited income, especially compared to their expectations. Affiliates’ participation in cybercrime was found to be entangled between legal and blackmarket programmes, as affiliates did not care about programmes’ legal status as long as they yielded money. This study contributes to the literature by providing additional evidence on the downsides of work specialisation emerging from the cybercrime industry.

Large Scale Analysis of DoH Deployment on the Internet

Large Scale Analysis of DoH Deployment on the Internet

DNS over HTTPS (DoH) is one of the standards to protect the security and privacy of users. The choice of DoH provider has controversial consequences, from monopolisation of surveillance to lost visibility by network administrators and security providers. More importantly, it is a novel security business. Software products and organisations depend on users choosing well-known and trusted DoH resolvers. However, there is no comprehensive study on the number of DoH resolvers on the Internet, its growth, and the trustworthiness of the organisations behind them. This paper studies the deployment of DoH resolvers by (i) scanning the whole Internet for DoH resolvers in 2021 and 2022; (ii) creating lists of well-known DoH resolvers by the community; (iii) characterising what those resolvers are, (iv) comparing the growth and differences. Results show that (i) the number of DoH resolvers increased 4.8 times in the period 2021–2022, (ii) the number of organisations providing DoH services has doubled, and (iii) the number of DoH resolvers in 2022 is 28 times larger than the number of well-known DoH resolvers by the community. Moreover, 94% of the public DoH resolvers on the Internet are unknown to the community, 77% use certificates from free services, and 57% belong to unknown organisations or personal servers. We conclude that the number of DoH resolvers is growing at a fast rate; also that at least 30% of them are not completely trustworthy and users should be very careful when choosing a DoH resolver.

Protecting the civil society through high quality research