Catania

Building Adaptative and Transparent Cyber Agents with Local Language Models

Building Adaptative and Transparent Cyber Agents with Local Language Models

Autonomous intelligent agents offer a transformative approach to cyber defense by operating independently in complex and dynamic environments. While much research has focused on defensive systems, offensive agents are equally important for testing system resilience and improving defenses through realistic adversarial interaction. Recent advances demonstrate that large language models can automate penetration testing effectively, rivaling traditional reinforcement learning methods, but their reliance on cloud-based services introduces significant concerns around privacy and reproducibility.

Out of the Cage: How Stochastic Parrots Win in Cyber Security Environments

Out of the Cage: How Stochastic Parrots Win in Cyber Security Environments

Large Language Models (LLMs) have gained widespread popularity across diverse domains involving text generation, summarization, and various natural language processing tasks. Despite their inherent limitations, LLM-based designs have shown promising capabilities in planning and navigating open-world scenarios. This paper introduces a novel application of pre-trained LLMs as agents within cybersecurity network environments, focusing on their utility for sequential decision-making processes.

Building Adaptative and Transparent Cyber Agents with Local Language Models

Building Adaptative and Transparent Cyber Agents with Local Language Models

Autonomous intelligent agents offer a transformative approach to cyber defense by operating independently in complex and dynamic environments. While much research has focused on defensive systems, offensive agents are equally important for testing system resilience and improving defenses through realistic adversarial interaction. Recent advances demonstrate that large language models can automate penetration testing effectively, rivaling traditional reinforcement learning methods, but their reliance on cloud-based services introduces significant concerns around privacy and reproducibility.

VelLMes: A High-Interaction AI-Based Deception Framework

VelLMes: A High-Interaction AI-Based Deception Framework

There are very few SotA deception systems based on Large Language Models. The existing ones are limited only to simulating one type of service, mainly SSH shells. These systems - but also the deception technologies not based on LLMs - lack an extensive evaluation that includes human attackers. Generative AI has recently become a valuable asset for cybersecurity researchers and practitioners, and the field of cyber-deception is no exception. Researchers have demonstrated how LLMs can be leveraged to create realistic-looking honeytokens, fake users, and even simulated systems that can be used as honeypots.

Towards Better Understanding of Cybercrime: The Role of Fine-Tuned LLMs in Translation

Towards Better Understanding of Cybercrime: The Role of Fine-Tuned LLMs in Translation

Understanding cybercrime communications is paramount for cybersecurity defence. This often involves translating communications into English for processing, interpreting, and generating timely intelligence. The problem is that translation is hard. Human translation is slow, expensive, and scarce. Machine translation is inaccurate and biased. We propose using fine-tuned Large Language Models (LLM) to generate translations that can accurately capture the nuances of cybercrime language. We apply our technique to public chats from the NoName057(16) Russian-speaking hacktivist group. Our results show that our fine-tuned LLM model is better, faster, more accurate, and able to capture nuances of the language. Our method shows it is possible to achieve high-fidelity translations and significantly reduce costs by a factor ranging from 430 to 23,000 compared to a human translator.

LLM in the Shell: Generative Honeypots

LLM in the Shell: Generative Honeypots

Honeypots are essential tools in cybersecurity for early detection, threat intelligence gathering, and analysis of attacker's behavior. However, most of them lack the required realism to engage and fool human attackers long-term. Being easy to distinguish honeypots strongly hinders their effectiveness. This can happen because they are too deterministic, lack adaptability, or lack deepness. This work introduces shelLM, a dynamic and realistic software honeypot based on Large Language Models that generates Linux-like shell output. We designed and implemented shelLM using cloud-based LLMs. We evaluated if shelLM can generate output as expected from a real Linux shell. The evaluation was done by asking cybersecurity researchers to use the honeypot and give feedback if each answer from the honeypot was the expected one from a Linux shell. Results indicate that shelLM can create credible and dynamic answers capable of addressing the limitations of current honeypots. ShelLM reached a TNR of 0.90, convincing humans it was consistent with a real Linux shell. The source code and prompts for replicating the experiments have been publicly available.

Prompt. Exploit. Repeat: Automating Network Security Testing with LLMs

Prompt. Exploit. Repeat: Automating Network Security Testing with LLMs

Large Language Models (LLMs) have demonstrated remarkable capabilities in various tasks, but their potential in cybersecurity remains largely unexplored. Despite their inherent limitations, LLM-based designs have shown promising ability in planning and navigating open-world scenarios. This paper investigates the application of pre-trained LLMs as agents in network security environments, a domain traditionally dominated by reinforcement learning (RL) approaches. We introduce a novel method that leverages LLMs for sequential decision-making in cybersecurity scenarios, that is flexible and does not need re-training to adapt to new scenarios.

Detecting DNS Threats: A Deep Learning Model to Rule Them All

Domain Name Service is a central part of Internet regular operation. Such importance has made it a common target of different malicious behaviors such as the application of Domain Generation Algorithms (DGA) for command and control a group of infected computers or Tunneling techniques for bypassing system administrator restrictions. A common detection approach is based on training different models detecting DGA and Tunneling capable of performing a lexicographic discrimination of the domain names. However, since both DGA and Tunneling showed domain names with observable lexicographical differences with normal domains, it is reasonable to apply the same detection approach to both threats. In the present work, we propose a multi-class convolutional network (MC-CNN) capable of detecting both DNS threats. The resulting MC-CNN is able to detect correctly 99% of normal domains, 97% of DGA and 92% of Tunneling, with a False Positive Rate of 2.8%, 0.7% and 0.0015% respectively and the advantage of having 44% fewer trainable parameters than similar models applied to DNS threats detection.

Deep Convolutional Neural Networks for DGA Detection

A Domain Generation Algorithm (DGA) is an algorithm to generate domain names in a deterministic but seemly random way. Malware use DGAs to generate the next domain to access the Command & Control (C&C) communication server. Given the simplicity of the generation process and speed at which the domains are generated, a fast and accurate detection method is required. Convolutional neural network (CNN) are well known for performing real-time detection in fields like image and video recognition. Therefore, they seemed suitable for DGA detection. The present work provides an analysis and comparison of the detection performance of a CNN for DGA detection. A CNN with a minimal architecture complexity was evaluated on a dataset with 51 DGA malware families and normal domains. Despite its simple architecture, the resulting CNN model correctly detected more than 97% of total DGA domains with a false positive rate close to 0.7%.

An Analysis of Convolutional Neural Networks for detecting DGA

A Domain Generation Algorithm (DGA) is an algorithm to generate domain names in a deterministic but seemly random way. Malware use DGAs to generate the next domain to access the Command Control (C&C) communication channel. Given the simplicity and velocity associated to the domain generation process, machine learning detection methods emerged as suitable detection solution. However, since the periodical retraining becomes mandatory, a fast and accurate detection method is needed. Convolutional neural network (CNN) are well known for performing real-time detection in fields like image and video recognition. Therefore, they seem suitable for DGA detection. The present work is a preliminary analysis of the detection performance of CNN for DGA detection. A CNN with a minimal architecture complexity was evaluated on a dataset with 51 DGA malware families as well as normal domains. Despite its simple architecture, the resulting CNN model correctly detected more than 97% of total DGA domains with a false positive rate close to 0.7%.

Detecting DGA malware traffic through behavioral models

Some botnets use special algorithms to generate the domain names they need to connect to their command and control servers. They are refereed as Domain Generation Algorithms. Domain Generation Algorithms generate domain names and tries to resolve their IP addresses. If the domain has an IP address, it is used to connect to that command and control server. Otherwise, the DGA generates a new domain and keeps trying to connect. In both cases it is possible to capture and analyze the special behavior shown by those DNS packets in the network. The behavior of Domain Generation Algorithms is difficult to automatically detect because each domain is usually randomly generated and therefore unpredictable. Hence, it is challenging to separate the DNS traffic generated by malware from the DNS traffic generated by normal computers. In this work we analyze the use of behavioral detection approaches based on Markov Models to differentiate Domain Generation Algorithms traffic from normal DNS traffic. The evaluation methodology of our detection models has focused on a real-time approach based on the use of time windows for reporting the alerts. All the detection models have shown a clear differentiation between normal and malicious DNS traffic and most have also shown a good detection rate. We believe this work is a further step in using behavioral models for network detection and we hope to facilitate the development of more general and better behavioral detection methods of malware traffic.

Protecting the civil society through high quality research