Deciding which XAI technique is best depends not only on the domain, but also on the given task, the dataset used, the model being explained, and the target goal of that model. We argue that the evaluation of XAI methods has not been thoroughly analyzed in the network security domain, which presents a unique type of challenge. While there are XAI methods applied in network security there is still a large gap between the needs of security stakeholders and the selection of the optimal method. We propose to approach the problem by first defining the stack-holders in security and their prototypical tasks. Each task defines inputs and specific needs for explanations. Based on these explanation needs (eg understanding the performance, or stealing a model), we created five XAI evaluation techniques that are used to compare and select which XAI method is best for each task (dataset, model, and goal). Our proposed approach was evaluated by running experiments for different security stakeholders, machine learning models, and XAI methods. Results were compared with the AutoXAI technique and random selection. Results show that our proposal to evaluate and select XAI methods for network security is well-grounded and that it can help AI security practitioners find better explanations for their given tasks.

Honeypots have been a long-established form of passive defense in a wide variety of systems. They are often used for the reliability and low false positive rate. However, the deployment of honeypots in the Active Directory (AD) systems is still limited. Intrusion detection in AD systems is a difficult task due to the complexity of the system and its design, where any authenticated account is able to query other entities in the system. Therefore, the positioning of the honeypot in such structures brings two main con trains: (i) the placement has to be organic, with similar properties to other, real entities in the structure, and (ii) the placement must not give away the nature of the honeypot to the attacker.