Safety Score as an Evaluation Metric for Machine Learning Models of Security Applications

Machine learning studies have traditionally used accuracy, F1 score, etc. to measure the goodness of models. We show that these conventional metrics do not necessarily represent risks in security applications and may result in models that are not optimal. This letter proposes 'Safety score' as an evaluation metric that incorporates the cost associated with model predictions. The proposed metric is easy to explain to system administrators. We evaluate the new metric for two security applications: general intrusion detection and injection attack detection. Compared to other metrics, Safety score proves its efficiency in indicating the risk in using the model.