Information Extraction of Cybersecurity Concepts: An LSTM Approach
Source abstract
Abstract
Extracting cybersecurity entities and the relationships between them from online textual resources such as articles, bulletins, and blogs and converting these resources into more structured and formal representations has important applications in cybersecurity research and is valuable for professional practitioners. Previous works to accomplish this task were mainly based on utilizing feature-based models. Feature-based models are time-consuming and need labor-intensive feature engineering to describe the properties of entities, domain knowledge, entity context, and linguistic characteristics. Therefore, to alleviate the need for feature engineering, we propose the usage of neural network models, specifically the long short-term memory (LSTM) models to accomplish the tasks of Named Entity Recognition (NER) and Relation Extraction (RE). We evaluated the proposed models on two tasks. The first task is performing NER and evaluating the results against the state-of-the-art Conditional Random Fields (CRFs) method. The second task is performing RE using three LSTM models and comparing their results to assess which model is more suitable for the domain of cybersecurity. The proposed models achieved competitive performance with less feature-engineering work. We demonstrate that exploiting neural network models in cybersecurity text mining is effective and practical.
Source: crossref · Quality: baseline
Research context
OpenAlex topics: Topic Modeling · Software Engineering Research · Network Security and Intrusion Detection
SciVX discovery tags are terminology-based. OpenAlex topics are source-provided classifications.
Affiliations
DISP Laboratory, Université Lumière Lyon 2, 69500 Lyon, France
Computer Science Department, College of Engineering, Qatar University, Doha P.O. Box 2713, Qatar;
References (35)
Show source-provided references
1. Finkel, J.R., Grenager, T., and Manning, C. (2005, January 25–30). Incorporating non-local information into information extraction systems by gibbs sampling. Proceedings of the 43rd Annual Meeting on Association for Computational Linguistics, Ann Arbor, MI, USA.10.3115/1219840.1219885 ↗
2. Bridges, R.A., Jones, C.L., Iannacone, M.D., Testa, K.M., and Goodall, J.R. (2013). Automatic labeling for entity extraction in cyber security. arXiv, Available online: https://arxiv.org/abs/1308.4941.
3. (2019, July 24). National Vulnerability Database, Available online: https://nvd.nist.gov/.
4. Liao, X., Yuan, K., Wang, X., Li, Z., Xing, L., and Beyah, R. (2016, January 24–28). Acing the ioc game: Toward automatic discovery and analysis of open-source cyber threat intelligence. Proceedings of the 2016 ACM SIGSAC Conference on Computer and Communications Security, Vienna, Austria.10.1145/2976749.2978315 ↗
5. More, S., Matthews, M., Joshi, A., and Finin, T. (2012, January 24–25). A knowledge-based approach to intrusion detection modeling. Proceedings of the IEEE Symposium on Security and Privacy Workshops (SPW), San Francisco, CA, USA.10.1109/SPW.2012.26 ↗
6. Nguyen, T.H., and Grishman, R. (2015, January 26–31). Event detection and domain adaptation with convolutional neural networks. Proceedings of the 53rd Annual Meeting of the Association for Computational Linguistics and the 7th International Joint Conference on Natural Language Processing, Beijing, China.10.3115/v1/P15-2060 ↗
7. Deep learning in neural networks: An overview10.1016/j.neunet.2014.09.003 ↗
8. End-to-End Recurrent Neural Network Models for Vietnamese Named Entity Recognition: Word-Level vs. Character-Level
9. Athavale, V., Bharadwaj, S., Pamecha, M., Prabhu, A., and Shrivastava, M. (2016). Towards deep learning in hindi ner: An approach to tackle the labelled data scarcity. arXiv, Available online: https://arxiv.org/abs/1610.09756.
10. Jones, C.L., Bridges, R.A., Huffer, K.M., and Goodall, J.R. (2015, January 7–9). Towards a relation extraction framework for cyber-security concepts. Proceedings of the 10th Annual Cyber and Information Security Research Conference, Oak Ridge, TN, USA.10.1145/2746266.2746277 ↗
11. Joshi, A., Lal, R., Finin, T., and Joshi, A. (2013, January 16–18). Extracting cybersecurity related linked data from text. Proceedings of the 2013 IEEE Seventh International Conference on Semantic Computing (ICSC), Irvine, CA, USA.10.1109/ICSC.2013.50 ↗
12. Collins, M. (2002, January 7–12). Discriminative training methods for hidden markov models: Theory and experiments with perceptron algorithms. Proceedings of the ACL-02 Conference on Empirical Methods in Natural Language Processing, Stroudsburg, PA, USA.10.3115/1118693.1118694 ↗
13. Mulwad, V., Li, W., Joshi, A., Finin, T., and Viswanathan, K. (2011, January 22–27). Extracting information about security vulnerabilities from web text. Proceedings of the 2011 IEEE/WIC/ACM International Conference on Web Intelligence and Intelligent Agent Technology (WI-IAT), Lyon, France.10.1109/WI-IAT.2011.26 ↗
14. McNeil, N., Bridges, R.A., Iannacone, M.D., Czejdo, B., Perez, N., and Goodall, J.R. (2013, January 4–7). Pace: Pattern accurate computationally efficient bootstrapping for timely discovery of cyber-security concepts. Proceedings of the 12th International Conference on Machine Learning and Applications (ICMLA), Miami, FL, USA.10.1109/ICMLA.2013.106 ↗
15. Bridges, R.A., Huffer, K.M., Jones, C.L., Iannacone, M.D., and Goodall, J.R. (2017, January 18–21). Cybersecurity Automated Information Extraction Techniques: Drawbacks of Current Methods, and Enhanced Extractors. Proceedings of the 16th IEEE International Conference on Machine Learning and Applications (ICMLA), Cancun, Mexico.10.1109/ICMLA.2017.0-122 ↗
16. A primer on neural network models for natural language processing10.1613/jair.4992 ↗
17. Graves, A., Mohamed, A., and Hinton, G. (2013, January 26–31). Speech recognition with deep recurrent neural networks. Proceedings of the 2013 IEEE International Conference on Acoustics, Speech and Signal Processing (Icassp), Vancouver, BC, Canada.10.1109/ICASSP.2013.6638947 ↗
18. Gers, F.A., Schmidhuber, J., and Cummins, F. (1999, January 7–10). Learning to forget: Continual prediction with LSTM. Proceedings of the 9th International Conference on Artificial Neural Networks: ICANN’99, Edinburgh, UK.10.1049/cp:19991218 ↗
19. Lample, G., Ballesteros, M., Subramanian, S., Kawakami, K., and Dyer, C. (2016). Neural architectures for named entity recognition. arXiv, Available online: https://arxiv.org/abs/1603.01360.10.18653/v1/N16-1030 ↗
20. Tjong Kim Sang, E.F., and Buchholz, S. (2000, January 13–14). Introduction to the CoNLL-2000 shared task: Chunking. Proceedings of the 2nd Workshop on Learning Language in Logic and the 4th Conference on Computational Natural Language Learning, Lisbon, Portugal.10.3115/1117601.1117631 ↗
21. Okazaki, N. (2019, July 24). CRFsuite: A fast implementation of Conditional Random Fields (CRFs). Available online: http://www.chokkan.org/software/crfsuite/.
22. Xu, Y., Mou, L., Li, G., Chen, Y., Peng, H., and Jin, Z. (2015, January 17–21). Classifying relations via long short term memory networks along shortest dependency paths. Proceedings of the 2015 Conference on Empirical Methods in Natural Language Processing, Lisbon, Portugal.10.18653/v1/D15-1206 ↗
23. Miwa, M., and Bansal, M. (2016). End-to-end relation extraction using lstms on sequences and tree structures. arXiv, Available online: https://arxiv.org/abs/1601.00770.10.18653/v1/P16-1105 ↗
24. Long short-term memory10.1162/neco.1997.9.8.1735 ↗
25. Sha, L., Chang, B., Sui, Z., and Li, S. (2016, January 11–16). Reading and thinking: Re-read lstm unit for textual entailment recognition. Proceedings of the COLING 2016, the 26th International Conference on Computational Linguistics: Technical Papers, Osaka, Japan.
26. Deep learning with word embeddings improves biomedical named entity recognition10.1093/bioinformatics/btx228 ↗
27. Mikolov, T., Sutskever, I., Chen, K., Corrado, G.S., and Dean, J. (2013, January 5–10). Distributed representations of words and phrases and their compositionality. Proceedings of the Advances in Neural Information Processing Systems, Lake Tahoe, NV, USA.
28. Implementing the Viterbi algorithm10.1109/79.410439 ↗
29. Huang, Z., Xu, W., and Yu, K. (2015). Bidirectional LSTM-CRF models for sequence tagging. arXiv, Available online: https://arxiv.org/abs/1508.01991.
30. Others Backpropagation through time: What it does and how to do it10.1109/5.58337 ↗
31. (2019, July 24). Stucco, Available online: https://www.ornl.gov/division/projects/stucco.
32. (2019, July 24). CVE—Common Vulnerabilities and Exposures (CVE). Available online: https://cve.mitre.org/.
33. Tjong Kim Sang, E.F., and De Meulder, F. (June, January 31). Introduction to the CoNLL-2003 shared task: Language-independent named entity recognition. Proceedings of the Seventh Conference on Natural Language Learning at HLT-NAACL 2003, Edmonton, AB, Canada.10.3115/1119176.1119195 ↗
34. (2019, July 24). Google Code Archive—Long-term storage for Google Code Project Hosting. Available online: https://code.google.com/archive/p/word2vec/.
35. (2019, July 24). Natural Language Toolkit—NLTK 3.4.4 documentation. Available online: https://www.nltk.org/.