ICVTest: A Practical Black-Box Penetration Testing Framework for Evaluating Cybersecurity of Intelligent Connected Vehicles

Intelligent connected vehicles (ICVs) are equipped with extensive electronic control units which offer convenience but also pose significant cybersecurity risks. Penetration testing, recommended in ISO/SAE 21434 “Road vehicles—Cybersecurity engineering”, is an effective approach to identify cybersecurity vulnerabilities in ICVs. However, there is limited research on vehicle penetration testing from a black-box perspective due to the complex architecture of ICVs. Additionally, no penetration testing framework has been proposed to guide security testers on conducting penetration testing for the whole vehicle. The lack of framework guidance results in the inexperienced security testers being uncertain about the processes to follow for conducting penetration testing. Moreover, the inexperienced security testers are unsure about which tests to perform in order to systematically evaluate the vehicle’s cybersecurity. To enhance the penetration testing efficiency of ICVs, this paper presents a black-box penetration testing framework, ICVTest. ICVTest proposes a standardized penetration testing process to facilitate step-by-step completion of the penetration testing, thereby addressing the issue of inexperienced testers lacking guidance on how to initiate work when confronted with ICV. Also, ICVTest includes 10 sets of test cases covering hardware and software security tests. Testers can select appropriate test cases based on the specific cybersecurity threats faced by the target object, thereby reducing the complexity of penetration testing tasks. Furthermore, we have developed a vehicle cybersecurity testing platform for ICVTest that seamlessly integrates various testing tools. The platform enables even novice testers to conduct vehicle black-box penetration testing in accordance with the given guidance which addresses the current industry’s challenge of an overwhelming number of testing tasks coupled with a shortage of skilled professionals. For the first time, we propose a comprehensive black-box penetration testing framework and implement the framework in the form of a cybersecurity testing platform. We apply ICVTest to evaluate an electric vehicle manufactured in 2021 for assessing the framework’s availability. With the aid of ICVTest, even testers with limited experience in automotive penetration can effectively evaluate the security risks of ICVs. In our experiments, numerous cybersecurity vulnerabilities were identified involving in-vehicle sensors, remote vehicle control systems, and in-vehicle controller area network (CAN) bus.