Context
Black-box testing is the most common approach used for identifying vulnerabilities by testing them dynamically. Tools that use a dynamic or Black-box testing approach are usually called Web Vulnerability Scanners (WVS).
Contribution: comprehensive systematic literature review on the effectiveness and characteristics of WVS
Approach
30 WVS are evaluated
- popularity
- license type
- last update date
- scanner technology
- run platform
- user interface type
- documentation availability
- capability of detecting OWASP top 10
Black-box vulnerability test:
- Planning phase: The rules and objectives for the test can be set in this phase
- Discovery phase: This phase is divided into two stages. The first stage includes the initiation of the test and the collection of information. The second stage performs vulnerability analysis, which occurs after the attack phase
- Attack phase: This phase examines the various vulnerabilities in the target application, which is also known as ‘‘the heart of the test’’
- Reporting phase: This phase provides documentation with a combination of other phases.
Results
- SQLIA (SQL injection attack) and XSS (cross site scripting) were the most common tested types among the OWASP Top Ten vulnerability types. The other types of vulnerabilities in the OWASP Top Ten list were almost not tested.
- A total of 13 studies evaluated SQLi and 8 studies evaluated XSS performance for several scanners; However, most studies only evaluated one or two scanners against only one or two non-standard web applications
- After analyzing and collating the efficacy results as published in the 15 evaluations, we found disparate and inconsistent efficacy reports
- We found no published evaluations assessing the usability or quality of use of web vulnerability scanners
- currently web scanners are not properly tested on a set of vulnerable web applications that can be used as reference. New standard and representative benchmark web applications should be created.
- Lack of standardization: evaluations of web vulnerability scanners should be based on the OWASP Top Ten vulnerability types or other common nomenclature for web vulnerabilities.