[WebAppSec] Automatic security and HackerSafe

Several people asked me about automatic assessment tools to check the security of an application stack. My opinion is that they may be a great support, but they cannot replace some manual work (oh, well, maybe). Rails test are a great way to make sure your application is safe, but you have to write them on your own. Security is not a plug-n-play product, but rather a process.

Automatic security
One automatic security scan is provided by McAfee. The HackerSafe certification is a service that detects vulnerabilities on web sites using automatic tools. If all tests pass, the site owner gets an HackerSafe logo, that he can put on his site. Over 80,000 sites, according to McAfee, use the HackerSafe logo to increase customer trust. According to the description, the service uses port scans, a vulnerability scan on the network layer, but also an automatic audit of the web application layer. On this layer, it looks for SQL Injection and Cross-Site Scripting (XSS) vulnerabilities. It is, however, not clear whether the service scans for other important attack methods, such as CSRF.

XSS and HackerSafe 
But, apparently, protection from XSS and other vulnerabilities is not a prerequisite for the HackerSafe logo. Russ McRee discovered several XSS holes in HackerSafe certified sites which allows the attacker to grab customer credentials or redirect the user to a malicious web site. He also made a video to demonstrate various XSS holes in five certified sites. Someone also found an SQL Injection vulnerability in a HackerSafe certified site. And I've seen at least one HackerSafe site which is vulnerable to CSRF. Although, maybe McAfee does not scan for CSRF, it is definitely an important attack method.
McAfee reacts 
A spokesperson for McAfee told British media that the company does not find XSS holes to be as critical as SQL Injection or other vulnerabilities. According to her, an XSS hole would not lead to revocation of the HackerSafe certification, although the company informs the customer about it. She added that XSS holes cannot be exploited to break into a server. No they cannot, but an attacker may steal the user's login credentials or credit card number, install malware on the client's computer or snitch his authentication cookie.
HackerSafe – just an image? 
After all, the HackerSafe certificate means that the site is protected against known attacks. It does, however, not mean that the site is not vulnerable to important attacks on the web application layer, which accounts for "an estimated 70% of all security breaches", according to the McAfee web site. And it does not mean that the site has no logic flaws, which gives an attacker (or another user) access to information he is not allowed to see. This can only be done by a manual security audit.