Several people asked me about automatic assessment tools to check the security of an application stack. My opinion is that they may be a great support, but they cannot replace some manual work (oh, well, maybe). Rails test are a great way to make sure your application is safe, but you have to write them on your own. Security is not a plug-n-play product, but rather a process.
One automatic security scan is provided by McAfee. The HackerSafe certification is a service that detects vulnerabilities on web sites using automatic tools. If all tests pass, the site owner gets an HackerSafe logo, that he can put on his site. Over 80,000 sites, according to McAfee, use the HackerSafe logo to increase customer trust. According to the description, the service uses port scans, a vulnerability scan on the network layer, but also an automatic audit of the web application layer. On this layer, it looks for SQL Injection and Cross-Site Scripting (XSS) vulnerabilities. It is, however, not clear whether the service scans for other important attack methods, such as CSRF.