Over the years we've all seen what appears to be a never-ending stream of security reports that supposedly help make it clear how secure a given software technology might be. There have been several such reports on individual products, as well as several comparative reports that pit products against each other--for example, Internet Explorer (IE) versus Firefox, or Windows versus one more Linux-based or BSD-based OS.
Although those reports are helpful, they always leave room for interpretation, particularly because they invariably rely on a flat count of known security bugs. Heated debates are often the result, and in the end some security administrators might misunderstand the overall implications and relative usefulness of such data. Not only that, but people who aren't so savvy about security can become misinformed because they don't have the requisite knowledge to understand security to begin with--which can lead to ridiculous beliefs such as "OS X is bulletproof, so I don't need a firewall or antivirus protection."
Mozilla recently announced that it intends to take a different approach in the future for measuring the security of its own products, including Firefox. The company said that instead of counting bugs, Mozilla will track various metrics over time. Hopefully that sort of information will lead to much better information and better products, which of course would benefit everyone.
Toward that goal, Mozilla has launched a new project currently called the Mozilla Security Metrics Project. Now in its initial stages of development, the project is the offspring of collaboration between Mozilla security team members and security researcher/analyst Rich Mogull of Securosis-–who formerly worked as a security analyst for Gartner Group. The stated goals of the project are to "develop a metrics based model to track the relative security of Firefox, evaluate the effectiveness of security efforts within the development and testing process, and measure the window of exposure of Firefox users to security vulnerabilities."
Taking that a step further, the company also said that a secondary goal is to "develop an open base model that can be standardized and expanded upon for other software development efforts to achieve the same goals." Thinking about those goals, it seems apparent that a primary focus is to improve overall the development software process, as well as to get as many users protected as soon as possible in the event of a security vulnerability.
As you might expect, the public is invited to participate in helping to shape guidelines for the project. And, as it stands now in its early initial stages, what exists is a spreadsheet document that includes the proposed metrics to be tracked, information on how to categorize and label those metrics, and a couple of channels to discuss the proposed metrics as well as how to proceed with project development.
If you're reading this article, you're obviously interested in security. If you also use Mozilla software, perhaps you'd like to participate in and possibly help shape the project. You can do so by reviewing the information currently available and providing your feedback. To take part, head over to the Mozilla Security Blog and read the related post at the URL below. There you'll find links to the existing information, as well as a contact email address to send comments and suggestions. You'll also find the usual blog comment form if you want to post your suggestions openly.
http://blog.mozilla.com/security/2008/07/02/mozilla-security-metrics-project