ISSRR Workshop Home

Frequently Asked Questions

1st ISSRR
May 2001

Proceedings (PDF)

Position Papers

Workshop Photos

ACSA Home

CALL FOR PARTICIPATION

FIRST WORKSHOP ON INFORMATION-SECURITY-SYSTEM RATING AND RANKING

(commonly but improperly known as "Security Metrics")

Sponsors

	Applied Computer Security Associates (ACSA)
	The MITRE Corporation

HIGHLIGHTS

• Date: Monday, May 21, 2001, 1:00 pm - Wednesday May 23, 2001, 12:00 pm
• Location: Williamsburg, Virginia
• Cost: $130
• Intellectual price of admission: Position paper (approximately 5 pages) germane to any aspect of the subject. Due no later than March 30, 2001. See Instructions below.
• About the position papers: Position papers will be grouped by topic and posted on the ACSA Conference Web site. These position papers are solely for the purpose of exchanging information among the workshop participants and will not be presented at the workshop. Attendees are expected to read the position papers prior to workshop attendance. Position papers may be used to identify topics for discussion and develop workshop structure.
• About the workshop: The workshop will be devoted to discussing the topics and producing a summary statement for each topic that satisfies the workshop goals, identified below. The position papers, final workshop agenda and registration details will be posted to http://www.acsac.org/measurement/. Please watch this site for updates.

INTRODUCTION

The available security technologies for any one application suite are multiple and mysterious, not to mention costly and sometimes inconvenient to the point of crippling. The confluence of several such suites in an integrated environment is not only common but mandated in the enterprise, and these suites are often difficult to evaluate for information security characteristics. When all the parts play together, are they pushovers for the teenaged vandal, or worse the vengeful laid-off former insider, or perhaps the identity thief? Too often, the answer is "yes."

"Security metrics" has at least a 20-year history involving product evaluation criteria identification, Information Assurance (IA) quantification, risk assessment/analysis methodology development, and other related activities. These activities have led to the widespread desire for a single number/digraph by which to rate/buy/commit to dangerous operation/improvement/retirement of an IT system, much as the military has single-digit Readiness and INFOCON levels, now cliches in TV pot-boilers.

Computer science has steadily frustrated these activities: it has provided neither generally accepted nor reliable measures for rating IT security or requisite security assurance. Also, inconsistent terminology usage has complicated the development of IT metrics, such as rating, ranking, quantifying, or scoring measurements. Further, computer scientists have severely criticized some efforts, even R&D activities, on the topic as attempts to square the circle.

As one of our participating organizers put it:

DARPA has been urging the application of the scientific method to a number of problem areas in information assurance and computer security. The push to develop quantifiable measures of assurance is but one step in this progression. While the goals of this movement are laudable, the underlying science is sadly lacking. Software development is, at best, in the craft era. The state of practice, as typified by commercial products such as Windows and many of its (too) closely integrated applications, is abominable. I would like to believe that metrics relating to security are possible, but there is little evidence to support this view at present. [Click Here to read position paper] Why has software devolved to this state? What can be done to help the needy consumer in this situation? What can be done about current "measures" that are expensive to generate and mean little, or worse, could seriously mislead?

The organizers have called a workshop that we hope will:

• Clarify what researchers and practitioners are talking about when they refer to IA metrics.
• Debunk the pseudo-science associated with assurance metrics.
• Find some indirect indicators of security.
• Precisely define the research problems in developing IA metrics methodologies.

• Recap the latest thinking on current IA metrics activities.
• Identify efforts that are successful in some sense, if they exist, and if none exist, reduce expectations on what might be achieved through IA metrics.
• Explore the unintended side effects of ratings/measures (e.g., inflating the numbers to ensure promotion, delay review by higher authority)
• Clarify what's measurable and what's not.
• Scope and characterize the measures to be addressed: For example, what does an IA measure of EJB Security, CORBA Security, and/or Microsoft DNA Security mean, assuming one or more exist? And explain what happens when several of these measures/applications co-exist in the same enterprise; do they augment each other or cancel each other out?
• Describe how measures should be used in the context of IA, especially to influence purchases and for general resource allocations.
• Identify misapplications of measures, including their description as "metrics"

INSTRUCTIONS FOR PREPARING YOUR POSITION PAPER:

• Rich Text Format (.RTF)
• Acrobat or PostScript format (.pdf or .ps)
• ASCII text (only if absolutely necessary)
• Microsoft Word

• Title or Topic Abstract (not to exceed 250 words)
• Author(s), Organizational Affiliation
• Phone numbers (voice and fax)
• Email address
• Point of Contact, if more than one author

Classified material or topics should NOT be submitted.

In addition to providing the position paper, authors are requested to submit a one page (or less) justification that cites the following information: (1) relates the subject of their position paper to the topic of the workshop; (2) identifies no more than two discussion topics, cited in their position paper, that they believe should be addressed by the workshop; (3) outlines their qualifications for participation; and (4) indicates their willingness to participate as a panel chair.

Send your position paper as an email attachment to position-paper@acsac.org. Authors of relevant position papers will receive registration information by email. If you do not receive acknowledgment of your electronic submission within three weeks of sending it, please send email to Marshall Abrams.

Authors and panel chairs selected to participate in the conference will be notified by April 20, 2001.

The position papers, final workshop agenda, and registration details will be posted to http://www.acsac.org/measurement/. Please watch this site for updates.

WORKSHOP COMMITTEE

Members:

Marshall Abrams, MITRE

Julie Connolly, MITRE

Jay Kahn, MITRE

Stuart Katzke, NSA

John McHugh, CERT

Don Peeples, Sparta

Ray Vaughn, Mississippi State University