Qualitative vs. quantitative: delivering better information security insight

Impact vs. risk: information security measurement for agile organizations
December 15, 2016
Show all

Qualitative vs. quantitative: delivering better information security insight

Today’s Chief Information Security Officer (CISO) has a blinding array of metrics available to describe the health of their information security organization.  Not surprisingly, as technology executives with a portfolio of the latest tools at their disposal, there’s often a heavy reliance on technical metrics.  The challenge with these metrics, as you might expect, is that they speak to technical vulnerabilities – things like mismanagement symptoms, malicious activities, or incident reports.  Dashboards routinely show things like instances of blocked attacks, number of vulnerabilities, or scores based on external scans.

Empirical values provide a sense of assurance, but without context can trigger a reactive response and mean your information security team spends time addressing non-issues. This is both the beauty and the challenge with quantitative measurement – numbers provide a sense of comfort, clear measurement, comparison, and calculation; but more detail is needed to give numbers meaning, and what’s missing from this approach is a holistic business-driven metric: maturity.

Maturity is something that people intuitively get: humans start as infants and work their way towards (usually) reasonable adults. The maturity of organizations is similar – most begin with a blank slate and get progressively more sophisticated. Maturity is useful because it is flexible; it can and should mean different things to different people; coupled with urgency of implementation, it’s clear what to do when, and why.   While quantitative and qualitative measurement approaches exist, a qualitative approach, informed by instinct and subjective data, provides a solid means of decision-making without onerous data collection or expensive tools.  A consistent approach can be followed, and with the right people in play, the data is easy to gather.

A qualitative approach is uniquely suited to capture value from team experience; although instinct is open to interpretation, it remains a critical ingredient in deciding where to focus and what to act on.  Information security professionals know what the problems are.  The issue is not awareness; it is understanding what to do, when, and what matters most.  According to Gartner, through 2020, 99% of vulnerabilities exploited will continue to be ones known by security and IT professionals for at least one year.  The issue is not lack of knowledge and the information security team has the right people to ask.

So when the executive team asks a simple question like “How are we doing?”, know that getting the answer need not be complex or involve interpreting an array of confusing data points.  Qualitatively measuring maturity, via a structured approach, yields solid results and provides a foundation from which to derive critical areas and a prioritized list of activities.