If you don't have anything important to say, don't say anything - Information Security Metrics
If you don’t have anything worthwhile to say, don’t say anything. (Could be one of the reasons why this blog has been silent for a while now…)
After infosec metrics became fashionable, a lot has been said about how to measure the effectiveness of your security program. ISO 27001:2013 made it worse - not only does it want you to measure the effectiveness of your security program, it also wants you to measure your information security objectives. With so many ideas and views and opinions floating around about what makes an effective security metric, I thought of writing a general guideline on how to identify what to measure in your organisation.
1. Ask - "What am I going to do with this information?"
Google “Measure information security”, and you will find hundreds of metrics and measures suggested by the who’s who of information security. Before you take the ones that seem most appealing (or the ones that seem most implementable) - stop. Ask yourself, “Once I get this metric, who is supposed to do what with it?”
This question, should be your single biggest factor
in deciding which metric to choose. You can measure anything from “The number of infosec meetings attended by the CEO” to “Mean cost to patch a vulnerability”. You can find how to set up the measure and the tools to gather the requisite data as well, but if your data is just going to make pretty graphs and not create any action items on anyone’s to-do list, then that metric is not for you!
2. Choose metrics that answer the ‘big’ questions.
The top management always seems to ask the most difficult questions - “How secure are we?” or “How compliant are we?”. Instead of quietly squirming in our seat and wondering about the purpose of life, we should provide the answer with security metrics.
How do we answer such questions? Let us think what the best answer can be. The management generally expects something of this nature - “Based on our current analysis (read security metrics) there is a 5% percent chance that one of our employees will fall prey to a phishing attempt” or “Considering previous data, there is hardly any chance that a virus attack will bring down our network.”
Our job as security professionals is to choose the metrics that best represent the holistic picture of the organisation. We could, possibly, answer “How secure are we?” by dividing it into two parts - cyber security and physical security and then identify metrics that to answer those questions. Here is an example of how the flow could be:
Level 1 - How secure are we? (Divided into cyber security and physical security)
Level 2 - How is our cyber security?(Divided into ‘network security’, ‘user cyber behaviour’ & ‘Malware protection’, etc.)
Level 3 - How secure is our network? (Now we are starting to talk statistics and metrics)
As you can see, the answer to the bigger questions rarely come from the base metrics. These are generally derived measures (metrics which are derived from the base metrics) which roll up from the base metrics to answer the key questions.
3. Choose metrics using two main criteria - Ease of collection & Reliability of data
Once you have a list of metrics from points 1 and 2 above, it is time to start choosing which metrics to start using. Here, it is best to filter metrics based on 2 factors:
Ease of data collection - You are more likely to get the data from operations if it is easy to collect. If the AV administrator can click on a button and send you a file, he is more likely to do it weekly than if he needs to manually look at some data and enter it into a complicated excel.
Reliability of data - Data generated directly from systems tends to be more reliable as compared to data that comes from manual collation. For example, it is safe to assume that your bank statement downloaded from your net-banking account is more accurate than the data in the little diary of expenses that your maintain.
Everyone has their own way of identifying and implementing security metrics. These three guiding principles should help you in choosing and implementing a metrics program that is meaningful and efficient.
Some interesting links: