Cyber Reporting Framework
It is an extremely productive exercise for a Security Operations Centre to establish a Reporting Framework that agrees on the types of Security Reporting that it would need to produce, how to produce it, when to produce it, what format it should be in and who would receive and how frequently.
Security Reporting Framework also provides a high-level view of the content of all reports, dashboards, frequency of production, production formats, delivery mechanisms and storage options.
Following definitions are provided for clarification purpose only:
Reports: Reports produced by the KPMG GSOC refer to an official document which may be produced as a one-off or periodically to reflect outcome of a thorough investigation / consideration or present amalgamation of facts by an appointed person or a body.
Dashboards: A dashboard is a user interface that organizes and presents information in a way that is easy to read. This includes but is not limited to persona based dashboards provided by RSA Archer SecOps. Dashboards may amalgamate one or more KPIs.
KPIs: A performance indicator or key performance indicator (KPI) is a type of performance measurement to reflect how the metrics being measured is performing. While KPIs provides building blocks for a number of reports, not all reports are based on KPIs especially where the elements being reported are more of a subjective nature.
Confidence Level: A confidence level value is usually attached to a threat intelligence advisory to represent its relevance to the environment being monitored by KPMG GSOC.
All types of Security Reporting can be classified into one or more of the following:
Individual Key Performance Indicators: A performance indicator or key performance indicator (KPI) is a type of performance measurement to reflect how the metrics being measured in performing. Security KPIs evaluate the success of security people, processes and technologies.
Reports: A report is a document that presents information in an organized format for a specific audience and purpose. Although summaries of reports may be delivered orally, complete reports are almost always in the form of written documents as a one-off or on a periodic basis.
Dashboards: A dashboard is a type of graphical user interface which often provides at-a-glance views of key performance indicators relevant to a particular objective or business process. In another usage, “dashboard” is another name for “progress report” or “report” and considered a form of data visualization. Security Dashboards are recommended to be persona-based to suit the needs of different stakeholders.
Building Blocks of the Report
Any reporting comprise of at least one or more of the following building blocks to convey meaningful information:
Rationale: SOCs suffer from information overload more than any other part of the business. Nothing then could be more detrimental to the efficient operations of the SOC to further add to that noise by producing any additional information without answering the fundamental questions of What, Why, Who, Where and How. Every metric that needs to be measured should not be considered, every dashboard that needs to setup should not be setup and every report that needs to be produced should not be produced without answering these five questions.
Therefore, the defining the Rationale becomes the first, and the most important, building block.
Metric/KPI: Metric and/or KPIs are the fundamental, atomic building block of reports and dashboards that deduce performance through standalone measures or aggregation of multiple data points. One example of such a metric is Mean Time To Respond (MTTR). While setting up your reporting framework, it is imperative to make sure you only identify the KPIs that matter and also to name them in an unambiguous manner.
Data Source: It defines the data sources which would be utilised to deduce this value. The authenticity, currency and the validity of the data sources contribute to the accuracy of the measurements provided by the selected metric.
Period: Defines the time period covered by the report or the dashboard, and also help compare and validate patterns between separate periods of time.
Frequency: How frequently a metric is measured, or a report or a dashboard is produced, e.g. yearly, quarterly, weekly or daily.
Method of Data Collection: Defines whether the method of collection of metrics and data required for the generation Reports or Dashboards is manual, automatic or hybrid.
Method of Report Generation: Defines whether the method of the generation of Reports or Dashboards is manual, automatic or hybrid.
Reporting System: Defines the reporting system being followed for each report and dashboard. Traffic Light Reporting (TLR) system offers a simple yet effective way to provide situational awareness using RED, AMBER and GREEN colours.
Type of Report: Defines which category the report or dashboard falls into:
- Baseline Reports, provide variance analysis against the established baseline and/or expected values.
- Trend Reports, provide insight into ongoing trend for a particular metric.
- Status Reports, provide update into the state of affairs for a particualr metric or a group of metrics.
- Notifications, provides regular updates on status or share advisory information.
- Alerts, provides information on out of bound occurrences.
Distribution Mechanism: Decides how each reporting format would be distributed to the target audience. Distribution mechanism should be secure and promote good security practices. For example, instead of sharing PDF reports through email, they should instead be upload onto corporate document repository which is safeguarded through Identity and Access control measures.
Target Audience: Each produced artefact (metric, report or dashboard) should have clarity of its target audience to correspondingly allow it to be concise and relevant in terms of its messaging. For example, a SOC Manager might be interested in understanding the overall operational effectiveness of Security Operations whereas C-Suite Executives are keen on understanding how are these SOC operations impacting their Enterprise Risk Profile.
Production Format: Defines how each artefact (metrics, report, dashboard) are intended to be distributed, e.g. through the corporate SharePoint, or email or in a printed format.
Classification Level: Defines the classification level of the information being shared, e.g. Public External, Confidential Internal, etc.