Detection optimization is focused on enhancing the detection of security incidents. This is to include the current rule set as well as detection gaps. The detection gaps can be the result of bad intelligence, missing indicators or misconfigured content. The goals of the Detection Optimization Process are to:

  1. Discover and capture gaps or improvements in detection processes
  2. Review the gaps/improvements
  3. Determine the remediation of the identified gaps
  4. Approval and implement the remediation plan

Identify Gaps

The discovery of gaps will rely on feedback models from each GSOC team. The below sections describes what data is provided from each GSOC team.

Lessons Learned

Level 1 and Level 2 analysts will provide feedback in the form of Lessons Learned that they will complete when an incident has been resolved and as indicated in the Incident Management Process. Examining the output from these questions will help identify broken processes, gaps in incident context and detection changes or new methods.

Content Management Metrics

As outlined in the Content Management Process, the bi-weekly report from the Tooling Engineer will be sufficient information to determine the effectiveness of the content deployed. This report will include metrics such as the most effective and least effective rules and false positive. Review of this report will help to show from a content perspective the type of alerts that are successful and the types that are prone to false positives.

Threat Intelligence Metrics

As part of the Threat Intelligence Process, The Threat Intelligence team will report on metrics that rate the effectiveness of the intelligence sources and the effectiveness of the disseminated intelligence.  Review of these metrics will provide insight into specific intelligence sources that need to be optimized.

The Threat Intelligence team will also be able to report on strategic and/or Operational intelligence that will help the Committee to identify detection gaps.

Operational Metrics

These metrics will provide an overall view of how the Level 1 Analyst and Level 2 Analysts are handling incidents. These metrics will be used to identify detection possible detection gaps and to gauge the complexity of the events being detected.

The committee will look for a rise in trends of metrics such as threat vectors and cyber kill chain, this will allow the committee to assess if they have adequate detection content in place for detection.

High analyst workload can be due to a lack of context during a incident, however it can also be due to highly complex content in which the analyst has to use more time to triage, it is here that the committee will focus on.

The metrics to focus on for Detection Optimization are below.

MetricDescription
Incidents by TargetWhat are the most common attacked targets? Is detection sufficient for these targets?
Threat Category/VectorWhat are the most common attacked vectors and methods? Is detection sufficient?
Analyst WorkloadAre analysts working large workloads due to high false positives or missing incident context?
Incidents by Cyber Kill Chain (Requires added field in SecOps)What stage of the Cyber Kill change is being detected the most? The least? Is detection sufficient?

Subsidiary Feedback

Feedback from the subsidiary organisation should be incorporated into the discovery process. At the moment the feedback from subsidiary will be informal and should be captured in the Lesson Learned or through emails sent to the SOC Management.

The SOC Management and Staff should forward any feedback received from the subsidiary organisations to the Level 3 Analyst.

Subsidiary organisations feedback can also come in the form of how fast they are able to remediate the incident passed over to them by the SOC. The assumption being that what was provided to the subsidiary organisations was accurate and actionable.

Review Gaps

In order to review the data provided by each SOC group and the subsidiary organisations, a Optimization committee should be formed that is lead by the Level 3 analyst and has 1 member from each SOC group.

On a monthly basis, the committee will review all the data from the various metric reports and lessons learned and identify any detection gaps.

Generally speaking the committee will be looking for:

  • What are the total False Positive Rates?
  • How effective are the intelligence sources? What is the most effective source? What is the least?
  • What is the threat landscape that the threat actors are currently attacking relative to your organisation?
  • Does the Level 1 or Level 2 analyst get enough information from incident detection to determine a real threat or not
  • Are the subsidiary organisations satisfied with the timeliness and context they receive as a result of the detection?

Remediation and Implementation

After the review and identification of gaps, the committee will work together to put a remediation plan together in order to address the gaps. The GSOC Manager must approve the remediation plan before the committee can continue. If the GSOC Manager does not approve the plan, the committee must revise as indicated and then resubmit to the GSOC Manager.

When the committee receives approval from the GSOC Manager they can start to assign implementation tasks to the individuals. Depending on the type of tasks, additional resources may be needed.

Depending on the type of implantation tasks, different processes may be relied on to finish the tasks. For instance, if new content is being created, then the Content Management Request Process will be initiated. If the implementation tasks meet the criteria for a Change Request, then the L3 must lead this part to get the change approved.

When the tasks are completed they are reviewed and validated by the Level 3 Analyst. If the Level 3 Analyst does not think the tasks have been completely implemented he will assign additional tasks to address the incompletions.

 Once the Level 3 Analyst validates completion the changes are documented.