Content Management Role and Responsibilities

As defined in the KPMG GSOC Job Description, the Tooling Engineer will assume the Content Management role for managing the GSOC Content.

GSOC content will be defined as:

  • RSA Security Analytics Parsers
  • RSA Security Analytics Feeds
  • RSA Security Analytic Event Stream Analysis Rules
  • RSA Security Analytic Application Rules
  • RSA Security Analytic Reporting Engine Rules
  • RSA Archer Security Operations CEF Templates
  • RSA Archer Security Operation Incident Response Procedures and Tasks

This role has two responsibilities:

  • GSOC Content Monitoring and Tuning
  • GSOC Content Management Requests

Content Monitoring and Tuning

The Tooling Engineer Analyst will regularly monitor the effectiveness of rules and tune as needed. To do this the Tooling Engineer will look for the following:

  • Above average alert fire rates
  • Below average alert fire rates
  • Alerts that have not fired
  • Alerts with a substantial performance impact

The Tooling Engineer must determine why rules are deviated from the average. This can be because the content is based on stale intelligence, needs to be tuned because of high false positive rates or because there is a high frequency of that threat on KPMG’s environment.

To confirm stale content, the Tooling Engineer will work with the Threat Intelligence Team as well as refer to their Content Tracking System as described in the Categorization and Management Section.

On a bi-weekly basis the Tooling Engineer is to compute and report the metrics of the success rate of the content to the GSOC Manager and Threat Intelligence Team.

Metrics to ReportDescription
Number of rules created via source (bi-weekly)A breakdown of rules created by Tooling Engineer in a bi-weekly period separated by the source. The sources will be Threat Intelligence, Content Management or Other
Number of rules tuned via request or content monitoring  (bi-weekly)A breakdown of rules created by Tooling Engineer in a bi-weekly period separated by the source. The sources will be Threat Intelligence, Content Management or Other
Number of rules removed via request or content monitoring  (bi-weekly)A breakdown of rules created by Tooling Engineer in a bi-weekly period separated by the source. The sources will be Threat Intelligence, Content Management or Other
False Positive RateA overall percentage rate of the False Positive Rate
False Positive Reported by SourceA percentage of where False Positives are being reported by source. The sources will be Level 1, Level 2, Member Firms or Other
Most Effective RulesShows the top five most effective rules by calculating ‘Alert Rate vs False Positive’
Least Effective RulesShows the top five least effectives rules by calculating ‘Alert Rate vs False Positive’

Content Management Requests

The Tooling Engineer will receive requests from the GSOC and Member Firms for Security content management purposes. These requests will be to create, modify or remove content in KPMG’s GSOC security devices. A workflow and process for handling these requests is in GSOC Content Management Request Process.

Responsibility and Authority

The Tooling Engineer will have the authority to create, modify or remove the below content types for RSA Security Analytics, RSA Archer Security Operations or any other KPMG GSOC security devices:

  • RSA Security Analytics Parsers
  • RSA Security Analytics Feeds
  • RSA Security Analytic Event Stream Analysis Rules
  • RSA Security Analytic Application Rules
  • RSA Security Analytic Reporting Engine Rules
  • RSA Archer Security Operations CEF Templates
  • RSA Archer Security Operation Incident Response Procedures and Tasks

The GSOC will not have the authority to apply any patches, version updates or configurations for RSA Security Analytics, RSA Archer Security Operations or any other KPMG GSOC security devices and must adhere to the change management process

Content Management Workflow

The responsibilities of the GSOC Content Management role are displayed in the below workflow.

Content Management Request Workflow

As discussed in the GSOC Content Management Requests Section, one of the responsibilities for the Content Management role is to process GSCOS content requests from the GSOC and Member Firms.

This section shows the workflow the Tooling Engineer will follow when processing content requests from the GSOC and Member Firms.

There are four primary requests that are handled by the Tooling Engineer.

  • New Detection Method/Response Procedure: This request will come from the Level 2 analysts detailing a request for a new detection method and associated response procedures
  • Content Tuning: This request will come from the Threat Intelligence team and is a request to tune or remove current rules or configurations
  • New Indicators: This request will come from the Threat Intelligence team and will be specific to intelligence gained from incident analysis in which the goal is to mitigate and/or prevent from happening again. Actionable Threat Intelligence that would require new or updated signatures would be requested here
  • False Positive: This request will come from the Level 1 Analyst, Level 2 Analyst or the Member Firms and is used to validate false positives and see if alert signatures can be adjusted to reduce the false positive

Content Management Request Process

The below sections describe the steps the Tooling Engineer should take when processing content requests from the GSOC and Member Firms.  They are separated by each type of request the Tooling Engineer will receive.

The numbers (1.1, 2.1, 2.2 etc…) align to steps that are taken in the workflow in Figure 2: Tooling Engineer Team Request(s) Workflow

  1. Formal Request
    1. When the GSOC staff or Member Firms make a request to the Tooling Engineer, they must provide the Tooling Analyst with answers to the fields in, Table 1: Content Management Fields Request Fields . This request should be made as a formal request using formats such as email, HTML forms or a database and not in person or over the phone

Table 1: Content Management Fields Request Fields

Requester FieldsRequest TypeRequest Description
Content Request TypeAllCreate new, tune existing or remove existing
Objective/ReasoningAllThe objective will provide a brief description of the detection objectives of the rule and what stage of the Cyber Kill Chain that the rule detects on. The Tooling Engineer may have to talk to other members of the GSOC team or external parties to get this information  
Content CategoryContent Creation or TuningExample types are IP, MD5, Domain and User-Agents
Content IndicatorsContent Creation or TuningActionable intelligence that is to be used to create the GSOC Content
Intelligence Confidence RatingContent Creation or TuningThe Threat Intelligence Analyst will provide the Tooling Engineer Analyst with the Confidence Rating that they determined during their Triage. The Tooling Engineer will use this to determine what type of content to create as well as any associated time limits  
Expiration dateContent Creation or TuningIf known, an expiration date should be assigned here
Workflow (Only for New Detection Requests)New Detection MethodThe workflow will provide a procedure of how to respond and remediate the incident related to the rule
  • Response Procedure/New Detection Method

This type of request is used for a net new detection capability; use case request and checklist or response procedures.

  • Response Procedure/New Detection requested by the Level 2 Analyst
    • Preliminary Analysis is performed by the Tooling Engineer
      • Is the detection method feasible with current technology?
      • The Tooling Engineer will ensure Response Procedure is complete, in RSA Archer Security Operations format and has a VERIS Threat Category assigned
      • The Tooling Engineer will check the Content Tracking System to ensure that content will not collide with existing content. Look for identical ‘Content Indicators’ or content with similar ‘Objectives/Reasoning’
    • Skip to Step 6.1
  • Tuning

This request is used to modify or remove a current signature for increased accuracy or enhanced detection capability.

  • A content tuning request is made by the Threat Intelligence team
    • Preliminary Analysis is performed by the Tooling Engineer
      • Is additional information needed from Threat Intelligence team?
      • The Tooling Engineer will check the Content Tracking System to ensure that content will not collide with existing content. Look for identical ‘Content Indicators’ or content with similar ‘Objectives/Reasoning’
      • The Tooling Engineer would then begin outlining the content modifications and ensure that the KPMG GSOC has the capability for detection. Possible complications would include insufficient log sources being fed to RSA Security Analytics or requiring access to non GSOC Security devices
    • Proceed to Step 6.1

A new indicator request would stem from an incident investigation, forensics analysis or malware analysis where new indicators are discovered. Newly discovered indicators need to be placed into detection operations and historical searched initiated.

  • The Threat Intelligence Team will provide all incident data that led to the indicator, this could consist of an IP, Domain, network traffic, host artefact or malware artefacts
    • Preliminary Analysis is performed by the Tooling Engineer
      • The Tooling Engineer would review the incident data and extract the actionable intelligence from the request
      • The Tooling Engineer will check the Content Tracking System to ensure that content will not collide with existing content. Look for identical ‘Content Indicators’ or content with similar ‘Objectives/Reasoning’
      • The Tooling Engineer would then begin outlining the content modifications and ensure that the KPMG GSOC has the capability for detection. Possible complications would include insufficient log sources being fed to RSA Security Analytics or requiring access to non GSOC Security devices
    • Proceed to Step 6.1

This request consists of a Level 1 Analyst, Level 2 Analyst or Member Firm escalation for multiple reports of false positives being detected. An increase in false positives should initiate a content request to improve the alert so that only true alerts are generating incidents.

  • A false positive is reported to the Tooling Engineer
    • Preliminary analysis on related alerts and to determine the validity of the false positive claim
    • Proceed to Step 6.1
  •         
    • and then skip to Step 6.6

The type of content creation, modification or removal will include, but is not limited to the below types:

  • Creating, revising or removing RSA Security Analytics Parsers
  • Creating, revising or removing RSA Security Analytics Feeds
  • Creating, revising or removing RSA Security Analytic Event Stream Analysis Rules
  • Creating, revising or removing RSA Security Analytic Application Rules
  • Creating, revising or removing RSA Security Analytic Reporting Engine Rules
  • Creating, revising or removing RSA Archer Security Operations CEF Templates
  • Creating, revising or removing RSA Archer Security Operation Incident Response Procedures and Tasks

For content creation and tuning requests, the Tooling Engineer will work in accordance with the confidence rating provided by the requester. Table 2: Intelligence Confidence Rating details the requirements for content creation based off of the Intelligence Confidence Rating

Table 2: Intelligence Confidence Rating

Intelligence Confidence RatingContent CreationEstimated Implementation TimesConstraints
High– SA RE or ESA Alerts – Portal Update – Threat reports (if applicable) for situational awarenessImmediatelyComplex rules or rules that have the potentially can cause network disruption cannot be used due to the minimal testing and validation
Reasonable– SA hourly/daily reports – SA RE or ESA Alerts when necessary – Portal Update – Threat reports (if applicable) for situational awareness1 Business DayN/A
Low– No content created. – Threat reports (if applicable) for situational awareness2+ Business DaysN/A
  • Proper testing of the content should be done in the QA/Pre-Prod environments. Testing needs to satisfy the below questions: 
    • Is the false-positive ratio at or below the minimum? RSA Security Analytic reports should help to determine the answer
    • Was testing successful? If ‘No’ go back Step 6.2, if ‘Yes’ proceed to the next step.
    • The content will be deployed to the Production environment and the Content Tracing System will be updated.

Figure 2: Tooling Engineer Team Request(s) Workflow

Categorization and Management

The Tooling Engineer will manage all content that is created, modified or removed. For every piece of content created the below fields should be captured. This will be done in a Content Tracking System such as a database or spreadsheet.

Table 3: Content Management Tracking Fields

RequesterLevel 2 Analyst, Threat Intelligence Analyst or external parties
Content RequestCreate new, tune existing or remove existing
Date RequestedDate of initial request
Change LogFor content tuning or removal requests, provide a date and additional information about what was changed
Type of contentExample types are IP, MD5, Domain and User-Agents
ObjectiveThe objective will provide a brief description of the detection objectives of the rule and what stage of the Cyber Kill Chain that the rule detects on. The TE may have to talk to other members of the GSOC team or external parties to get this information
Intelligence Confidence RatingThe Threat Intelligence Analyst will provide the Tooling Engineer Analyst with the Confidence Rating that they determined during their Triage. The Tooling Engineer will use this to determine what type of content to create as well as any associated time limits
Expiration dateIf known, an expiration date should be assigned here
Content DeviceThe main options will be RSA SA and Archer SecOps
Content Rule NameThe naming of the rule should be as specific as possible so that an analyst can look at just the name and have a good understanding of what malicious activity was detected. Rule names should be in the format of DeviceName_RuleType_LogSource_VERISThreatAction_RuleDescription DeviceName: SA will be what is primarily used, however additional tools like QRADAR, Arcsight or IDS devices are also validRuleType: In SA, rule types will either be from the Reporting Engine (RE) or from the Event Stream Analytics (ESA)LogSource: Windows, AV, multiple etc..VERIS Threat Action: VERIS is an industry standard for classifying an incident.  A list of the valid VERIS Threat Actions to be used can be found here, http://veriscommunity.net/actions.htmlRule Description: A short one to three word description of the rule
Content Rule LogicDescription of the logic used for detection
WorkflowThe workflow will provide a procedure of how to respond and remediate the incident related to the rule