Key Performance Metrics

Wiki > Key Performance Metrics
KPIMetric CategoryKPI Definition/Reasons for measurement.BoardExecutive ManagementMiddle ManagementOperational Management
Source of Incidents CreatedSecurity IncidentsDefines the sources of incident detection or reporting e.g Phone, DNS, Proxy and DHCP
Time Duration between Event Detected and Incident Record CreatedSecurity IncidentsTime between when the event was detected in SIEM and when the Incident record was created
Time Duration between Incident Acknowledgement and Incident ContainmentSecurity IncidentsThe time between Incident Acknowledgement and its containment.
Time duration between event detected and incident containedSecurity IncidentsThis is the overall circle from Event detection to Incident Containment. Incident Resolution Time. Average time for resolving an incident grouped into categories 
Incident False Positive %Security IncidentsDisplays the percentage of incidents recorded that were identified as False Positives later on
Incident % handled purely at L1Security IncidentsThe percentage of incidents handled at L1. It could be said to be First Time Resolution Rate. Costs is low if this is sorted at L1 level. Target for this metric is usually >= 80%. 
Incident % Escalated from L1 to L2Security IncidentsThe percentage of incidents escalated from L1 to L2 within the SOC distributed by week.
Incident % Escalated from L2 to L3Security IncidentsThe percentage of incidents escalated from L2 to L3 within the SOC distributed by week.
Incidents Created & ClosedSecurity IncidentsCounts of Incidents Created & Closed over time (typically per week)
Incident Status CountSecurity IncidentsTotal Incident Count by Status. It is the Incident Count based on following Status: New, Assigned, In Progress, Escalated, Returned to Level 1, Remediation Requested, Remediation in Progress, Remediation in Completed, Resolved.
Incident Count by SubsidiarySecurity IncidentsDistribution of Incidents laid out by subsidiary on weekly basis.
No. of Total Investigation by SubsidiarySecurity IncidentsNo. of Total Investigation by subsidiary
No. of Total Breaches by SubsidiarySecurity IncidentsNo. of Total Breaches by subsidiary
Number of Breaches by StatusSecurity IncidentsNumber of Breaches by Status
Number of Breaches by TimeSecurity IncidentsNumber of Breaches by Time
Number of Incidents by TimeSecurity IncidentsNumber of Incidents by Time
Number of Investigations by TimeSecurity IncidentsNumber of Investigations by Time
Incident count by source operating systemSecurity IncidentsIncident count by source operating system
Number of Incidents by PrioritySecurity IncidentsNumber of Incidents by Priority: P1, P2, P3 and P4. Gives a snapshot of where we are at per time
Number of Investigations by PrioritySecurity IncidentsNumber of Incident Investigations by Priority: P1, P2, P3 and P4. Gives us a count of Incidents found versus Incidents being investigated
Incidents by ConfirmationSecurity IncidentsIncidents by Confirmation: Confirmed, Suspected and False Positive
Number of Incidents by StatusSecurity IncidentsNumber of Incidents by Status
Number of Investigations by StatusSecurity IncidentsNumber of Investigations by Status
New Incidents by WeekWeekly Security TrendsNew Incidents by Week. Gives a count of New Incidents by week for comparison on how many we get over time
Incident Trend by Status – WeeklyWeekly Security TrendsWeekly Incident Trend by Status: New, Assigned, In Progress, Escalated, Returned to Level 1, Remediation Requested, Remediation in Progress, Remediation in Completed, Resolved. Gives a summary of Incidents by Status for weekly comparisons
Incident Trend by Priority – WeeklyWeekly Security TrendsWeekly Incident Trend by Priority: P1, P2, P3 and P4
Weekly Average time to escalateWeekly Security TrendsWeekly Average time to escalate incidents
Weekly Average time to closeWeekly Security TrendsWeekly Average time to close incidents
Weekly trend of false positivesWeekly Security TrendsWeekly trend of false positives
Lessons learned
• most valuable mistake
• conversion rate
Weekly Security TrendsLessons learned
• most valuable mistake
• conversion rate
Intelligence types
• value by volume
• value by severity
Weekly Security TrendsIntelligence types
• value by volume
• value by severity
Count of Use Case usage (how often each use case is executed)Accuracy of RulesCount of Use Case usage (how often each use case is executed)
Use Case killsAccuracy of RulesUse Case kills
Use Case false positivesAccuracy of RulesUse Case false positives
Control EfficacyAccuracy of RulesThe efficacy of the detection based on the invocation of the Content rules. Display top 10 content rules triggered for the Subsidiary – few samples below:
■ Resulting Alert Classification
■ Resulting Alert Category
■ Assets targeted
Incidents distribution by Threat Category
• Social: Phishing
• Malware: Unknown
• Malware: C2
• Hacking: Footprinting
• Hacking: Brute force
• Hacking: Abuse of Functionality
Threat MetricsInsights into the subsidiary incident profile for the last week/month distributed by the assigned Incident Categories (Malware, C2, etc.) including incident prioritization (P1, P2, etc.)
Incidents by Attack VectorThreat MetricsIncidents by Attack Vector
Incidents by Target typesThreat MetricsIncidents by Target types
Attacker objectiveThreat MetricsAttacker objective
Tactics, Tech. & ProceduresThreat MetricsTactics, Tech. & Procedures
Incidents by BusinessBusiness & Breach MetricsIncidents by Business
Incidents by FacilityBusiness & Breach MetricsIncidents by Facility
Incidents by Asset CriticalityBusiness & Breach MetricsIncidents by Asset Criticality
Incidents by Business ImpactBusiness & Breach MetricsIncidents by Business Impact
Breach by Business ImpactBusiness & Breach MetricsBreach by Business Impact
Breach Notification StatusBusiness & Breach MetricsBreach Notification Status
Breach Task StatusBusiness & Breach MetricsBreach Task Status
Types of Data BreachedBusiness & Breach MetricsTypes of Data Breached
Actors: OriginVERIS Based Defines the geographic distribution of the attack/incident e.g. China, Russia etc
Actors: Motive (is this measurable?)VERIS Based Defines the motive behind the attack/Incident e.g. Fun, Fear, Grudge etc.
Actions: VectorVERIS Based Defines the vector of the attack/incident as per the VERIS classification e.g. Email, IM, Direct Install, Remote Injection etc.
Actions: MalwareVERIS Based Defines the distribution of the variety of malware used as per the VERIS classification of the incident e.g. RAT, Dropper, DoS, adware etc.
Actions: Hacking VERIS Based Defines the distribution of the type of hacking attack used as per the VERIS classification e.g. LDAP, XSS, Cache etc.
Asset: ManagementVERIS Based Defines the distribution of the asset management (asset managed by which part of the business or a third party) as per the VERIS classification of the incident.
Attributes (Variety)VERIS Based Defines the distribution of the incidents by the platform per the VERIS classification e.g. Iphone, Win7, Win2008 etc.
Kill ChainVERIS Based Attack penetration by Cyber Kill Chain (delivery, exploit, install, C2, Exfil). A kill chain is a systematic process to target and engage an adversary to create desired effects. It Basically tells us at what stage we capture an attack. The stages are: Recon>Weaponization>Delivery>Exploitation>Installation>Command & Control (C2)>Actions on Objectives
Incident Queue by Analyst IDAnalyst WorkloadInsight into the workload of Analysts to help SOC capacity planning. E.g. Incidents handled by all L1/L2 Analysts
Incident Queue by All AnalystsAnalyst WorkloadIncidents handled by All Analysts based on Incident Status: New, Assigned, In Progress, Escalated, Returned to Level 1, Remediation Requested, Remediation in Progress, Remediation in Completed, Resolved.
Analyst incident Count sent backAnalyst WorkloadAnalyst incident Count sent back
Overhead costs per analyst and incidentAnalyst WorkloadOverhead costs per analyst and incident
Average Time to AcknowledgeTriageDepicts the average time to acknowledge a task in one hour intervals over the previous 24 hour reporting period with incident priority categorization.
Average Time to CloseTriageDepicts the average time to close a task in one hour intervals over the previous 24 hour reporting period with incident priority categorization.
Closure RateTriageDepicts the task closure rate in one hour intervals over the previous 24 hour period with incident priority categorization.
Incident RateTriageDepicts the incident rate in one hour intervals over the previous 24 hour reporting period with incident priority categorization.
Last Modified TasksTriageDepicts the most recently modified task entries with incident priority categorization.
Longest Open TasksTriageDepicts the tasks that have been open for the longest amount of time with incident priority categorization.
Longest Unacknowledged TasksTriageDepicts the tasks that have been unacknowledged for the longest amount of time with incident priority categorization.
Open Tasks by OwnerTriageDepicts the number of open tasks for each unique owner contained in the Task Triage database with incident priority categorization.
Open Tasks by PriorityTriageDepicts the percentage of open tasks by priority level with incident priority categorization.
Tasks by Priority and OwnerTriageDepicts the number of open items by priority for a specified user with incident priority categorization.
Top 10 Source Addresses of AlarmsIncidents (Correlated Sources)Displays the top 10 source addresses of intrusion detection alarms.
Top 10 AlarmsIncidents (Correlated Sources)Displays the top 10 alarms by signature ID that have been generated.
Top 10 Destinations of AlarmsIncidents (Correlated Sources)Displays the top 10 destination IP addresses that have been targeted for attack.
Top 10 Requested URL/FTP DestinationsIncidents (Correlated Sources)Displays the top 10 URL or FTP destinations requested by internal users.
Top 20 Bandwidth PortsIncidents (Correlated Sources)Displays the 20 ports with the most bandwidth usage.
Top 20 Bandwidth UsersIncidents (Correlated Sources)Displays the top 20 bandwidth users.
Top 20 Connections by AddressIncidents (Correlated Sources)Displays the top 20 users of connections.
Top 20 Connections by PortIncidents (Correlated Sources)Displays the 20 ports with the most connections.
Top 20 Denied Inbound by AddressIncidents (Correlated Sources)Displays the top 20 foreign addresses that were denied inbound access.
Top 20 Denied Inbound by PortIncidents (Correlated Sources)Displays the 20 ports with the most denied connections.
Top 20 Denied Outbound by AddressIncidents (Correlated Sources)Displays the top 20 local addresses that were denied outbound access.
Return-on-InvestmentFuture MetricsThis will be valid when the operational metrics are tied back to the Risk Profile and Business Impact Analysis provided by RSA Archer SecOps. It can help quantify how much potential cost saving has been made for both time and monetary terms due to mitigated incidents and/or timely response.
Compliance PostureFuture MetricsRSA Archer SecOps can be capitalized to measure and report against the mandatory compliance requirements, however this will require further analysis and a wider stakeholder discussions to measure its viability and implementation.
Vulnerability ProfileFuture MetricsVulnerability profile can be reported against as the corresponding details are gradually shared by the subsidiary. Common reporting areas are:
Vulnerability Profile: Least Recently ScannedFuture MetricsLists assets in order of the longest duration since last scan.
Vulnerability Profile: Most Vulnerable Assets by Business RatingFuture MetricsLists the assets in order of business rating and the aggregate vulnerability severity score.
Vulnerability Profile: Most Vulnerable Assets by CountFuture MetricsLists the assets in order of the number of vulnerabilities associated with an asset.
Vulnerability Profile: Most Vulnerable Assets by SeverityFuture MetricsLists the assets in order of the aggregate vulnerability severity score.
Vulnerability Profile: Vulnerability by SeveritiesFuture MetricsDepicts the detected vulnerabilities as a percentage of the total organized by severity value.
Number of implemented Preventive Measures/RulesITIL KPIs Information Security ManagementNumber of preventive security measures which were implemented in response to identified security threats
Implementation DurationITIL KPIs Information Security ManagementDuration from the identification of a security threat to the implementation of a suitable counter measure
Number of major Security IncidentsITIL KPIs Information Security ManagementNumber of identified security incidents, classified by severity category
Number of Security-related Service DowntimesITIL KPIs Information Security ManagementNumber of security incidents causing service interruption or reduced availability
Number of Security TestsITIL KPIs Information Security ManagementNumber of security tests and trainings carried out
Number of identified Shortcomings during Security TestsITIL KPIs Information Security ManagementNumber of identified shortcomings in security mechanisms which were identified during tests
Total Number of RFCs raisedITIL KPIs Change ManagementNumber of major changes assessed by the CAB (Change Advisory Board)
Percentage for each Change CategoryITIL KPIs Change ManagementShows Routine, Standard, Expedited and Emergency change percentages
Number of successful changesITIL KPIs Change ManagementNumber of successful changes
Number of changes backed outITIL KPIs Change ManagementNumber of changes backed out
Number of failed changesITIL KPIs Change ManagementNumber of failed changes
Number of CAB MeetingsITIL KPIs Change ManagementNumber of CAB (Change Advisory Board) meetings
Time for Change Approval/ RejectionITIL KPIs Change ManagementAverage time from registering an RFC with Change Management until a decision on the RFC is reached (i.e. until it is either approved or rejected)
Change Acceptance RateITIL KPIs Change ManagementNumber of accepted vs. rejected RFCs
Number of Emergency ChangesITIL KPIs Change ManagementNumber of Emergency Changes assessed by the ECAB (Emergency Change Advisory Board)
Number of ProblemsITIL KPIs Problem ManagementNumber of Problems registered by Problem Management grouped into categories
Problem Resolution TimeITIL KPIs Problem ManagementAverage time for resolving Problems grouped into categories
Number of unresolved ProblemITIL KPIs Problem ManagementNumber of Problems where the underlying root cause is not known at a particular time
Number of Incidents per Known ProblemITIL KPIs Problem ManagementNumber of reported Incidents linked to the same Problem after problem identification
Time until Problem IdentificationITIL KPIs Problem ManagementAverage time between first occurance of an Incident and identification of the underlying root cause
Problem Resolution EffortITIL KPIs Problem ManagementAverage work effort for resolving Problems grouped into categories