Application-based Alerts

Feature Summary and Revision History

Summary Data

Table 1. Summary Data

Applicable Product(s) or Functional Area

AMF

Applicable Platform(s)

SMI

Feature Default Setting

Disabled – Configuration required to enable

Related Documentation

Not Applicable

Revision History

Table 2. Revision History

Revision Details

Release

First introduced.

2021.04.0

Feature Description

When the system detects an anomaly, it generates an alert notification. The system statistics are the cause for these alert notifications. You can set an expression to trigger an alert when the expression becomes true.

How it Works

This section describes how this feature works.

The Common Execution Environment (CEE) uses the Prometheus Alert Manager for alerting operations. The CEE YANG model - either through CLI or API - allows users to view the active alerts and alert history. Also, the applications can call the alert API directly to add or clear alerts. The Prometheus Alert Manager API (v2) is the standard API used.

The Prometheus Alerts Manager includes the following options:

  • Defining Alert Rules: This option defines the types of alerts that the Alert Manager should trigger. Use the Prometheus Query Language (PromQL) to define the alerts.

  • Defining Alert Routing: This option defines the action the Alert Manager should take after receiving the alerts. At present, the SNMP Trapper is supported as the outbound alerting. Also, the CEE provides an Alert Logger for storing the generated alerts.

Configuring the Alert Rules

To configure the alert rules, use the following configuration:

config 
   alerts rules group alert_group_name 
   interval-seconds seconds 
   rule rule_name 
      expression promql_expression 
      duration duration 
      severity severity_level 
      type alert-type 
      annotation annotation_name 
      value annotation_value 
      end

NOTES:

  • alerts rules —Specify the Prometheus alerting rules.

  • group alert_group_name —Specify the Prometheus alerting rule group. One alert group can have multiple lists of rules. alert-group-name is the name of the alert group. The alert-group-name must be a string in the range of 0–64 characters.

  • interval-seconds seconds —Specify the evaluation interval of the rule group in seconds.

  • rule rule_name —Specify the alerting rule definition. rule_name is the name of the rule.

  • expression promql_expression —Specify the PromQL alerting rule expression. promql_expression is the alert rule query expressed in PromQL syntax.

  • duration duration —Specify the duration of a true condition before it’s considered true. duration is the time interval before the alert is triggered.

  • severity severity_level —Specify the severity of the alert. severity-level is the severity level of the alert. The severity levels are critical, major, minor, and warning.

  • type alert_type —Specify the type of the alert. alert_type is the user-defined alert type. For example, Communications Alarm, Environmental Alarm, Equipment Alarm, Indeterminate Integrity Violation Alarm, Operational Violation Alarm, Physical Violation Alarm, Processing Error Alarm, Quality of Service Alarm, Security Service Alarm, Mechanism Violation Alarm, or Time Domain Violation Alarm.

  • annotation annotation_name —Specify the annotation to attach to the alerts. annotation_name is the name of the annotation.

  • value annotation_value —Specify the annotation value. annotation_value is the value of the annotation.

Configuration Example

The following is an example configuration.

The following example configures an alert that is triggered when the percentage of registration procedure success is less than the specified threshold limit. 

config
   alerts rules group AMFProcStatus
   interval-seconds 300
   rule UeRegistration
      expression "sum(amf_procedure_total{proc_type='UE Registration',proc_status='ProcStatusComplete',status='success'}) / sum(amf_procedure_total{proc_type='UE Registration',status='attempted'}) < 0.95"
      severity major
      type Communications Alarm
      annotation annotation_name
      value summary
 value "This alert is fired when the UE registration procedure success is below specified threshold”
      end

Configuration Verification

To verify the configuration.

show running-config alerts rules group AMFProcStatus 
alerts rules group AMFProcStatus
 rule UeRegistration
  expression "sum(amf_procedure_total{proc_type='UE Registration',proc_status='ProcStatusComplete',status='success'}) / sum(amf_procedure_total{proc_type='UE Registration',status='attempted'}) < 0.95"
  severity   major
  type       "Communications Alarm"
  annotation summary
   value "This alert is fired when the UE registration procedure success is below specified threshold” "
  exit
 exit
exit

Viewing Alert Logger

By default, alert logger stores all the generated alerts. You can view the stored alerts using the following show command.

show alert history [ detail | summary ] [ filtering ]

You can narrow down the result using the following filtering options:

  • annotations—Displays the annotations of the alert.

  • endsAt—Displays the end time of the alert.

  • labels—Displays the additional labels of the alert.

  • severity—Displays the severity of the alert.

  • source—Displays the source of the alert.

  • startsAt—Displays the start time of the alert.

  • type—Displays the type of the alert.

Use the following show command to view the history of the alerts configured in the system:
show alerts history detail
alerts history detail UEReg 11576e6a86da
 severity    major
 type        "Communications Alarm"
 startsAt    2021-10-24T07:56:24.857Z
 endsAt      2021-10-24T08:31:24.857Z
 source      System
 summary     "fired when ue reg fails"
 labels      [ "alertname: UEReg" "cluster: amf-cndp-b19-4_cee-cisco" "monitor: prometheus" "replica: amf-cndp-b19-4_cee-cisco" "severity: major" ]
 annotations [ "summary: fired when ue reg fails" "type: Communications Alarm" ]

You can view the active alerts using show alerts active command. The alerts remain active as long as the evaluated expression is true. 

show alerts active detail
alerts active detail UeRegistration 92b6dcdd8726
 severity    major
 type        "Communications Alarm"
 startsAt    2021-10-24T14:56:42.732Z
 source      System
 summary     "This alert is fired when the UE registration procedure success is below specified threshold"
 labels      [ "alertname: UeRegistration" "cluster: amf-cndp-b19-4_cee-cisco" "monitor: prometheus" "replica: amf-cndp-b19-4_cee-cisco" "severity: major" ]
 annotations [ "summary: This alert is fired when the UE registration procedure success is below specified threshold" "type: Communications Alarm" ]

Call Flow Procedure Alerts

This section describes commands that are required to configure alerts related to various call flow procedures.

Paging Success

To configure alerts related to the Paging Success procedure, use the following configuration:

alerts rules group AMFProcStatus
 rule Paging
  expression "sum(amf_procedure_total{proc_type='Paging',proc_status='ProcStatusComplete',status='success'}) / sum(amf_procedure_total{proc_type='Paging',status='attempted'}) < 0.95"
  severity   major
  type       "Communications Alarm"
  annotation summary
   value "This alert is fired when the Paging procedure success is below specified threshold"
  exit
 exit
exit

Service Request Success

To configure alerts related to the Service Request Success procedure, use the following configuration:

alerts rules group AMFProcStatus
 rule ServiceRequest
  expression "sum(amf_procedure_total{proc_type='Service Request',proc_status='ProcStatusComplete',status='success'}) / sum(amf_procedure_total{proc_type='Service Request',status='attempted'}) < 0.95"
  severity   major
  type       "Communications Alarm"
  annotation summary
   value "This alert is fired when the Service request procedure success is below specified threshold"
  exit
 exit
exit

UE Deregistration Success

To configure alerts related to the UE Deregistration procedure, use the following configuration:

alerts rules group AMFProcStatus
 interval-seconds 300
 rule UeDeRegistration
  expression "sum(amf_procedure_total{proc_type='UE DeRegistration',proc_status='ProcStatusComplete',status='success'}) / sum(amf_procedure_total{proc_type='UE DeRegistration',status='attempted'}) < 0.95"
  severity   major
  type       "Communications Alarm"
  annotation summary
   value "This alert is fired when the UE deregistration procedure success is below specified threshold"
  exit
 exit
exit

UE Registration Success

To configure alerts related to the UE Registration procedure, use the following configuration:

alerts rules group AMFProcStatus
 interval-seconds 300
 rule UeRegistration
  expression "sum(amf_procedure_total{proc_type='UE Registration',proc_status='ProcStatusComplete',status='success'}) / sum(amf_procedure_total{proc_type='UE Registration',status='attempted'}) < 0.95"
  severity   major
  type       "Communications Alarm"
  annotation summary
   value "This alert is fired when the UE registration procedure success is below specified threshold”
  exit
 exit
exit

Message Level Alerts

This section describes commands that are required to configure alerts related to various message.

N1 Registration Accept

To configure alerts related to the N1 Registration Accept Request, use the following configuration:

alerts rules group AMFSvcStatus 
 interval-seconds 300
 rule AMFN1RegistrationSuccess 
  expression "sum(increase(amf_nas_message_total{message_type=~'N1RegistrationAccept_.*'}[5m])) / sum(increase(amf_nas_message_total{message_type=~'N1RegRequest_RegType_.*'}[5m])) < 0.95" 
  severity major
  type "Communications Alarm"
  annotation summary
  value "This alert is fired when the percentage of Registration Accept sent is lesser than threshold.”      
  exit 
exit

N1 Service Accept

To configure alerts related to the N1 Service Accept Request, use the following configuration:

alerts rules group AMFSvcStatus 
 interval-seconds 300
 rule AMFN1ServiceRequestSuccess  
  expression "sum(increase(amf_nas_message_total{message_type='N1ServiceAcc'}[5m])) / sum(increase(amf_nas_message_total{message_type='N1ServiceReq'}[5m])) < 0.95"  
  severity major
  type "Communications Alarm"
  annotation summary
  value "This alert is fired when the percentage of Service Accept sent is lesser than threshold.”       
  exit 
exit

N1 UE Initiated Deregistration

To configure alerts related to the N1 UE Initiated Deregistration Request, use the following configuration:

alerts rules group AMFSvcStatus 
 interval-seconds 300
 rule AMFN1UeInitDeregSuccess   
  expression "sum(increase(amf_nas_message_total{message_type='N1DeRegAccept_UeOriginatingDereg'}[5m])) / sum(increase(amf_nas_message_total{message_type='N1DeRegReq_UeOriginatingDereg'}[5m])) < 0.95"   
  severity major
  type "Communications Alarm"
  annotation summary
  value "This alert is fired when the percentage of Deregistration Accept sent is lesser than threshold.”        
  exit 
exit

N1 Network Initiated Deregistration

To configure alerts related to the N1 Network Initiated Deregistration Request, use the following configuration:

alerts rules group AMFSvcStatus 
 interval-seconds 300
 rule AMFN1NwInitDeregSuccess   
  expression "sum(increase(amf_nas_message_total{message_type='N1DeRegAccept_UeTerminatedDereg'}[5m])) / sum(increase(amf_nas_message_total{message_type='N1DeRegReq_UeTerminatedDereg'}[5m])) < 0.95"    
  severity major
  type "Communications Alarm"
  annotation summary
  value "This alert is fired when the percentage of Deregistration Accept received is lesser than threshold.”        
  exit 
exit

N2 ICSR Success

To configure alerts related to the N2 ICSR Success Request, use the following configuration:

alerts rules group AMFSvcStatus 
 interval-seconds 300
 rule AMFN2IcsrSuccess   
  expression "sum(increase(amf_ngap_message_total{message_type='N2InitialContextSetupRsp'}[5m])) / sum(increase(amf_ngap_message_total{message_type='N2InitialContextSetupReq'}[5m])) < 0.95"    
  severity major
  type "Communications Alarm"
  annotation summary
  value "This alert is fired when the percentage of Initial Context Setup Response is lesser than threshold.”         
  exit 
exit

N2 PDU Setup Success

To configure alerts related to the N2 PDU Setup Success Request, use the following configuration:

alerts rules group AMFSvcStatus 
 interval-seconds 300
 rule AMFN2PduSetupRequestSuccess    
  expression "sum(increase(amf_ngap_message_total{message_type='N2PduSessResourceSetupRsp'}[5m])) / sum(increase(amf_ngap_message_total{message_type='N2PduSessResouceSetupReq'}[5m])) < 0.95"    
  severity major
  type "Communications Alarm"
  annotation summary
  value "This alert is fired when the percentage of Ngap PDU Setup Response is lesser than threshold.”        
  exit 
exit

N2 PDU Modify Success

To configure alerts related to the N2 PDU Modify Success Request, use the following configuration:

alerts rules group AMFSvcStatus 
 interval-seconds 300
 rule AMFN2PduModifySuccess   
  expression "sum(increase(amf_ngap_message_total{message_type='N2PduSessResourceModifyRsp'}[5m])) / sum(increase(amf_ngap_message_total{message_type='N2PduSessResouceModifyReq'}[5m])) < 0.95"     
  severity major
  type "Communications Alarm"
  annotation summary
  value "This alert is fired when the percentage of Ngap PDU Modify Response is lesser than threshold.”         
  exit 
exit

N2 PDU Release Success

To configure alerts related to the N2 PDU Release Success Request, use the following configuration:

alerts rules group AMFSvcStatus 
 interval-seconds 300
 rule AMFN2PduReleaseSuccess   
  expression "sum(increase(amf_ngap_message_total{message_type='N2PduSessResourceReleaseRsp'}[5m])) / sum(increase(amf_ngap_message_total{message_type='N2PduSessResouceReleaseReq'}[5m])) < 0.95"     
  severity major
  type "Communications Alarm"
  annotation summary
  value "This alert is fired when the percentage of Ngap PDU Release Response is lesser than threshold.”          
  exit 
exit

N8 UECM Registration Request

To configure alerts related to the N8 UECM Registration Request, use the following configuration:

alerts rules group AMFSvcStatus 
 interval-seconds 300
 rule AMFN8UecmRegSuccess 
  expression "sum(increase(n8_service_stats{message_type='NudmUecmRegistrationRsp', status='success'}[5m])) / sum(increase(n8_service_stats{message_type='NudmUecmRegistrationReq', status='success'}[5m])) < 0.95" 
  severity major
  type "Communications Alarm"
  annotation summary
  value "This alert is fired when the percentage of UECM registration  responses received is lesser than threshold.” 
  exit 
exit

N8 UECM Deregistration Request

To configure alerts related to the N8 UECM Deregistration Request, use the following configuration:

alerts rules group AMFSvcStatus 
 interval-seconds 300
 rule AMFN8UecmDeRegSuccess 
  expression "sum(increase(n8_service_stats{message_type='NudmUecmDeRegistrationRsp', status='success'}[5m])) / sum(increase(n8_service_stats{message_type='NudmUecmDeRegistrationReq', status='success'}[5m])) < 0.95"
  severity major
  type "Communications Alarm"
  annotation summary
  value "This alert is fired when the percentage of UECM deregistration  responses received is lesser than threshold.” 
  exit 
exit

N8 SDM Data Request

To configure alerts related to the N8 SDM Data Request, use the following configuration:

alerts rules group AMFSvcStatus 
 interval-seconds 300
 rule AMFN8SdmDataReqSuccess 
  expression "sum(increase(n8_service_stats{message_type='NudmSdmDataRsp', status='success'}[5m])) / sum(increase(n8_service_stats{message_type='NudmSdmDataReq', status='success'}[5m])) < 0.95" 
  severity major
  type "Communications Alarm"
  annotation summary
  value "This alert is fired when the percentage of SDM Data responses received is lesser than threshold.” 
  exit 
exit

N8 SDM Subscription Request

To configure alerts related to the N8 SDM Subscription Request, use the following configuration:

alerts rules group AMFSvcStatus 
 interval-seconds 300
 rule AMFN8SdmSubscriptionSuccess 
  expression "sum(increase(n8_service_stats{message_type='NudmSdmSubscriptionRsp', status='success'}[5m])) / sum(increase(n8_service_stats{message_type='NudmSdmSubscriptionReq', status='success'}[5m])) < 0.95" 
  severity major
  type "Communications Alarm"
  annotation summary
  value "This alert is fired when the percentage of SDM Subscription responses received is lesser than threshold.” 
  exit 
exit

N8 SDM Unsubscribe Request

To configure alerts related to the N8 SDM Unsubscribe Request, use the following configuration:

alerts rules group AMFSvcStatus 
 interval-seconds 300
 rule AMFN8SdmUnSubscriptionSuccess 
  expression "sum(increase(n8_service_stats{message_type='NudmSdmUnSubscriptionRsp', status='success'}[5m])) / sum(increase(n8_service_stats{message_type='NudmSdmUnSubscriptionReq', status='success'}[5m])) < 0.95" 
  severity major
  type "Communications Alarm"
  annotation summary
  value "This alert is fired when the percentage of SDM UnSubscription responses received is lesser than threshold.” 
  exit 
exit

N8 PCSCF Restoration Request

To configure alerts related to the N8 PCSCF Restoration Request, use the following configuration:

alerts rules group AMFSvcStatus 
 interval-seconds 300
 rule AMFN8PcscfRestorationSuccess 
  expression "sum(increase(n8_service_stats{message_type='NudmPcscfRestorationRsp', status='success'}[5m])) / sum(increase(n8_service_stats{message_type='NudmPcscfRestorationReq', status='attempted'}[5m])) < 0.95" 
  severity major
  type "Communications Alarm"
  annotation summary
  value "This alert is fired when the percentage of Pcscf Restoration responses sent is lesser than threshold.” 
  exit 
exit

N11 SM Create

To configure alerts related to the N11 SM Create Request, use the following configuration:

alerts rules group AMFSvcStatus 
 interval-seconds 300
 rule AMFN11SMCreateSuccess 
  expression "sum(increase(rpc_response_total{msg_type='PostSmCtxtsRequestPB',rpc_name='SMF',status_code='201'}[5m]))/
sum(increase(rpc_response_total{msg_type='PostSmCtxtsRequestPB',rpc_name='SMF'}[5m])) < 0.95"   
  severity major
  type "Communications Alarm"
  annotation summary
  value "This alert is fired when the percentage of Update SM context responses received is lesser than threshold.”    
  exit 
exit

N11 SM Release

To configure alerts related to the N11 SM Release Request, use the following configuration:

alerts rules group AMFSvcStatus 
 interval-seconds 300
 rule AMFN11SMReleaseSuccess 
  expression "sum(increase(rpc_response_total{msg_type='PostSmCtxtsReleaseRequest',rpc_name='SMF',status_code='204'}[5m])) /
sum(increase(rpc_response_total{msg_type='PostSmCtxtsReleaseRequest',rpc_name='SMF'}[5m]))  <  0.95"
  severity major
  type "Communications Alarm"
  annotation summary
  value "This alert is fired when the percentage of Release SM context responses received is lesser than threshold.”     
  exit 
exit

N11 SM Update

To configure alerts related to the N11 SM Update Request, use the following configuration:

alerts rules group AMFSvcStatus 
 interval-seconds 300
 rule AMFN11SMUpdateSuccess 
  expression "sum(increase(rpc_response_total{msg_type='PostSmCtxtsModifyRequestPB',rpc_name='SMF',status_code=~'200|204'}[5m])) / sum(increase(rpc_response_total{msg_type='PostSmCtxtsModifyRequestPB',rpc_name='SMF'}[5m])) <  0.95"
  severity major
  type "Communications Alarm"
  annotation summary
  value This alert is fired when the percentage of Update SM context responses received is lesser than threshold.”      
  exit 
exit

N12 UeAuth Req

To configure alerts related to the N12 UeAuth Request, use the following configuration:

alerts rules group AMFSvcStatus 
 interval-seconds 300
 rule AMFN12UeAuthReqSuccess 
  expression "sum(increase(n12_service_stats{message_type='NausfUeAuthRsp', status='success'}[5m])) / sum(increase(n12_service_stats{message_type='NausfUeAuthReq', status='success'}[5m])) < 0.95"   
  severity major
  type "Communications Alarm"
  annotation summary
  value "This alert is fired when the percentage of Ausf UE Auth responses received is lesser than threshold.”   
  exit 
exit

N15 AM Policy Control Create

To configure alerts related to the N15 AM Policy Control Create Request, use the following configuration:

alerts rules group AMFSvcStatus 
 interval-seconds 300
 rule AMFN15PolicyControlCreateSuccess 
  expression "sum(increase(n15_service_stats{message_type='NpcfAmPolicyControlCreateRsp', status='success'}[5m])) / sum(increase(n15_service_stats{message_type='NpcfAmPolicyControlCreateReq', status='success'}[5m])) < 0.95"  
  severity major
  type "Communications Alarm"
  annotation summary
  value "This alert is fired when the percentage of Policy control create  responses received is lesser than threshold.”  
  exit 
exit

N15 AM Policy Control Delete

To configure alerts related to the N15 AM Policy Control Delete Request, use the following configuration:

alerts rules group AMFSvcStatus 
 interval-seconds 300
 rule AMFN15PolicyControlDeleteSuccess 
  expression "sum(increase(n15_service_stats{message_type='NpcfAmPolicyControlDeleteRsp', status='success'}[5m])) / sum(increase(n15_service_stats{message_type='NpcfAmPolicyControlDeleteReq', status='success'}[5m])) < 0.95"  
  severity major
  type "Communications Alarm"
  annotation summary
  value "This alert is fired when the percentage of  Policy control delete responses received is lesser than threshold.”  
  exit 
exit