Scheduled policies monitor diagnostic data that is generated by runtime MBeans according to a specific schedule. These policies can also be used to execute an action at a specific time or on a schedule.
Log policies monitor messages generated into the server or domain logs.
Instrumentation policies, also known as Event Data policies, monitor events generated by the WLDF Instrumentation component.
This chapter explains how to configure each policy type and includes the following sections:
For information about how to create a policy using the WebLogic Server Administration Console, see Create policies for a diagnostic system module in Oracle WebLogic Server Administration Console Online Help.
You can use any of the following tools to configure policies for diagnostic system modules:
WebLogic Server Administration Console
Fusion Middleware Control
WLST
REST
JMX application
This chapter refers primarily to using the WebLogic Server Administration Console or WLST.
The following table summarizes the attributes, elements, and options that you configure when creating a policy, and also identifies any requirements each configuration item has for specific policy types.
Item | Description | Policy Requirement |
---|---|---|
Attribute that determines the policy's type. The default is |
Must be specified for log and instrumentation policies. Optional for scheduled policies. |
|
Attribute that establishes the language used in the policy expression. The two supported languages are Java Expression Language (EL), and WLDF query language (deprecated). |
Use EL in all policy types. The WLDF query language is supported, but deprecated. |
|
Expression that identifies a situation or condition that you want to trap for monitoring or diagnostic purposes. The expression can analyze log records, data events, or MBean metrics, depending on the rule type setting. |
Optional for scheduled policies, but required for all others. If a scheduled policy does not include an expression, the policy always executes the associated actions according to the Policy Schedule. |
|
One or more operations that are executed when a policy expression is evaluated to |
Optional. |
|
A calendar-based schedule that determines when a scheduled policy is evaluated. |
Required for all scheduled policies. Not available for log or instrumentation policies. |
|
Options that determine whether, or when, a policy can be evaluated again after it has been evaluated to The default is |
Optional for all policy types. |
|
Log message severity value that, when the policy is evaluated to
The default is |
Optional for all policy types. |
|
Flags that either enable or disable a policy from being evaluated. The default is |
Optional for all policy types. |
When creating a policy, you must define its type in its rule type attribute. Policies with different rule types differ in two ways:
The syntax for specifying the conditions being monitored are unique to the rule type.
Log and instrumentation policies are triggered in real time, whereas scheduled policies are triggered by settings on the WLDFScheduleBean
interface, described in Policy Schedule.
The way to define the rule type depends on the tool you use to create the policy:
If you are using the WebLogic Server Administration Console or Fusion Middleware Control, the rule type is determined by the policy type you are creating. For each of the policy types you can choose in either console, the following table identifies the corresponding rule type and WLDFWatchBean.RuleType
attribute value that is defined for that policy:
Policy Type . . . | Rule Type | WLDFWatchBean.RuleType Value |
---|---|---|
Smart Rule |
Harvester |
Harvester |
Calendar Based |
Harvester |
Harvester |
Collected Metrics |
Harvester |
Harvester |
Server Log |
Log |
Log |
Domain Log |
Log |
DomainLog |
Event Data |
Instrumentation |
EventData |
For information about choosing a policy type using the WebLogic Server Administration Console, see Create policies for a diagnostic system module in Oracle WebLogic Server Administration Console Online Help.
If you are using WLST, REST, or JMX to configure a policy, you set the WLDFWatchBean.RuleType
attribute as follows:
Policy Type | Rule Type Attribute |
---|---|
Scheduled policy |
Harvester |
Log policy |
|
Instrumentation |
|
Policy expressions can be created using either of the following languages:
Java Expression Language (EL) (recommended)
WLDF query language (deprecated in WebLogic Server 12.2.1)
Note:
The policies described in this chapter are Java EL based. For information about configuring policies that use the WLDF query language, see WLDF Query Language-Based Policies.
If you have diagnostic system modules created with a previous release of WebLogic Server, WLDF supports expressions that use the WLDF query language. If you are creating new policies for either an existing or a new diagnostic system module, Oracle strongly recommends using Java EL as the policy expression language.
For more information about Java (EL), see the JSR-000341 Expression Language 3.0 specification at https://jcp.org/aboutJava/communityprocess/final/jsr341/index.html
.
A policy expression encapsulates all information necessary for specifying a rule that, when evaluated to true
, causes the associated actions to be executed. When you use Java EL as the expression language, you can construct a policy expression that uses the following out-of-the-box resources to set the conditions that determine whether to fire an associated action:
Beans
A bean is a Java object that can represent nearly anything in the domain, such as metrics from MBeans, or structured data surfaced by other beans.
Functions
Functions are a set of operations that are provided either by EL itself, or by WLDF, that can be invoked on metric data.
Smart rules
Smart Rules are special set of functions that encapsulate more complex logic and monitoring capabilities, and have specialized support in both the WebLogic Server Administration Console and Fusion Middleware Control. They can be used by themselves, or with other expression components as part of a larger, more complex expression.
Each policy can be associated with one or more actions that are executed whenever the policy evaluates to true
. For information about configuring actions, see Configuring Actions.
All scheduled policies must be configured with a schedule. Scheduling allows policies to be evaluated according to a calendar schedule, at a specific time, after a duration of time, or at timed intervals.
You configure a policy schedule by setting attributes on the WLDFScheduleBean
interface, which is a property of the WLDFWatchBean
. You can set these attributes using the WebLogic Server Administration Console, WLST, REST, or a JMX application. When you are configuring new policies, the WebLogic Server Administration Console and Fusion Middleware Control provide convenient assistants and workflows for configuring common scheduling scenarios.
Note:
The WLDFScheduleBean
is used for policy evaluation only when:
The configured policy rule type is "Harvester"
.
The configured expression language for the policy is "EL"
.
Note also that although scheduled policies that use the WLDFScheduleBean
for scheduling are configured as Harvester types, the WLDF Harvester component is not used for scheduling.
Table 10-1 lists the attributes of the WLDFScheduleBean
and their default values, which are the same as for the javax.ejb.ScheduleExpression
interface. In addition, the syntax for specifying a value, range, list, or interval for a specific unit of time is also the same as that described for the ScheduleExpression
interface. For more information, see http://docs.oracle.com/javaee/7/api/javax/ejb/ScheduleExpression.html
.
Table 10-1 WLDFScheduleBean Attributes and Default Values
Attribute | Description | Default | Allowable Values and Examples |
---|---|---|---|
second |
One or more seconds within a minute |
0 |
Allowable values: Can be a value, range, list, or interval. To specify every For example:
|
minute |
One or more minutes within an hour |
*/5 |
Allowable values: Can be a value, range, list, or interval. To specify every For example:
|
hour |
One or more hours within a day |
* |
Allowable values: Can be a value, range, list, or interval. For example:
|
dayOfWeek |
One or more days within a week |
* |
Allowable values:
Can be a value, range, or list. For example:
|
dayOfMonth |
One or more days within a month |
* |
Allowable values:
Can be a value, range, or list. For example:
|
month |
One or more months within a year |
* |
Allowable values:
Can be a value, range, or list. For example:
|
year |
A specific calendar year |
* |
Allowable values: a four-digit calendar year. You can specify one value. For example:
|
timezone |
Time zone for the schedule |
null |
Defaults to the local VM time zone. You may use this attribute to specify a non-default time zone ID in whose context the schedule specification is to be evaluated. |
A policy that has been evaluated to true
is referred to as having been triggered. For policies that are run repeatedly, an alarm determines when a policy can be evaluated again after it has been triggered. If a policy is configured with an alarm, a triggered policy is not evaluated again until the alarm is reset. For policies that are evaluated repeatedly, you can optionally define a minimum time that must transpire after a policy has been triggered before the policy can be evaluated again.
An alarm is important to configure for a policy that is run repeatedly to prevent the associated actions from being executed too frequently, such as generating a flood of emails or JMX notifications. For example, if you have a scheduled policy that executes a scale up action on a dynamic cluster, you should set an alarm that delays evaluating the policy again until the dynamic cluster is fully scaled up and is processing incoming requests. This delay is referred to as the alarm reset period. Without a proper alarm reset period, the scale up action could be executed again prematurely and counterproductively.
To configure an alarm for a policy, you specify the following:
The alarm type
The alarm reset period
The following table lists and describes each of the available alarm types:
Alarm Type | Description |
---|---|
None |
Allows the policy to be triggered whenever possible. This is the default. |
AutomaticReset |
Allows the policy to be triggered whenever possible, except that subsequent occurrences cannot occur any sooner than the interval specified in the alarm reset period. |
ManualReset |
Allows the policy to be triggered only once. After it is triggered, you must manually reset it to fire again. You can reset an alarm using a run-time MBean operation, either programmatically or with WLST. For example, you can use the |
Note the following alarm state behaviors:
When the alarm type is AutomaticReset
, a policy enters the alarm state when triggered and stays in that state until the time interval specified by the alarm reset period has expired.
If a policy is configured with a ManualReset
alarm, the policy enters the alarm state when triggered, and remains in that state until you manually reset it.
When a policy is in the alarm state, the policy is not evaluated again until the alarm is reset.
If a policy's alarm type is None
, the configured action can be executed every time that the policy is triggered. The alarm state is never set in these cases.
Whenever a policy is triggered, a message is automatically generated in the logging system. The severity option is an optional value you can configure that:
Gets assigned as the severity value of the log message generated in the logging system.
Is also passed to the actions that are configured with the policy.
The severity option must be one that is defined for the WebLogic logging service in the weblogic.logging.Severities
class. The accepted values are Info
, Notice
, Warning
, Error
, Critical
, Alert
, and Emergency
. The default is Notice
.
Each policy can be individually enabled and disabled by using the Enabled
attribute on that policy. The value you specify for this attribute is true
or false
. When disabled, a policy is not evaluated and its configured actions are not executed.
However, note that the WLDFWatchNotificationBean
, which is the parent of all policy and action configurations in a diagnostic system module, also has an Enabled
attribute. If the value of the WLDFWatchNotificationBean.Enabled
attribute is false
, all individual policies in the diagnostic system modules are disabled regardless of whether its policies are individually configured as enabled.
Scheduled policies monitor diagnostic data that consists of data coming from MBeans within the WebLogic Server Runtime MBean Server, including the read-only configuration MBeans in the WebLogic Server Runtime MBean Server. These values, called metrics, originate from common WebLogic Server JMX data sources such as the following:
WebLogic Server Runtime MBean Server
Domain Runtime MBean Server
JVM platform MBean server
Scheduled policies are useful for monitoring run-time state information in the WebLogic Server environment. Examples of diagnostic data that is useful to monitor using scheduled policies are:
Changes over time in average JVM heap usage
If the average amount of free heap reaches a particular threshold that is defined in the policy expression, the configured action is executed, such as sending an email to the system administrator.
Data from multiple services that are considered together, such as response-time metrics reported by a load balancer and message-backlog metrics from a message queue
If the combination of data meets a particular set of criteria defined in the policy expression, the policy can fire a scaling action
The following sections explain how to configure, and show examples of, the three scheduled policy types:
See also Chaining Policies for information about how to create a policy expression that can reference the state of other policies defined within the same WLDF module as the beans. Policy chaining allows the state of one policy to be part of the expression of another.
The simplest type of scheduled policy is the calendar based policy. You can use a calendar based policy to fire any associated actions according to the policy's schedule.
Calendar-based policies are simply scheduled policies with no associated expression. This enables purely schedule-driven action execution; that is, the ability to unconditionally perform a set of actions on a specified schedule. If no expression is provided, when the scheduled time occurs, the policy treats the empty expression as a true
result and executes the associated actions.
Note:
Calendar based policies are supported only for policies that: have the following configuration attributes:
The rule type specified as 'Harvester'
The expression language specified as 'EL'
The following example shows the configuration of a calendar based policy using WLST. This policy fires a scale up action at 3:00 a.m. on December 26.
calendarScaleUp=wn.lookupWatch('ChristmasReturnsScaleUpWatch') if calendarScaleUp == None: print "Creating scale-up for the post-Christmas returns rush on Dec 26 at 3am" calendarScaleUp=wn.createWatch('ChristmasReturnsScaleUpWatch') calendarScaleUp.setRuleType('Harvester') calendarScaleUp.setExpressionLanguage('EL') calendarScaleUp.getSchedule().setHour('3') calendarScaleUp.getSchedule().setMinute('0') calendarScaleUp.getSchedule().setSecond('0') calendarScaleUp.getSchedule().setDayOfMonth('26') calendarScaleUp.getSchedule().setMonth('Dec') calendarScaleUp.setEnabled(false) calendarScaleUp.addNotification(scaleUp)
Smart rules are prepackaged functions that greatly simplify the creation of policy expressions. The WebLogic Server Administration Console and Fusion Middleware Control, in particular, each contain a smart rule editor to greatly simplify the task of configuring a smart rule for the policy you are creating.
Smart rules perform a number of complex operations, but surface only a small number of configuration parameters that you set. Details about the specific low level metrics that are collected, how they are used, and so on, are hidden, thereby making them easy to use. Smart rules return only a Boolean value, which determines whether the policy is evaluated to true
.
You use a smart rule as a predicate in policy expression by simply specifying the parameters required by that smart rule. For example, to evaluate whether a particular increase exists in the average thread pool queue length in the local server, you create a policy that specifies the ServerHighQueueLength
smart rule as the policy expression and provide the following parameters:
The sampling period for collecting the value of the ThreadPoolRuntimeMBean.QueueLength
attribute
Duration, or the most recent window of time, in which samples are retained
A threshold value that establishes the maximum acceptable number of threads in the queue
The smart rule takes responsibility for sampling the appropriate metrics over the specified time interval, computing averages, comparing threshold values, and determining whether the smart rule evaluates to true
.
Note:
Smart rules are supported for use only in scheduled policies that are configured with Java EL as the expression language.
Smart rules can monitor trends in metrics in a server or cluster over time, including:
Average system throughput
Process CPU load
Pending user request count
Idle or stuck thread count
Incoming request queue size
System load average
JVM free heap size
Any metric value visible from JMX, such as custom MBean values
You can use smart rules as building blocks in policy expressions. In the simplest case, a single smart rule can be used by itself in a policy expression. You can also combine a smart rule with others, as well as with other EL constructs, to form more complex expressions.
For example, you can construct a policy that sends an email notification if all of the following conditions occur simultaneously in a server instance or cluster:
Low JVM free heap percentage
High number of stuck threads
High incoming requests queue size
For details about all the smart rules provided by WLDF, see Smart Rule Reference.
The ClusterLowHeapFreePercent smart rule compares the average free heap across all Managed Servers in a cluster by monitoring the value of the JVMRuntimeMBean.HeapFreePercent
attribute. A policy expression that uses this smart rule will be evaluated to true
if a minimum percentage of Managed Servers in the cluster have an average free heap that is less than a particular threshold value.
The ClusterLowHeapFreePercent
smart rule takes the following input parameters:
Cluster name
Sampling period — The frequency with which the value of the HeapFreePercent
metric is collected
Retention window — A sliding window of time during which samples are retained. For example, the most recent five minutes.
percentFreeLimit
— A value that represents the low free heap percentage threshold.
percentServersLimit
— A percentage of Managed Servers in the cluster that must have an average free heap that is less than percentFreeLimit
to cause the expression to evaluate to true
.
The following is an example configuration of the ClusterLowHeapFreePercent
smart rule:
wls:ClusterLowHeapFreePercent("myCluster","30 seconds","10 minutes",20,60)
From every Managed Server in myCluster
, this smart rule collects the value of the HeapFreePercent
every 30 seconds, retaining the most recent 10 minutes of data, and evaluates to true
if at least 60 per cent of the Managed Servers in myCluster
have an average free heap percentage that is less than 20 per cent.
This smart rule could be configured to fire an action when it evaluates to true
, such as sending an email to the system administrator to report that a low free heap condition exists in the cluster. The system administrator can then take remedial action as necessary.
You can use smart rules in conjunction with scaling actions, described in Section 11.9, "Configuring Elastic Actions", to configure policy based scaling of a dynamic cluster. This capability enables automated elasticity in that cluster. For more information, including a demo that you can download and run, see Configuring Elasticity in Dynamic Clusters for Oracle WebLogic Server.
Collected metrics based policies, which are a type of scheduled policy, can use WLDF-provided beans and functions within their policy expressions. These beans are JavaBean objects that can obtain access to common WebLogic Server JMX data sources, such as the following:
WebLogic Server Runtime MBean Server
Domain Runtime MBean Server
JVM platform MBean server
The following sections explain how to configure collected metrics based policies using beans and functions:
Table 10-2 summarizes the beans provided by WebLogic Server. For complete reference information about each of these beans, see WLDF Beans Reference.
Table 10-2 Beans Provided by WebLogic Server
Name | Prefix | Scope | Summary |
---|---|---|---|
runtime |
wls |
Only available from partition scope diagnostic system module deployments and partitions |
Provides access to MBeans in the local WebLogic Server Runtime MBean Server. These MBeans include both the read-only configuration MBean and RuntimeMBean instances. |
domainRuntime |
wls |
Administration Server |
Provides access to MBeans on the Domain Runtime MBean Server (Administration Server only). |
clusterRuntime |
wls |
Administration Server |
Provides a domain-wide access to cluster member data (Administraton Server only). |
instrumentationEvent |
wls |
Administration Server, Managed Server, and partitions |
Provides access to instrumentation fields in instrumentation policy expressions. |
log |
wls |
Administration Server, Managed Server, and partitions |
Provides access to log event fields in log policy expressions. |
platform |
wls |
Administration Server or Managed Server |
Provides access to the JVM's platform MBean server. Note that in the majority of cases, the |
partition |
wls |
Partition scope WLDF diagnostic system module deployments. |
Provides access to partition scope metrics. This bean is available only to policies that are configured in a diagnostic system module that is deployed in the same partition to which this bean is scoped. |
resource |
n/a |
Administration Server, Managed Server, and partitions |
Provides access to beans and state information within a diagnostic system module deployment. Access is restricted to policies that are configured within the same diagnostic system module. |
In addition to the bundled functions and operators that come with Java EL by default, you can also use WLDF-provided functions within policy expressions for common operations with metric data and for retaining a set of metrics with history. For reference information about all the functions included in WLDF, see Functions Reference.
Within the same diagnostics system module, the expression in one policy can reference other policies as beans within that expression. In this way, complex policy expressions can be reused and "chained" together to allow the state of one policy to be part of the expression of another. This allows more complex, interrelated policies to be written, while keeping such policy configurations more readable and maintainable.
To allow access to policy states within an expression, you can use the resource
bean within the global bean name space for each policy. The resource
bean supports a Map
attribute named watches
, where each key in the map is the name of a policy defined within the same diagnostics system module.
Each value in the policy's map is a bean representing the named policy. These policy beans support a simple Boolean alarm attribute, which has the following semantics:
If the policy is configured with an alarm type other than None
, the alarm attribute returns true
if the policy is currently in the alarm state.
If no alarm type is configured on the policy, the alarm attribute yields the most recently evaluated result.
If the alarm attribute on a policy bean is accessed before the named policy has successfully completed an evaluation cycle, a NotEnoughDataException
is thrown. This occurrence also has the effect of invalidating the expression during that evaluation cycle: the policy isn't disabled, but it is effectively a non-result when it occurs.
Use log policies to monitor the occurrence of specific messages or strings in the server or domain log. Policies of this type are triggered as a result of a log message containing the specified data being issued.
When creating a log policy, you can use the log bean in an EL expression to obtain access to log message fields. See logfor details about the log bean attributes.
Note:
Any log policies that search for the RUNNING state message ID should search for message ID BEA-000365
, and not BEA-000360
. The message ID BEA-000360
is issued immediately before the state change to RUNNING, and BEA-000365
is issued immediately afterward. Therefore, such log policies are able to find only message ID BEA-000365
.
An example configuration for a server log policy is shown in Example 10-1.
Example 10-1 Sample Configuration for a Log Watch (in DIAG_MODULE.xml)
<wldf-resource xmlns="http://xmlns.oracle.com/weblogic/weblogic-diagnostics" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:schemaLocation="http://xmlns.oracle.com/weblogic/weblogic-diagnostics/1.0/weblogic-diagnostics.xsd"> <name>mywldf1</name> <watch-notification> <enabled>true</enabled> <log-watch-severity>Info</log-watch-severity> <watch> <name>myLogWatch</name> <enabled>true</enabled> <rule-type>Log</rule-type> <rule-expression>log.messageId == 'BEA-000360'</rule-expression> <expression-language>EL</expression-language> <alarm-type>ManualReset</alarm-type> </watch> <smtp-notification> <name>myMailNotif2</name> <enabled>true</enabled> <mail-session-jndi-name>myMailSession</mail-session-jndi-name> <subject>This is a log alert</subject> <recipient>[email protected]</recipient> </smtp-notification> </watch-notification> </wldf-resource>
In Example 10-1, note how the <rule-type> of Log
causes messages or strings entered in the server log to be monitored. A <rule-type> of DomainLog
monitors messages or strings in the domain log.
You use instrumentation policies to monitor the events from the WLDF Instrumentation component. Policies of this type are triggered as a result of the event being posted.
Example 10-2 shows an example configuration for an Instrumentation policy.
Example 10-2 Sample Configuration for an Instrumentation Policy (in DIAG_MODULE.xml)
<watch-notification> <watch> <name>myInstWatch</name> <enabled>true</enabled> <rule-type>EventData</rule-type> <rule-expression>instrumentationEvent.payload > 100000000 && instrumentationEvent.monitor == 'Servlet_Around_Service'</rule-expression> <expression-language>EL</expression-language> <alarm-type>ManualReset</alarm-type> <notification>mySMTPNotification</notification> </watch> <smtp-notification> <name>mySMTPNotification</name> <enabled>true</enabled> <mail-session-jndi-name>myMailSession</mail-session-jndi-name> <subject xsi:nil="true"></subject> <body xsi:nil="true"></body> <recipient>[email protected]</recipient> </smtp-notification> </watch-notification>