Summary
Adds on server side:
- a general data annotation mechanism to data collection (see details below)
- SQL max execution time metric based on SQL tracing
- a watch for SQL max execution time with threshold from pool configuration
- extends watch conditions
- adds series tag name wildcards, ala
?:foo(any tag name with valuefooincluding no tag) to allow explicit selection of family of series where some have further tags and others have not like used for lists of health check alerts or a general alert list - REST API for data requests supports better selection of the needed type of data (points, alerts, annotations, watches) to avoid sending data between client and server that isn't used by the client
- extends request tracing metric so it does allow adding useful watch
- adds a watch to cause alerts in case request tracing is above threshold
- adds "compaction" for alert frames to limit memory consumption and client-server data exchange
Adds on client side:
- shows thresholds of multiple watches in charts as stacked threshold indicator lines/areas
- fixes threshold indicator gradient and line alignment
- fixes alert condition formatting
- adds annotations to alerts with setting to hide them
- adds new widget type Annotations that lists just the matching annotations in a list or table
- adds new widget setting Mode to switch between Table or List layout for Annotations widgets
- widgets now have a page wide unique id so that same series (pattern) can be used multiple times on the same page as done for SQL, request tracing or alerts.
- adds a new SQL preset page that show slow SQL tracing information
- fixes alert table vertical overflow by using a scrollbar when needed
- adds list of tracing data and tracing metric with alerts to request tracing page
- adds a general Alerts page preset that shows all alerts organised in groups
Data Annotations
Annotations are a new concept in collection of metrics to attach key-value data to a particular metric at a particular point in time. Such annotation should only be done in situations that are in some way noteworthy such as a metric's value exceeding a particular threshold. In general annotations are event like information that are stored per series in a queue of fixed size. Newer annotations eventually replace older annotations to avoid number of annotations growing out of control. Current limit is set at 20 per metric series.
The annotation data is used by the client to enhance views such as lists of alerts with more data for the period of the alert or to allow widgets of type annotation that for example can list the slow SQL meta data in table.
Annotations provide a general mechanism that makes such values available for any metric and from all instances. To verify the general nature of the concept it has also been applied to request tracing which allows to also list the trace span attributes in a table.
Testing
New unit tests were added for
* extended variants of Condition logic
* extended Series patterns using the added ? wild-card
* Alert.Frame compaction
Testing Performed
Manual testing of various features. This PR does change general parts and extends others. It makes sense to test usual workflows even though they were not an explicit target of the changes.
Testing Instructions
General Setup:
1. build, install and start the server
2. use set-monitoring-console-configuration --enabled=true to deploy MC
3. open MC at http://localhost:8080/monitoring-console/
4. make sure browser cache for JS/CSS is cleared for MC's domain
5. check that following pages do exist: SQL and Alerts (if not most likely a browser cache issue - or get in touch)
Testing slow SQL tracing:
1. Open admin console at http://localhost:4848/
2. navigate to Resources => JDBC => JDBC Connection Pools
3. select H2Pool tab Advanced
4. change Slow Query Log Threshold from -1 (disabled) to 0.001 (1ms)
5. open page SQL in MC
6. deploy e.g. https://github.com/javaee/tutorial-examples/tree/master/persistence/order (some app using the default pool)
7. deployment should already cause alert(s) and annotations being shown on the SQL page.
8. use the order app some and see how "slow" queries cause alerts and SQL is available in the table on SQL page
NOTE: the order app causes errors during deploy on server restart. Undeploy the app and redeploy with server running. AFAIK this is a shortcoming of the app's setup/scripts.
Testing Request Tracing (again):
1. Open admin console at http://localhost:4848/
2. navigate to Configurations => server-config => Request Tracing
3. check Enabled
4. set Target Count to 2, Time Value to 20, Time Unit to SECONDS
5. set Threshold Value to somewhere between 5 to 20, Threshold Unit to MILLISECONDS
6. save changes
7. open MC Request Tracing page. The polling of MC itself should be enough to cause some data to appear in the widgets
Testing Health Checks (again): 1. Open admin console at http://localhost:4848/ 2. navigate to Configurations => server-config => HealthCheck 3. open tab CPU Usage , check Enabled and save changes 4. eventually open other tabs to enable more checks, the Enabled checkbox on General tab does not have to be checked. 5. open Health Checks page in MC and check that the graphs for the enabled checks do show data and thresholds 6. go back to admin console configuration and change a threshold, e.g. heap usage to a level that is in between the low and high point currently present for your GC cycle (see graph). 7. check that changed threshold is updated in graph and that alerts are caused when heap usage is exceeded (note that the alert condition requires the threshold to be exceeded for average of last 15 points)
Further things in MC to test (again) * create a new page, name it, add some widgets, remove some widgets * reset preset pages after changes * change settings of widgets * change colour scheme * clear local storage for the page and reload the page (full reset or client side settings)
该提问来源于开源项目：payara/Payara
