Dennis Hammett
Container performance monitoring is essential to ensure the smooth running of applications. With the right tools, you can monitor containers in real-time, track bugs, and enhance security. Kibana, an open-source data visualisation interface, is one such tool that can be used in conjunction with other software to monitor container performance. When used with Elastic Observability, for example, Kibana can help visualise data from Kubernetes deployments, providing insights into the health and performance of hosts, containers, and applications. Similarly, when used with ECS clusters, Kibana can provide insights into CPU and memory utilisation, helping to balance clusters and tasks efficiently. To get started with monitoring container performance in Kibana, you will need to configure the settings in the kibana.yml file and ensure that data collection is enabled.
| Characteristics | Values |
|---|---|
| Monitoring tool | Kibana |
| Monitoring data | Logs and metrics from Kubernetes and applications |
| Visualisation | Line graphs, gauges and all other graphs |
| Data source | Docker containers |
| Data destination | Elasticsearch |
| Data sampling interval | 10 seconds |
| Log entries displayed | 10 |
What You'll Learn
- Utilise Kibana's monitoring collection settings to enable data collection from the Kibana NodeJS server
- Specify the location of the Elasticsearch cluster where monitoring data is stored
- Enable cross-cluster search of monitoring data
- Specify the username used by Kibana monitoring to establish a persistent connection
- Specify the password used by Kibana monitoring to establish a persistent connection
Utilise Kibana's monitoring collection settings to enable data collection from the Kibana NodeJS server
To utilise Kibana's monitoring collection settings to enable data collection from the Kibana NodeJS server, follow these steps:
- Ensure that Stack Monitoring is enabled. By default, Stack Monitoring is enabled, but data collection is disabled. When you first start Kibana monitoring, you will be prompted to enable data collection.
- Adjust how monitoring data is collected from Kibana and displayed by configuring settings in the `kibana.yml` file. The `monitoring.kibana.collection.enabled` setting should be set to `true` to enable data collection from the Kibana NodeJS server.
- Specify the interval for data sampling on the Kibana NodeJS server using the `monitoring.kibana.collection.interval` setting. This setting determines the number of milliseconds to wait between data sampling for the metrics displayed in the Kibana dashboards. The default value is 10000 (10 seconds).
- Verify that `monitoring.ui.enabled` is set to `true` in the `kibana.yml` file. This setting ensures that the Stack Monitoring page is visible and accessible.
- If you are using Elastic Stack security features, you must be signed in as a user with the `cluster:manage` privilege to enable data collection. The built-in superuser role has this privilege, and the built-in elastic user has this role.
- Configure encryption for communication between the Kibana server and the monitoring cluster if required.
By following these steps, you can utilise Kibana's monitoring collection settings to enable and customise data collection from the Kibana NodeJS server, providing valuable insights into the performance and health of your containers.
Testing a PVM Monitor: Pre-Purchase Guide
You may want to see also
Specify the location of the Elasticsearch cluster where monitoring data is stored
To specify the location of the Elasticsearch cluster where monitoring data is stored, you need to configure the setting monitoring.ui.elasticsearch.hosts in the kibana.yml file. This setting allows you to use a single Kibana instance to visualise data in your production cluster and monitor data sent to a dedicated monitoring cluster.
By default, the value of monitoring.ui.elasticsearch.hosts is the same as elasticsearch.hosts. However, you can modify it to specify the location of the Elasticsearch cluster where your monitoring data is stored. This setting enables you to use Kibana to visualise and search data in your production cluster while also monitoring data sent to a separate, dedicated monitoring cluster.
In addition to the monitoring.ui.elasticsearch.hosts setting, there are other related settings that you can configure:
- Monitoring.ui.elasticsearch.username: Specifies the username used by Kibana monitoring to establish a persistent connection and verify licensing status.
- Monitoring.ui.elasticsearch.password: Specifies the password used by Kibana monitoring for authentication and licensing verification.
- Monitoring.ui.elasticsearch.serviceAccountToken: Specifies a service account token for the Elasticsearch cluster, providing an alternative to using the username and password.
- Monitoring.ui.elasticsearch.pingTimeout: Sets the time to wait for Elasticsearch to respond to internal health checks; the default is 3000 milliseconds.
- Monitoring.ui.elasticsearch.ssl: Configures encrypted communication between Kibana and the monitoring cluster, sharing the same configuration as elasticsearch.ssl.
Monitoring Data Usage: TDS Telecommunications Service Guide
You may want to see also
Enable cross-cluster search of monitoring data
Cross-cluster search (CCS) is a feature that allows users to configure multiple remote clusters across different locations and enables federated search queries across all the configured remote clusters. This is particularly useful when you want to filter and analyse log data stored on clusters in different data centres.
To enable CCS, your deployments must meet the following criteria:
- Local and remote clusters must be in compatible versions.
- The steps, information, and authentication method required to configure CCS can vary depending on where the clusters you want to use as remote are hosted.
CCS supports the following APIs:
- Multi-search template
- Painless execute API
- EQL search
- SQL search
- Vector tile search
- ES|QL
To set up remote clusters, you can refer to the documentation provided by Elastic for setting up remote clusters on Elastic Cloud or for self-hosted clusters.
Disabling Remote Neural Monitoring: A Guide to Regaining Privacy
You may want to see also
Specify the username used by Kibana monitoring to establish a persistent connection
The username used by Kibana monitoring to establish a persistent connection to the Elasticsearch monitoring cluster is specified by the setting monitoring.ui.elasticsearch.username. This setting is found in the kibana.yml file, which allows you to adjust how monitoring data is collected from Kibana and displayed in Kibana.
If the monitoring.ui.elasticsearch.username setting is not set, Kibana will use the value of the elasticsearch.username setting. It is important to note that the authenticated user's credentials must be the same on both the Elasticsearch monitoring cluster and the Elasticsearch production cluster.
In addition to the username, you can also configure the password or service account token used by Kibana monitoring for establishing a persistent connection. This can be done through the monitoring.ui.elasticsearch.password or monitoring.ui.elasticsearch.serviceAccountToken settings, respectively. Similar to the username setting, if these values are not set, Kibana will default to the elasticsearch.password setting.
Measuring LCD Monitors: Size, Resolution, and More
You may want to see also
Specify the password used by Kibana monitoring to establish a persistent connection
To specify the password used by Kibana monitoring to establish a persistent connection in Kibana to the Elasticsearch monitoring cluster, the setting to adjust is monitoring.ui.elasticsearch.password. This setting also verifies the licensing status on the Elasticsearch monitoring cluster when using monitoring.ui.elasticsearch.hosts.
The default behaviour is for Kibana to use the value of the elasticsearch.password setting. However, if this is not set, you can specify the password in the kibana.yml file.
Ankle Monitors: Effective Solution or Costly Immigration Control?
You may want to see also
Frequently asked questions
To monitor containers in Kibana, you need to have Elasticsearch set up to store and search your observability data. You can then use Kibana to visualise and manage this data.
To set up monitoring, you need to add the Kubernetes integration to your policy and configure the metrics and logs you want to collect. You can then download and update your manifest, and finally, explore logs and metrics using Kibana.
Some key metrics to monitor are CPU and memory utilisation, task and service counts, read/write into storage, and container instance counts. Monitoring these can help you balance your clusters and tasks, and a well-tuned scaling policy will help reduce costs while maintaining a highly responsive service stack.
Monitoring containers helps with the early detection and resolution of issues, avoiding breakdowns during production. It also enables the safe implementation of new feature additions and updates, as the entire application is monitored.
