Apigee Monitoring with Influx, Grafana and Telegraf

Apigee Monitoring

Apigee Edge Private Cloud Monitoring can really be a dauting task. Apigee Supports JMX and you can use any of the JMX tools like JConsole to monitor system parameters. This community article provides insight on monitoring with Grafana, Influx and Telegraf.

Refer to the git project https://github.com/rajeshm7910/apigee-monitoring for all more details.

Architecture

Dashboard Monitoring

Influxdb

Influxdb is a time series database. This will hold events, system and application related monitoring data as time series.

Install Influxdb

Please refer https://docs.influxdata.com/influxdb/v1.5/introduction/installation/ for installing influxdb. We are supporting influxdb 1.2.2+.

Requirements:

  • CPU: 2 Core
  • RAM: 4 GB
  • Disk: 500 GB SDD, 3000 IOPS
  • Ports 8086 and 8183 must be open on the influxdb node

In case you are using RHEL/CentOS, you can install with following commands:

$ curl -O https://dl.influxdata.com/influxdb/releases/influxdb-1.5.3.x86_64.rpm
$ sudo yum localinstall influxdb-1.5.3.x86_64.rpm

Enable and start influxdb:

$ systemctl enable influxdb
$ systemctl start influxdb

Configure Influxdb

Once influxdb is installed, go to the influx terminal to create initial databases:

$ influx
Connected to http://localhost:8086 version 1.5.3
InfluxDB shell version: 1.5.3
>
  • Show databases:
> SHOW databases
name: databases
name
----
_internal
> 
  • Create databases - system, events, application:
> CREATE database application
> CREATE database system
> CREATE database events
> SHOW databases
name: databases
name
----
_internal
system
events
application
> exit

Grafana

Grafana UI allows visual representation of events and metrics. In this case we will import some pre-built dashboards that will give some useful insights into Apigee.

Install Grafana

Please refer to http://docs.grafana.org/installation/rpm/ for installing grafana. You can install grafana on management server node or any other nodes.

  • Add the following to a new file at /etc/yum.repos.d/grafana.repo. If the file does not exist, create it:
[grafana]
name=grafana
baseurl=https://packagecloud.io/grafana/stable/el/7/$basearch
repo_gpgcheck=1
enabled=1
gpgcheck=1
gpgkey=https://packagecloud.io/gpg.key https://grafanarel.s3.amazonaws.com/RPM-GPG-KEY-grafana
sslverify=1
sslcacert=/etc/pki/tls/certs/ca-bundle.crt
  • Install Grafana as follows:
$ sudo yum install grafana
  • Start the server (via systemd):
$ sudo systemctl daemon-reload
$ sudo systemctl start grafana-server
$ sudo systemctl status grafana-server
  • Enable the systemd service to start at boot:
$ sudo systemctl enable grafana-server.service

Configure Grafana

Create 3 datasources : application-dc-1 , system-dc-1 and events-dc-1 and point them to application, system and events database of influx.

An example configuration looks like:

  • name: application-dc-1
  • type : Choose the type to influxDB
  • HTTP URL : This is the influxdb machines ip address. The default port is 8086.
  • Influx DB Details: -- Database : application -- User: root -- Password: root

Application Data Source

Application Data Source

Events Data Source

Events Data Source

System Data Source System Data Source

Import Dashboards

Import all the pre-built dashboards provided in GIT. You can also build your own dashboards. Upload the dashboard JSON files from the GIT repo at https://github.com/rajeshm7910/apigee-monitoring.

Import Dasboards Import Dasboard

Telegraf

Telegraf is the collection agent that collects all metrics from all apigee components.

Install and start Telegraf

Install telegraf agent on all Apigee nodes which needs monitoring. This step installs telegraf from the Apigee repo, so it disables the influxdb repo, it it is configured:

$ sudo yum --disablerepo=influxdb install telegraf
$ sudo systemctl start telegraf
$ sudo systemctl enable telegraf
$ sudo systemctl status telegraf

Configure Telegraf

This needs to be configured in all nodes.

  • Copy telegraf.conf from samples provided in git telegraf/telegraf.conf to /etc/telegraf/telegraf.conf on the host.
  • Edit /etc/telegraf/telegraf.conf and replace 127.0.0.1 with IP address of the influxdb server.
  • Copy relevant .toml files present in telegraf/telegraf.d/ to /etc/telegraf/telegraf.d/ directories based on the Apigee component running on that node. For example, if node is running Apigee Cassandra and Apigee Zookeeper only, copy apigee-cassandra.toml and apigee-zookeeper.toml to /etc/telegraf/telegraf.d/
  • If the Management Server is running, copy edge-management-server.toml.
  • Restart telegraf service:
$ sudo systemctl restart telegraf

Finishing Up

You can now browse the Grafana UI at http://grafanaIPorDNS:3000

Login Credentials:

  • user : admin
  • password: admin

Hurray!!! You have just setup basic monitoring.

Comments
manojkumarpalav
New Member

I just followed all the above steps, I am able to monitor CPU, Memory ..etc but the real monitoring for apigee private cloud is not happening, i.e I am not able see policy errors,apigee errors ..etc in grafana. Can someone help me here.

Thanks in advance

rajeshmi
Staff

This works with apigee-telegraf resolved from apigee third party library. Please check the version of telegraf

manojkumarpalav
New Member

I have wrong rpm installed for telegraph. I removed the old rpm and downloaded telegraf rpm from apigee repository and got it resolved. Thanks for your help Rajesh.

prasad-vepa
New Member

Hi, Does this provides the JVM level monitoring dashboards with jolokia? In the previous beta version, grafana dashboards provided wholistic monitoring.

pbhaskaruni
New Member

I'm also facing the same issue. After installing telegraf I'm not receiving any logs.

I used Telegraf 1.10.3 (git: HEAD 294bb666)

Should I need to change it?

ritesh-khobraga
New Member

Hi,

I have installed telegraph wget https://dl.influxdata.com/telegraf/releases/telegraf-1.13.4-1.x86_64.rpm

but still apigee private cloud monitoring is not showing up on grafana.

Please help on the same.

Thanks

Ritesh

Version history
Last update:
‎06-04-2018 06:05 PM
Updated by: