Setting up the Grafana Dashboard#

Now that you have your node up and running, you'll probably want to have a convenient way to monitor everything about it at a glance to make sure it's functioning correctly (and what kind of earnings it's generating for you).

There are many tools out there that do this job. One of the most popular is called Grafana - an easy-to-use, general-purpose dashboard system that you can access with a browser.

Rocket Pool comes out-of-the-box with support for Grafana and its dependencies; it even comes with a pre-built dashboard for each of the Consensus clients. For example, here is a snapshot of what the dashboard looks like on the Holesky test network:

The standard dashboard includes the following information all in a convenient format:

• Top left: some important statistics about your machine's health and performance, and any pending system updates
• Top right: the activity and performance of your validators on the Beacon Chain, along with some Execution and Consensus client stats
• Bottom left: details about the entire Rocket Pool network, for reference
• Bottom right: details about your staking rewards, both ETH and RPL

In this guide, we'll show you how to enable Rocket Pool's metrics system so you can use this dashboard - or even build your own!

Overview of the Rocket Pool Metrics Stack#

If you choose to enable metrics during the Smartnode configuration process, your node will add the following processes:

• Prometheus - a data collection, storage, and reporting system that captures all of the metrics you see above (any many more) and stores them, so they can be looked at over time
• The Prometheus Node Exporter - a service that collects information about your machine's health (such as CPU usage, RAM usage, free disk space and swap space, etc.) and reports it to Prometheus
• Grafana, the tool that exposes Prometheus's data through a convenient website hosted on your node
• An optional custom set of scripts that will report any available Operating System updates to Prometheus, so you know if your system needs to be patched

The default configuration will create Docker containers with all of these services that live alongside the rest of the Smartnode's Docker containers. It will open up a port on your node machine for Grafana, so you can access its dashboard from any machine on your local network with a browser.

Enabling the Metrics Server#

rocketpool service config

rocketpool service install-update-tracker

docker restart rocketpool_exporter

Configure firewall to allow connections for monitoring#

RP_NET=$(docker inspect rocketpool_net | grep -Po "(?<=\"Subnet\": \")[0-9./]+")
sudo ufw allow from $RP_NET to any port 9105 comment "Allow Prometheus access to Execution Client"
sudo ufw allow from $RP_NET to any port 9100 comment "Allow Prometheus access to Consensus Client"
sudo ufw allow from $RP_NET to any port 9103 comment "Allow Prometheus access to Exporter"

You can then open the firewall to allow external devices access to your Grafana dashboard.

# This assumes your local IP structure is 192.168.1.xxx
sudo ufw allow from 192.168.1.0/24 proto tcp to any port 3100 comment 'Allow grafana from local network'

Setting up Grafana#

Now that the metrics server is ready, you can access it with any browser on your local network.

Refer to the tabs below for your Smartnode installation mode.

Navigate to the following URL, substituting the variables with your setup as necessary:

For example, if you node's IP was 192.168.1.5 and you used the default Grafana port of 3100, then you would go to this URL in your browser:

You will see a login screen like this:

The default Grafana information is:

You will then be prompted to change the default password for the admin account. Pick something strong and don't forget it!

docker exec -it rocketpool_grafana grafana-cli admin reset-admin-password admin

Thanks to community member tedsteen's work, Grafana will automatically connect to your Prometheus instance so it has access to the metrics that it collects. All you need to do is grab the dashboard!

Importing the Rocket Pool Dashboard#

Now that you have Grafana attached to Prometheus, you can import the standard dashboard (or build your own using the metrics that it provides, if you are familiar with that process).

Start by going to the Create menu (the plus icon on the right-side bar) and click on Import:

When prompted for the dashboard ID in the Import via grafana.com box, enter 21863 or use the full URL ((https://grafana.com/grafana/dashboards/21863) and press the Load button.

You will be prompted with some information about the dashboard here, such as its name and where you'd like to store it (the default General folder is fine unless you use a lot of dashboards and want to organize them).

Under the Prometheus drop-down at the bottom, you should only have a single option labeled Prometheus (default). Select this option.

Your screen should look like this:

If yours matches, click the Import button and you will be immediately taken to your new dashboard.

At first glance, you should see lots of information about your node and your validators. Each box comes with a handy tooltip on the top left corner (the i icon) that you can hover over to learn more about it. For example, here is the tooltip for the Your Validator Share box:

However, we aren't done setting things up yet - there is still a little more configuration to do.

Tailoring the Hardware Monitor to your System#

Now that the dashboard is up, you might notice that a few boxes are empty such as SSD Latency and Network Usage. We have to tailor the dashboard to your specific hardware so it knows how to capture these things.

CPU Temp#

To update your CPU temperature gauge, click the title of the CPU Temp box and select Edit from the drop down. Your screen will now look something like this:

This is Grafana's edit mode, where you can change what is displayed and how it looks. We're interested in the query box highlighted in red, to the right of the Metrics browser button.

By default, that box has this in it:

There are two fields in this text that are currently blank: chip and sensor. These are unique to each machine, so you'll have to fill them in based on what your machine provides.

To do this, follow these steps:

1. Remove the , sensor="" portion so it ends with chip=""}. For clarity, the whole thing should now be node_hwmon_temp_celsius{job="node", chip=""}.
2. Put your cursor in-between the quote marks of chip="" and press Ctrl+Spacebar. This will bring up an auto-complete box with the available options, which looks like this:

3. Select the option that corresponds to your system's CPU.
4. Once that's selected, add , sensor="" back into the string. Place your cursor in-between the quote marks of sensor="" and press Ctrl+Spacebar to get another auto-complete menu. Select the sensor you want to monitor.

Once you're happy with your selections, click the blue Apply button in the top right corner of the screen to save the settings.

SSD Latency#

The SSD Latency chart tracks how long it takes for read/write operations to take. This is helpful in gauging how fast your SSD is, so you know if it becomes a bottleneck if your validator suffers from poor performance. To update the SSD you want to track in the chart, click on the SSD Latency title and select Edit.

This chart has four query fields (four textboxes) with eight device="" portions in total. You'll need to update the first four of these portions with the device you want to track.

Simply place your cursor in-between the quote marks and press Ctrl+Spacebar to get Grafana's auto-complete list, and select the correct option from there for each of the device="" portions. You want to start from the leftmost empty setting first, or the auto-complete list may not appear.

Optionally, you can also track latency of a second disk on your system. This is aimed at people that keep their Operating System and chain data on separate drives. To set this up, simply follow the instructions above for the last two query fields, substituting device="" portion values with those of the disk you want to track.

Once you're happy with your selections, click the blue Apply button in the top right corner of the screen to save the settings.

Network Usage#

This chart tracks how much data you're sending and receiving over a particular network connection. As you might expect, the dashboard needs to know which network you want it to track.

To change it, click on the Network Usage title and select Edit.

This chart has two query fields with two device="" portions in total. You'll need to update these with the network you want to track.

Place your cursor in-between the quote marks and press Ctrl+Spacebar to get Grafana's auto-complete list, and select the correct option from there for each of the device="" portions.

sudo route

Once you're happy with your selections, click the blue Apply button in the top right corner of the screen to save the settings.

Total Net I/O#

This tracks the total amount of data you've sent and received. You might find it useful to watch if, for example, your ISP limits you to a certain amount of data per month.

The setup is identical to the Network Usage box above, so simply follow those instructions for this box too.

Disk Space Used#

This keeps tabs on how full your Operating System disk is getting, so you know when it's time to clean up (and if your chain data lives on the same drive, time to prune Geth or Nethermind).

The steps are the same as the SSD Latency box above, so simply follow those instructions for this box too. As a reminder, you want the drive that houses the partition which has / in the MOUNTPOINT column for this one because that will be your Operating System drive. Fill this into the first query field.

Optionally, you can also track the free space of a second disk on your system. This is aimed at people that keep their Operating System and chain data on separate drives. Set this up by following the same process, but instead of looking at which partition has / in the MOUNTPOINT column, you want to look for the one that has whatever your second drive's mount point is. Update the second query field with the disk associated with that partition.

Disk Temp#

This tracks the current temperature of your Operating System disk. The steps are the same as the CPU Temp box above, so simply follow those instructions for this box too, substituting CPU chip and sensor values with those of your Operating System disk. Fill these values into the first query field.

Optionally, you can also track the current temperature of a second disk on your system. Set this up by following the same process, substituting the chip and sensor values with those of your second drive. Fill these values into the second query field.

Customizing the Dashboard#

While the standard dashboard tries to do a good job capturing everything you'd want to see at a glance, it's quite easy to customize a Grafana dashboard however you want. You can add new graphs, change the way graphs look, move things around, and much more!

Take a look at Grafana's Tutorials page to learn how to play with it and set it up to your liking.

Customizing the Metrics Stack#

The tools used in the Rocket Pool Metrics Stack offer a wide array of configuration options beyond what is included in the default Rocket Pool installation. This section includes configuration examples for different use cases.

In general, Grafana configuration options should be passed through using environment variables in override/grafana.yml. Any config option can be converted to an environment variable using the following syntax:

Grafana SMTP Settings for Sending Emails#

To send emails from Grafana, e.g. for alerts or to invite other users, SMTP settings need to be configured in the Rocket Pool Metrics Stack. See the Grafana SMTP configuration page for reference.

version: "3.7"
services:
grafana:
x-rp-comment: Add your customizations below this line
environment:
## SMTP settings start, replace values with those of your SMTP provider
- GF_SMTP_ENABLED=true
- GF_SMTP_HOST=mail.example.com:`port` # Gmail users should use smtp.gmail.com:587
- [email protected]
- GF_SMTP_PASSWORD=password
- [email protected]
- GF_SMTP_FROM_NAME="Rocketpool Grafana Admin"
## SMTP server settings end

After making these modifications, run the following to apply the changes:

docker stop rocketpool_grafana

rocketpool service start

To test the SMTP settings, go to the Alerting menu and click Contact points.

Click New contact point and select Email as the Contact point type. Enter an email address in the Addresses section and click Test.

Check to see that the test email was received. Click Save contact point* when finished.