Automating the Installation of Prometheus and Related Tools

In this guide, we walk through the automated installation and setup of Prometheus, Alertmanager, Blackbox Exporter, Grafana, and Node Exporter on a Linux system using a Bash script. This script not only simplifies the deployment of these tools but also ensures a structured approach with user prompts, customizable settings, and systemd service integration.

Note: This installation process is specifically designed for x86 architecture and is not compatible with ARM architecture.

Problem Statement

Setting up monitoring and observability tools like Prometheus and its ecosystem often involves repetitive tasks, such as downloading specific versions, configuring directories, setting up system services, and ensuring proper permissions. These steps, if done manually, are prone to errors and inefficiencies. Without proper automation, deploying and maintaining these tools can be time-consuming, especially for teams managing multiple systems or environments.

Solution

setup.sh

#!/bin/bash

set -e

read -p "Enter the number of days for data retention: " RETENTION_DAYS
DATA_RETENTION="${RETENTION_DAYS}d"

echo "Fetching available Prometheus versions..."
VERSION_LIST=$(curl -s https://api.github.com/repos/prometheus/prometheus/releases | jq -r '.[].tag_name' | sed 's/^v//' | head -n 5)
echo "Top 5 available versions:"
echo "$VERSION_LIST"

read -p "Enter the Prometheus version you want to install (do not include 'v'; default is 2.43.0): " USER_PROMPT
PROMETHEUS_VERSION=${USER_PROMPT:-2.43.0}  # Default to 2.43.0 if no input is provided

INSTALL_DIR="/opt/prometheus"
DATA_DIR="/data/prometheus"
USER="prometheus"
SERVICE_FILE="/etc/systemd/system/prometheus.service"

echo "Updating the system..."
sudo apt update && sudo apt upgrade -y && sudo apt-get install -y wget curl vim net-tools jq

if id "$USER" &>/dev/null; then
    echo "User '$USER' already exists."
else
    echo "Creating Prometheus user..."
    sudo useradd --no-create-home --shell /bin/false $USER
fi

echo "Creating directories..."
sudo mkdir -p $INSTALL_DIR
sudo mkdir -p $DATA_DIR

echo "Downloading Prometheus version v$PROMETHEUS_VERSION..."
wget "https://github.com/prometheus/prometheus/releases/download/v$PROMETHEUS_VERSION/prometheus-$PROMETHEUS_VERSION.linux-amd64.tar.gz"

echo "Extracting Prometheus..."
tar xvf "prometheus-$PROMETHEUS_VERSION.linux-amd64.tar.gz"

echo "Moving binaries to $INSTALL_DIR..."
sudo mv prometheus-$PROMETHEUS_VERSION.linux-amd64/* $INSTALL_DIR/.

echo "Setting permissions for Prometheus binaries..."
sudo chown $USER:$USER $INSTALL_DIR/prometheus
sudo chown $USER:$USER $INSTALL_DIR/promtool
sudo chown -R $USER:$USER $DATA_DIR
sudo chown -R $USER:$USER $INSTALL_DIR

sudo mv $INSTALL_DIR/prometheus /usr/local/bin/.
sudo mv $INSTALL_DIR/promtool /usr/local/bin/.
sudo chown $USER:$USER /usr/local/bin/prometheus
sudo chown $USER:$USER /usr/local/bin/promtool

echo "Creating systemd service file..."
cat <<EOF | sudo tee $SERVICE_FILE
[Unit]
Description=Prometheus Monitoring System
Wants=network-online.target
After=network-online.target

[Service]
User=$USER
Group=$USER
Type=simple
ExecStart=/usr/local/bin/prometheus \
  --config.file=$INSTALL_DIR/prometheus.yml \
  --storage.tsdb.path=$DATA_DIR/data \
  --web.listen-address=0.0.0.0:9090 \
  --storage.tsdb.retention.time=$DATA_RETENTION

[Install]
WantedBy=multi-user.target
EOF

echo "Starting and enabling Prometheus service..."
sudo systemctl daemon-reload
sudo systemctl start prometheus
sudo systemctl enable prometheus

echo "Cleaning up..."
rm -rf "prometheus-$PROMETHEUS_VERSION.linux-amd64.tar.gz"
rm -rf "prometheus-$PROMETHEUS_VERSION.linux-amd64"

echo "Prometheus installation completed and running as a service."

echo "Fetching available Alertmanager versions..."
LATEST_VERSIONS=$(curl -s https://api.github.com/repos/prometheus/alertmanager/releases | \
    jq -r '.[].tag_name' | sed 's/^v//' | sort -V | tail -n 5)

echo "Last available versions:"
echo "$LATEST_VERSIONS"

# Prompt for the Alertmanager version
read -p "Enter the Alertmanager version you want to install (default is 0.26.0, do not include 'v'): " USER_VERSION
ALERTMANAGER_VERSION=${USER_VERSION:-0.26.0}  # Default to 0.26.0 if no input is provided

INSTALL_DIR="/opt/alertmanager"
BIN_DIR="/usr/local/bin"
SERVICE_FILE="/lib/systemd/system/alertmanager.service"

# Update the system
echo "Updating the system..."
sudo apt update && sudo apt upgrade -y && sudo apt-get install -y wget curl vim net-tools netcat-openbsd

echo "Downloading Alertmanager version $ALERTMANAGER_VERSION..."
wget "https://github.com/prometheus/alertmanager/releases/download/v$ALERTMANAGER_VERSION/alertmanager-${ALERTMANAGER_VERSION}.linux-amd64.tar.gz"

echo "Extracting files..."
tar -xvf "alertmanager-${ALERTMANAGER_VERSION}.linux-amd64.tar.gz"

# Check if user exists
if id "alertmanager" &>/dev/null; then
    echo "User 'alertmanager' already exists."
else
    echo "Creating Alertmanager user..."
    sudo useradd -rs /bin/false alertmanager
fi

# Create necessary directories
if [ ! -d "$INSTALL_DIR" ]; then
    echo "Creating installation directory..."
    sudo mkdir -p $INSTALL_DIR
fi

echo "Moving executable files to $BIN_DIR..."
sudo mv alertmanager-${ALERTMANAGER_VERSION}.linux-amd64/alertmanager $BIN_DIR
sudo mv alertmanager-${ALERTMANAGER_VERSION}.linux-amd64/amtool $BIN_DIR
sudo mv alertmanager-${ALERTMANAGER_VERSION}.linux-amd64/* $INSTALL_DIR

# Set ownership for Alertmanager files
echo "Setting ownership for Alertmanager files..."
sudo chown alertmanager:alertmanager $BIN_DIR/alertmanager
sudo chown alertmanager:alertmanager $BIN_DIR/amtool
sudo chown -R alertmanager:alertmanager $INSTALL_DIR

echo "Creating systemd service file..."
sudo bash -c "cat <<EOT > $SERVICE_FILE
[Unit]
Description=Alertmanager Service
Wants=network-online.target
After=network-online.target

[Service]
Type=simple
User=alertmanager
Group=alertmanager
ExecStart=$BIN_DIR/alertmanager \
  --config.file=$INSTALL_DIR/alertmanager.yml \
  --storage.path=$INSTALL_DIR/data

Restart=always

[Install]
WantedBy=multi-user.target
EOT"

echo "Enabling and starting Alertmanager service..."
sudo systemctl daemon-reload
sudo systemctl enable alertmanager.service
sudo systemctl start alertmanager.service

# Clean up
echo "Cleaning up..."
rm -rf "alertmanager-${ALERTMANAGER_VERSION}.linux-amd64.tar.gz"
rm -rf "alertmanager-${ALERTMANAGER_VERSION}.linux-amd64"

echo "Alertmanager version $ALERTMANAGER_VERSION installation completed."

echo "Fetching available Blackbox Exporter versions..."
LATEST_VERSIONS=$(curl -s https://api.github.com/repos/prometheus/blackbox_exporter/releases | \
    jq -r '.[].tag_name' | sed 's/^v//' | sort -V | tail -n 5)

echo "Last available versions:"
echo "$LATEST_VERSIONS"

# Prompt for the Blackbox Exporter version
read -p "Enter the Blackbox Exporter version you want to install (default is 0.24.0, do not include 'v'): " USER_VERSION
BLACKBOX_VERSION=${USER_VERSION:-0.24.0}  # Default to 0.24.0 if no input is provided

INSTALL_DIR="/opt/blackbox_exporter"
BIN_DIR="/usr/local/bin"
SERVICE_FILE="/lib/systemd/system/blackbox_exporter.service"

echo "Updating the system..."
sudo apt update && sudo apt upgrade -y && sudo apt-get install -y wget curl vim net-tools netcat-openbsd

echo "Downloading Blackbox Exporter version $BLACKBOX_VERSION..."
wget "https://github.com/prometheus/blackbox_exporter/releases/download/v$BLACKBOX_VERSION/blackbox_exporter-${BLACKBOX_VERSION}.linux-amd64.tar.gz"

echo "Extracting files..."
tar -xvf "blackbox_exporter-${BLACKBOX_VERSION}.linux-amd64.tar.gz"

if id "blackbox" &>/dev/null; then
    echo "User 'blackbox' already exists."
else
    echo "Creating Blackbox user..."
    sudo useradd -rs /bin/false blackbox
fi

sudo mkdir -p "$INSTALL_DIR"

echo "Moving executable file to $BIN_DIR..."
sudo mv blackbox_exporter-${BLACKBOX_VERSION}.linux-amd64/blackbox_exporter $BIN_DIR

echo "Setting ownership for the Blackbox Exporter binary..."
sudo chown blackbox:blackbox $BIN_DIR/blackbox_exporter

sudo mv blackbox_exporter-${BLACKBOX_VERSION}.linux-amd64/* $INSTALL_DIR
sudo chown -R blackbox:blackbox $INSTALL_DIR

echo "Creating systemd service file..."
sudo bash -c "cat <<EOT > $SERVICE_FILE
[Unit]
Description=Blackbox Exporter Service
Wants=network-online.target
After=network-online.target

[Service]
Type=simple
User=blackbox
Group=blackbox
ExecStart=$BIN_DIR/blackbox_exporter \
  --config.file=$INSTALL_DIR/blackbox.yml

Restart=always

[Install]
WantedBy=multi-user.target
EOT"

echo "Enabling and starting Blackbox Exporter service..."
sudo systemctl daemon-reload
sudo systemctl enable blackbox_exporter.service
sudo systemctl start blackbox_exporter.service

echo "Cleaning up..."
rm -rf "blackbox_exporter-${BLACKBOX_VERSION}.linux-amd64.tar.gz"
rm -rf "blackbox_exporter-${BLACKBOX_VERSION}.linux-amd64"

echo "Blackbox Exporter version $BLACKBOX_VERSION installation completed."

echo "Fetching available Grafana versions..."
LATEST_VERSIONS=$(curl -s https://api.github.com/repos/grafana/grafana/releases | \
    jq -r '.[].tag_name' | sed 's/^v//' | sort -V  | tail -n 5)

echo "Last available versions:"
echo "$LATEST_VERSIONS"

# Prompt user for the desired Grafana version
read -p "Enter the Grafana OSS version you want to install (do not include 'v', default is 11.3.0): " GRAFANA_VERSION
GRAFANA_VERSION=${GRAFANA_VERSION:-11.3.0}

echo "Installing prerequisites..."
sudo apt-get install -y apt-transport-https software-properties-common wget

echo "Importing the GPG key..."
sudo mkdir -p /etc/apt/keyrings/
wget -q -O - https://apt.grafana.com/gpg.key | gpg --dearmor | sudo tee /etc/apt/keyrings/grafana.gpg > /dev/null

echo "Adding stable Grafana repository..."
echo "deb [signed-by=/etc/apt/keyrings/grafana.gpg] https://apt.grafana.com stable main" | sudo tee -a /etc/apt/sources.list.d/grafana.list

echo "Adding beta Grafana repository..."
echo "deb [signed-by=/etc/apt/keyrings/grafana.gpg] https://apt.grafana.com beta main" | sudo tee -a /etc/apt/sources.list.d/grafana.list

echo "Updating the list of available packages..."
sudo apt-get update

echo "Installing Grafana OSS version $GRAFANA_VERSION..."
sudo apt-get install -y grafana=$GRAFANA_VERSION

echo "Setting ownership for Grafana directories..."
sudo chown -R grafana:grafana /etc/grafana
sudo chown -R grafana:grafana /var/lib/grafana
sudo chown -R grafana:grafana /var/log/grafana

echo "Starting and enabling Grafana service..."
sudo systemctl daemon-reload
sudo systemctl start grafana-server
sudo systemctl enable grafana-server

echo "Grafana OSS version $GRAFANA_VERSION installation completed."

echo "Starting Node Exporter installation..."

VERSIONS_URL="https://api.github.com/repos/prometheus/node_exporter/releases"
echo "Fetching available Node Exporter versions..."
VERSIONS=$(curl -s $VERSIONS_URL | grep 'tag_name' | cut -d '"' -f 4 | sed 's/^v//' | sort -V | tail -n 5)
if [ -z "$VERSIONS" ]; then
    echo "Failed to fetch version information. Please check your internet connection."
    exit 1
fi
echo "Available versions (last 5):"
echo "$VERSIONS"

# Default version
DEFAULT_VERSION="1.3.1"

# Ask user to choose a version
read -p "Enter the Node Exporter version you want to install (Default: ${DEFAULT_VERSION}): " NODE_EXPORTER_VERSION
NODE_EXPORTER_VERSION=${NODE_EXPORTER_VERSION:-$DEFAULT_VERSION}

NODE_EXPORTER_USER="node_exporter"
INSTALL_DIR="/usr/bin"
SERVICE_FILE="/etc/systemd/system/node_exporter.service"

# Download and extract Node Exporter
echo "Downloading Node Exporter v${NODE_EXPORTER_VERSION}..."
wget https://github.com/prometheus/node_exporter/releases/download/v${NODE_EXPORTER_VERSION}/node_exporter-${NODE_EXPORTER_VERSION}.linux-amd64.tar.gz

echo "Extracting archive..."
tar xvf node_exporter-${NODE_EXPORTER_VERSION}.linux-amd64.tar.gz

# Move binary to install directory
echo "Moving binary to ${INSTALL_DIR}..."
cd node_exporter-${NODE_EXPORTER_VERSION}.linux-amd64
mv node_exporter ${INSTALL_DIR}
cd ..

# Cleanup temporary files
echo "Cleaning up temporary files..."
rm -rf node_exporter-${NODE_EXPORTER_VERSION}.linux-amd64 node_exporter-${NODE_EXPORTER_VERSION}.linux-amd64.tar.gz

# Create user if it doesn't exist
if ! id -u ${NODE_EXPORTER_USER} > /dev/null 2>&1; then
    echo "Creating Node Exporter user..."
    sudo useradd --no-create-home --shell /bin/bash ${NODE_EXPORTER_USER}
fi

# Set permissions
echo "Setting permissions..."
sudo chown ${NODE_EXPORTER_USER}:${NODE_EXPORTER_USER} ${INSTALL_DIR}/node_exporter

# Create systemd service
echo "Creating systemd service..."
echo "[Unit]
Description=Node Exporter
Wants=network-online.target
After=network-online.target

[Service]
User=${NODE_EXPORTER_USER}
Group=${NODE_EXPORTER_USER}
Type=simple
ExecStart=${INSTALL_DIR}/node_exporter

[Install]
WantedBy=multi-user.target" | sudo tee ${SERVICE_FILE}

# Start and enable service
echo "Starting and enabling Node Exporter service..."
sudo systemctl daemon-reload
sudo systemctl start node_exporter
sudo systemctl enable node_exporter

# Installation completed
echo "====================================================="
echo "✅ Node Exporter v${NODE_EXPORTER_VERSION} has been successfully installed and started."
echo "====================================================="

sleep 5
clear
systemctl | grep "prometheus\|blackbox\|grafana\|alertmanager\|node_exporter"

To address this, the provided Bash script automates the end-to-end process of installing Prometheus and its ecosystem. It offers the following:

• Customizable installation: Users can specify versions for Prometheus, Alertmanager, Blackbox Exporter, Grafana, and Node Exporter. Defaults are provided for convenience.
• System service integration: Automatically creates systemd service files for all tools, enabling them to start at boot and be managed consistently.
• User management: Ensures each tool runs under a dedicated, non-privileged user for improved security.
• Cleanup: Removes temporary files post-installation to keep the system organized.
• Data retention configuration: Allows users to set custom retention periods for Prometheus metrics.

Key Features

1. Prometheus Installation: Downloads and installs the selected version, sets up the storage directory, and creates a systemd service file with customizable data retention.
2. Alertmanager Setup: Installs Alertmanager with support for version selection and systemd integration.
3. Blackbox Exporter Installation: Automates the deployment of Blackbox Exporter for endpoint probing, complete with service configuration.
4. Grafana Deployment: Fetches and installs the desired Grafana OSS version, adds repositories, and sets up the service.
5. Node Exporter Installation: Automates the setup of Node Exporter for system metrics collection, including a systemd service for simplified management.

Notes About Solutions

• Architecture Limitation: This script is designed and tested for x86 architecture systems. It is not compatible with ARM-based systems.
• Prerequisites: The script assumes a Linux environment with sudo privileges. Necessary tools such as wget, curl, and jq are installed automatically.
• Service Files: All systemd service files are generated dynamically, ensuring compatibility with user-specified configurations.
• Error Handling: The script uses set -e to terminate on errors, preventing partial or inconsistent installations.
• Security: Each service is run under a dedicated user with restricted permissions to minimize security risks.

Limitations

• The script assumes internet connectivity to fetch packages and binaries from GitHub and official repositories.
• Custom configurations (e.g., Prometheus or Alertmanager YAML files) must be set up manually after installation.

Installation Steps

To run the installation script, follow these steps:

1. Switch to root user: Run the command sudo su to switch to the root user.
2. Create or open the script file: Run vi [filename].sh to create and open the file in the vi editor. Replace [filename] with the desired name of the script (e.g., install_prometheus.sh).
3. Paste the script content: Once inside the vi editor, press i to enter insert mode, and paste the script content into the file. After pasting the script, press Esc and type :wq to save and exit the editor.
4. Make the script executable: Replace [filename] with the name of your script (e.g., install_prometheus.sh). Run the command chmod +x [filename].sh to make the script executable.
5. Run the script: To start the installation process, execute the script by running ./[filename].sh.
6. Follow the on-screen prompts: The script will guide you through the installation process by asking you for the desired versions of Prometheus, Alertmanager, Blackbox Exporter, Grafana, and Node Exporter. It will also
7. handle the installation of necessary dependencies, create directories, configure permissions, and set up system services for each tool.

By following this structured automation approach, teams can deploy Prometheus and its ecosystem efficiently, with minimal manual intervention and consistent configurations. However, please note that this solution is designed specifically for x86 architecture, and users with ARM-based systems will need alternative installation methods.

Further Information

For more details on the individual tools and their configurations, consult the following resources:

• Prometheus Documentation: https://prometheus.io/docs/introduction/overview/
• Alertmanager Documentation: https://prometheus.io/docs/alerting/latest/alertmanager/
• Blackbox Exporter Documentation: https://github.com/prometheus/blackbox_exporter
• Grafana Documentation: https://grafana.com/docs/grafana/latest/
• Node Exporter Documentation: https://github.com/prometheus/node_exporter