We continue our blog series about Prometheus, today addressing a particularly interesting but often overlooked topic: Prometheus Instrumentation.

What does that mean? I’m not telling you anything new when I say that Prometheus allows you to collect tons of metrics, assess the health of your systems, and keep an eye on everything that’s happening, thanks to a wide range of official and unofficial exporters available to you.

However, for some reason, the available exporters may not provide you with the specific metrics you need for your application. This is where Prometheus libraries come into play.

A Prometheus library provides you with a straightforward way to add instruments to your application code to track and expose metrics in Prometheus. These libraries allow you to track and expose metrics in the expected Prometheus format, similar to an exporter.

Prometheus has several official client libraries written in Go, Java, Python, Ruby, and Rust. Additionally, there are unofficial third-party client libraries available for other languages. Moreover, if desired, you can even write your own client library for any unsupported language.

Throughout this blog, I will explain how to instrument a Python application and collect metrics from an API.

Installation and starting of our application

For the purpose of this blog, I will create an API using a web framework called Flask, which is a popular web framework in Python that allows for relatively easy API development. I will need to have Python and Flask installed on my system.

I have written a small media management script that allows me to list, modify, add, and delete content.

My application consists of two files: my Python script called app.py and an index.html file that serves as an HTML render to make our demonstrations more intuitive.

├── app.py
└── templates
    └── index.html

2 directories, 2 files

The app.py script is a basic Flask application that provides a RESTful API for managing a collection of books.
I’ve set up the basic routes and functions for managing books through HTTP requests, allowing users to get all books, create a new book, update an existing book, and delete a book.

from flask import Flask, jsonify, request, render_template, redirect, url_for

app = Flask(__name__)

# Sample initial book data
books = [
    {"id": 1, "title": "The Expanse", "novel_title": "Leviathan Wakes", "author": "James S. A. Corey", "publisher": "Orbit Book"},
    {"id": 2, "title": "The Expanse", "novel_title": "Caliban's War", "author": "James S. A. Corey", "publisher": "Orbit Book"},
    {"id": 3, "title": "The Expanse", "novel_title": "Abaddon's Gate", "author": "James S. A. Corey", "publisher": "Orbit Book"},
    {"id": 4, "title": "The Expanse", "novel_title": "Cibola Burn", "author": "James S. A. Corey", "publisher": "Orbit Book"},
    {"id": 5, "title": "The Expanse", "novel_title": "Nemesis Games", "author": "James S. A. Corey", "publisher": "Orbit Book"},
    {"id": 6, "title": "The Expanse", "novel_title": "Babylon's Ashes", "author": "James S. A. Corey", "publisher": "Orbit Book"},
]

# Welcome route for the root URL ("/")
@app.route('/')
def home():
    return render_template('index.html')

# Get all books
@app.route('/books', methods=['GET'])
def get_books():
    return jsonify(books)

# Create a new book
@app.route('/books', methods=['POST'])
def create_book():
    new_book = {
        'id': len(books) + 1,
        'title': request.form['title'],
        'author': request.form['author'],
        'publisher': request.form['publisher'],
        'novel_title': request.form['novel_title']
    }
    books.append(new_book)
    return redirect(url_for('home'))

# Update a book
@app.route('/books/<int:book_id>', methods=['PUT'])
def update_book(book_id):
    book = next((book for book in books if book['id'] == book_id), None)
    if book:
        book['title'] = request.form.get('title', book['title'])
        book['novel_title'] = request.form.get('novel_title', book['novel_title'])
        book['author'] = request.form.get('author', book['author'])
        book['publisher'] = request.form.get('publisher', book['publisher'])
        return jsonify(book)
    return jsonify({"message": "Book not found"}), 404

# Delete a book
@app.route('/books/<int:book_id>', methods=['DELETE'])
def delete_book(book_id):
    book = next((book for book in books if book['id'] == book_id), None)
    if book:
        books.remove(book)
        return jsonify({"message": "Book deleted"})
    return jsonify({"message": "Book not found"}), 404

if __name__ == '__main__':
    app.run(host='0.0.0.0', port=8080)

The index.html file will be used to create a user interface for managing books.

<!DOCTYPE html>
<html>
<head>
    <title>Book Management System</title>
    <style>
        .button-container {
            display: flex;
            gap: 10px;
            margin-bottom: 10px;
        }
    </style>
</head>
<body>
    <h1>Blog: Instrument your python application - Book Management System</h1>
    <form action="/books" method="POST">
        <label for="title">Title:</label>
        <input type="text" id="title" name="title">
        <label for="novel_title">Novel Title:</label>
        <input type="text" id="novel_title" name="novel_title">
        <label for="author">Author:</label>
        <input type="text" id="author" name="author">
        <label for="publisher">Publisher:</label>
        <input type="text" id="publisher" name="publisher">
        <button type="submit">Add Book</button>
    </form>
    <br>
    <div class="button-container">
        <button onclick="listBooks()">List Books</button>
    </div>
    <div id="bookList"></div>

    <script>
        function listBooks() {
            fetch('/books')
                .then(response => response.json())
                .then(data => {
                    const bookList = document.getElementById('bookList');
                    bookList.innerHTML = '';

                    data.forEach(book => {
                        const bookInfo = document.createElement('p');
                        bookInfo.textContent = `Book ID: ${book.id}, Title: ${book.title}, Novel Title: ${book.novel_title}, Author: ${book.author}, Publisher: ${book.publisher}`;

                        const updateButton = document.createElement('button');
                        updateButton.textContent = 'Update Book';
                        updateButton.onclick = () => updateBook(book.id);

                        const deleteButton = document.createElement('button');
                        deleteButton.textContent = 'Delete Book';
                        deleteButton.onclick = () => deleteBook(book.id);

                        bookInfo.appendChild(updateButton);
                        bookInfo.appendChild(deleteButton);
                        bookList.appendChild(bookInfo);
                    });
                });
        }

        function updateBook(bookId) {
            const title = prompt('Enter the new title:');
            const author = prompt('Enter the new author:');
            const publisher = prompt('Enter the new publisher:');
            const novelTitle = prompt('Enter the new novel title:');

            fetch(`/books/${bookId}`, {
                method: 'PUT',
                headers: {
                    'Content-Type': 'application/x-www-form-urlencoded'
                },
                body: `title=${encodeURIComponent(title)}&author=${encodeURIComponent(author)}&publisher=${encodeURIComponent(publisher)}&novel_title=${encodeURIComponent(novelTitle)}`
            })
            .then(response => {
                if (response.ok) {
                    alert('Book updated successfully');
                    listBooks();
                } else {
                    alert('Failed to update book');
                }
            });
        }

        function deleteBook(bookId) {
            fetch(`/books/${bookId}`, {
                method: 'DELETE'
            })
            .then(response => {
                if (response.ok) {
                    alert('Book deleted successfully');
                    listBooks();
                } else {
                    alert('Failed to delete book');
                }
            });
        }
    </script>
</body>
</html>

First, we will start the application, which can be done by executing the Python script as below:

python3 app.py

➜ python3 app.py

 * Serving Flask app 'app'
 * Debug mode: off
WARNING: This is a development server. Do not use it in a production deployment. Use a production WSGI server instead.
 * Running on all addresses (0.0.0.0)
 * Running on http://127.0.0.1:8080
 * Running on http://10.*.*.*:8080
Press CTRL+C to quit

How does it work?

Once the application is started, you simply need to open a web page and enter the URL of your application, in our case it will be a local URL like http://127.0.0.1:8080, the application should present itself as shown in the following picture:

This application will allow us to generate HTTP traffic using different types of request methods, simply by utilizing the application’s features.


To start, click on the “List Books” button, which corresponds to the “GET” request for the “/book” route and will return a list of books from “The Expanse.”

As a side note, “The Expanse” is a series of science fiction novels set in a future where humanity explores and colonizes the solar system, following a group of characters involved in a conspiracy that jeopardizes the balance of power between planets and factions. I highly recommend it to science fiction enthusiasts.

Now, let’s focus on our application.
Each button corresponds to a METHOD of request, this means that the:

  • List Books corresponds to the GET request
  • Add Book corresponds to the POST request
  • Update Book corresponds to the PUT request
  • Delete Book corresponds to the DELETE request

If I translate this into an image, it would show the list of books when I click on “List Books” (GET method).

127.0.0.1 - - [07/Jul/2023 11:38:24] "GET /books HTTP/1.1" 200 -

If I click on the “Delete Book” button for “Book ID: 6” (Babylon’s Ashes), the entry will be deleted (DELETE method).

If I fill in the empty fields to enter a new book, a new entry will be added (POST method).

127.0.0.1 - - [07/Jul/2023 11:50:14] "POST /books HTTP/1.1" 302 -
127.0.0.1 - - [07/Jul/2023 11:50:14] "GET / HTTP/1.1" 200 -


Please note that I intentionally introduce a mistake by switching the Author and Publisher fields.
To correct this error, all we need to do is edit the incorrect entry by clicking on the update button and re-enter the information in the appropriate corresponding fields (PUT Method).

127.0.0.1 - - [07/Jul/2023 11:58:35] "PUT /books/6 HTTP/1.1" 200 -
127.0.0.1 - - [07/Jul/2023 11:58:37] "GET /books HTTP/1.1" 200 -

We have seen how our application works and how to initiate different HTTP requests.
Now, it’s time to focus on the part that interests us, the instrumentation of our application, as so far we haven’t implemented anything related to Prometheus.

Installation and configuration of metrics.


Let’s start by installing the Prometheus client. To do this, we will execute the command pip install prometheus-client

pip install prometheus_client

This command will allow us to install the library so that we can access it in our Python code.

Next, we need to import classes from the Prometheus client library that we are interested in.

There are several classes available from the Prometheus client library, such as Counter, Gauge, Histogram, etc. We will start by importing the Counter class from Prometheus with the intention of implementing a counter-type metric.

from prometheus_client import Counter

Then, in our code, we will define a variable called REQUESTS to initialize the Counter object by providing the metric name (http_request_total) and its description (Total number of requests).

REQUESTS = Counter('http_request_total',
                   'Total number of requests')

The script should look like:

from flask import Flask, jsonify, request, render_template, redirect, url_for

from prometheus_client import Counter

REQUESTS = Counter('http_request_total',
                   'Total number of requests')

app = Flask(__name__)

(...)

Now that we have this counter object, the question remains: When do we want to increment our counter?

Since this counter represents the total number of requests, we need to increment it whenever we receive a request (GET / PUT / POST / DELETE).

In our code, requests are made to the endpoint :
@app.route(‘/books’, methods=[‘GET’|’POST’|’PUT’|’DELETE’]
so we will place our counter increment REQUESTS.inc() every time our “/books” route is called.

# Get all books
@app.route('/books', methods=['GET'])
def get_books():
    REQUESTS.inc()
    return jsonify(books)

# Create a new book
@app.route('/books', methods=['POST'])
def create_book():
    REQUESTS.inc()

# Update a book
@app.route('/books/<int:book_id>', methods=['PUT'])
def update_book(book_id):
    REQUESTS.inc()

# Delete a book
@app.route('/books/<int:book_id>', methods=['DELETE'])
def delete_book(book_id):
    REQUESTS.inc()

The incrementation method “.inc()” will increase the counter by one unit, but keep in mind that the increment value is customizable, and you can define it within the parentheses if incrementing by one unit does not suit your needs.
Your code should look like this:

from flask import Flask, jsonify, request, render_template, redirect, url_for

from prometheus_client import Counter

REQUESTS = Counter('http_request_total',
                   'Total number of requests')


app = Flask(__name__)

# Sample initial book data
books = [
    {"id": 1, "title": "The Expanse", "novel_title": "Leviathan Wakes", "author": "James S. A. Corey", "publisher": "Orbit Book"},
    {"id": 2, "title": "The Expanse", "novel_title": "Caliban's War", "author": "James S. A. Corey", "publisher": "Orbit Book"},
    {"id": 3, "title": "The Expanse", "novel_title": "Abaddon's Gate", "author": "James S. A. Corey", "publisher": "Orbit Book"},
    {"id": 4, "title": "The Expanse", "novel_title": "Cibola Burn", "author": "James S. A. Corey", "publisher": "Orbit Book"},
    {"id": 5, "title": "The Expanse", "novel_title": "Nemesis Games", "author": "James S. A. Corey", "publisher": "Orbit Book"},
    {"id": 6, "title": "The Expanse", "novel_title": "Babylon's Ashes", "author": "James S. A. Corey", "publisher": "Orbit Book"},
]

# Welcome route for the root URL ("/")
@app.route('/')
def home():
    return render_template('index.html')

# Get all books
@app.route('/books', methods=['GET'])
def get_books():
    REQUESTS.inc()
    return jsonify(books)

# Create a new book
@app.route('/books', methods=['POST'])
def create_book():
    REQUESTS.inc()
    new_book = {
        'id': len(books) + 1,
        'title': request.form['title'],
        'author': request.form['author'],
        'publisher': request.form['publisher'],
        'novel_title': request.form['novel_title']
    }
    books.append(new_book)
    return redirect(url_for('home'))

# Update a book
@app.route('/books/<int:book_id>', methods=['PUT'])
def update_book(book_id):
    REQUESTS.inc()
    book = next((book for book in books if book['id'] == book_id), None)
    if book:
        book['title'] = request.form.get('title', book['title'])
        book['novel_title'] = request.form.get('novel_title', book['novel_title'])
        book['author'] = request.form.get('author', book['author'])
        book['publisher'] = request.form.get('publisher', book['publisher'])
        return jsonify(book)
    return jsonify({"message": "Book not found"}), 404

# Delete a book
@app.route('/books/<int:book_id>', methods=['DELETE'])
def delete_book(book_id):
    REQUESTS.inc()
    book = next((book for book in books if book['id'] == book_id), None)
    if book:
        books.remove(book)
        return jsonify({"message": "Book deleted"})
    return jsonify({"message": "Book not found"}), 404

if __name__ == '__main__':
    app.run(host='0.0.0.0', port=8080)

The configuration of our metric is in place, now we need to make sure it is accessible.

So, how do we proceed? We simply call an HTTP service that will start a web server and expose all our metrics. To do this, we first need to import the start_http_server class from your Prometheus-Client library, and then start this server on an unused port of our choice.

from flask import Flask, jsonify, request, render_template, redirect, url_for

from prometheus_client import Counter, start_http_server

REQUESTS = Counter('http_request_total',
                   'Total number of requests')

(...)

if __name__ == '__main__':
    start_http_server(9090)
    app.run(host='0.0.0.0', port=8080)


In our example, the metrics will be exposed on port 9090. Once you have updated your script and restarted your application, you can curl “curl http://127.0.0.1:9090” your endpoint or simply access to the endpoint through a webrowser as below

et’s test our counter, but before that, please note the value of our counter http_request_total in the previous screenshot, which is currently zero. Let’s proceed with a GET request to the API by clicking on the “List books” button.

Let’s check the value of our counter and we can see that it has been incremented by 1.


Now, if I delete the last 3 books (“Book ID: 4 | 5 | 6”), it corresponds to 3 DELETE requests and 3 GET requests to update the display after each deletion.

127.0.0.1 - - [07/Jul/2023 14:22:44] "DELETE /books/6 HTTP/1.1" 200 -
127.0.0.1 - - [07/Jul/2023 14:22:46] "GET /books HTTP/1.1" 200 -
127.0.0.1 - - [07/Jul/2023 14:22:47] "DELETE /books/5 HTTP/1.1" 200 -
127.0.0.1 - - [07/Jul/2023 14:22:48] "GET /books HTTP/1.1" 200 -
127.0.0.1 - - [07/Jul/2023 14:22:49] "DELETE /books/4 HTTP/1.1" 200 -
127.0.0.1 - - [07/Jul/2023 14:22:50] "GET /books HTTP/1.1" 200 -

If I check my counter from my endpoint, I can see that my counter http_request_total shows “7” total requests, which corresponds to the initial 1 request from the first test, plus the 6 requests we just performed.

Congratulations, you have just implemented your first metric in your application! I invite you to join me in part 2 of this blog, where I will show you how to use labels in your metrics. We will also take the opportunity to implement gauge and histogram-type metrics.

See you soon!

Post Views: 2,404
Thumbnail [60x60]
by
Nabil Laabid