Integrating Uffizzi Ephemeral Environments with your Github Actions Pipeline

‍

Introduction

Brief overview of Uffizzi and Github Actions

Overview of Uffizzi

Uffizzi helps developers create and manage ephemeral environments which can be used for testing and previewing your application. It simplifies the process of setting up dynamic ephemeral environments while reducing the time and resources needed to verify and release changes. To learn more about these environments check out Uffizzi’s preview environment guide where we dive deep in one of the most popular use cases Uffizzi’s ephemeral environments.

‍

Overview of Github Actions

The most popular and easily accessible CI/CD system: GitHub Actions (GHA), is a continuous integration and delivery system that enables developers to build, test, and deploy code directly from their GitHub repositories. To learn more, check the GitHub Actions documentation. We will see how Uffizzi can be integrated with GHA in a few simple steps.

‍

Extending Github Actions with Uffizzi to create ephemeral environments

By integrating Uffizzi with GitHub Actions, teams can seamlessly create ephemeral environments for every code change, ensuring that each modification is tested and validated before being deployed to production.

‍

Easing the path to integrate ephemeral environments with existing workflows

Uffizzi’s goal is to make ephemeral environments as approachable and easy to use as possible. Integrating with github actions helps cover a majority of the developer user base which would benefit from Uffizzi. The user doesn’t have to login to create an account if they are setting Uffizzi up for their open source project. We have made it as easy as possible for you to set up Uffizzi with your existing github actions pipeline. We will be following along the quickstart repo to aid us in understanding the process.

‍

The two GHA integration methods based on the kind of project: Internal or Open Source

On our way to ease creation of ephemeral environments, we created the reusable workflow which is provided by the Uffizzi preview-action github action. We will be using this github action which will aid us in extending our workflow for the preview environment.

‍

Understanding the application and configuring our ephemeral environment

Prerequisites

Uffizzi runs on top of Kubernetes but the end user doesn’t have to bother with all that. If the application has a preconfigured Dockerfile and docker-compose ready to go with it then configuring Uffizzi is going to be straightforward. If your application doesn’t already have a Dockerfile and/or a docker-compose config we will be discussing a little bit about that in this blog, but these two configurations are prerequisites for Uffizzi.

‍

Structure of the application

We are going to set up ephemeral environments for a simple voting application which has the following architecture

‍

As you can see above, there are 6 moving parts at play here. Each of the 6 components are containers here and they are communicating via the container networking present in the Pod. Let’s go step by step to understand what each of the components do via the user story we have here.

‍

1. When the request comes to the ephemeral environment created, the loadbalancer (nginx) by default routes it to / but in this case if you are a voter you would be going to the UFFIZZI_URL/vote.
2. Here the vote app is a python frontend web app. Here you will be able to vote for your favourite kind of pet. Cat or Dog.
3. The python app collects all the votes and sends them out to a redis cache. Considering a lot of people will be voting all at once we cannot directly add them all to the database, instead we employ a worker which will collect all of these votes.
4. This worker collects all the votes from the redis cache and updates the database with the votes.
5. The postgres database is the main source of truth for the result app which showcases the state of the poll.
6. If the user goes to the UFFIZZI_URL directly they would be able to see which candidates and their allocated votes.

Now we know how the application works, let’s take a peek at the Uffizzi configuration derived from the docker-compose configuration.

‍

Uffizzi Configuration

‍

# Uffizzi configuration (required)
x-uffizzi:
  ingress:
    service: loadbalancer
    port: 8080

# Application
services:
  redis:
    image: redis:latest

  postgres:
    image: postgres:9.6
    environment:
    # Add your credentials as GitHub Actions Secrets, then use the commented syntax below:
    POSTGRES_USER: postgres # "${PGUSER}"
    POSTGRES_PASSWORD: postgres # "${PGPASSWORD}"
    deploy:
      resources:
        limits:
          memory: 500M

  worker:
    image: "${WORKER_IMAGE}"
    environment:
    # Add your credentials as GitHub Actions Secrets, then use the commented syntax below:
      PGUSER: postgres # "${PGUSER}"
      PGPASSWORD: postgres # "${PGPASSWORD}"
    deploy:
      resources:
        limits:
          memory: 250M

  result:
    image: "${RESULT_IMAGE}"
    environment:
    # Add your credentials as GitHub Actions Secrets, then use the commented syntax below:
      PGUSER: postgres # "${PGUSER}"
      PGDATABASE: postgres # "${PGUSER}"
      PGPASSWORD: postgres # "${PGPASSWORD}"
      PORT: 8088

  vote:
    image: "${VOTE_IMAGE}"
    deploy:
      resources:
        limits:
          memory: 250M
      environment:
        PORT: 8888

  loadbalancer:
    image: "${LOADBALANCER_IMAGE}"
    environment:
      PORT: 8080
      VOTE_HOST: "localhost"
      VOTE_PORT: "8888"
      RESULT_HOST: "localhost"
      RESULT_PORT: "8088"
    deploy:
      resources:
        limits:
          memory: 125M

‍

The above Uffizzi configuration has been derived from a docker-compose configuration for the same voting application. All these containers will be created in the same pod and have access to the same pod network. So seeing to it that all of them are exposed on unique ports is important.

‍

The x-uffizzi configuration above is the most important here. A uffizzi configuration is derived from a docker-compose so adding this configuration is important to let Uffizzi know that this configuration is indeed meant to be used with Uffizzi. Here we describe the ingress service and port which would be accessed. The loadbalancer will be taking care of all the requests coming into the ephemeral environment and then routing it as necessary.

‍

In the above configuration we can see that the result, vote and worker images have placeholders. These placeholders will be replaced once the images are built using github actions and are then ready to be consumed. The postgres credentials are populated from github actions when creating the ephemeral environment.

‍

Now that we have set up the configuration for our ephemeral environment, let’s see how we can use this with Uffizzi.

‍

Automating ephemeral environment creation via Github Actions

Now that we have finalised our Uffizzi configuration, let’s use it to create ephemeral environments for our application on every pull request. Let’s consider two scenarios:

‍

1. Internal Project : The project and environments are going to be used by an internal team who has access to the github repository and has access to run workflows on the repo.
2. Open Source Project : The project is open sourced and multiple users internal and external contribute to the project.

‍

For each of the two above, the configuration changes based on the access control to the project. Setting up Uffizzi + Github Actions using the reusable workflow is straightforward for internal projects as execution rights are only in the hands of project maintainers and developers. This is a default Github Actions behavior. For us to configure Uffizzi + Github Actions for an open source project, we have to use the two stage workflow. We will briefly talk about how it works and how you can configure it for use in the subsequent sections. First, let’s look at how we can extend Github Actions to use Uffizz for an internal project using the reusable workflow.

‍

The Reusable Workflow : Uffizzi + Github Actions for Internal Projects

The quickstart repo already has github actions set up to use the reusable workflow. To understand the pipeline  let’s look at uffizzi-preview.yaml file where the necessary images are being built and then pushed to a container registry. Here we can see 4 images being built and then pushed to the Uffizzi Ephemeral Image registry. As all the build jobs follow the same pattern, let’s look at a single job to understand what is happening in the build job:

‍

build-result:
  name: Build and Push `result`
  runs-on: ubuntu-latest
  if: ${{ github.event_name != 'pull_request' || github.event.action != 'closed' }}
  outputs:
    tags: ${{ steps.meta.outputs.tags }}
  steps:
  - name: Checkout git repo
    uses: actions/checkout@v3
  - name: Generate UUID image name
    id: uuid
    run: echo "UUID_RESULT=$(uuidgen)" >> $GITHUB_ENV
  - name: Docker metadata
    id: meta
    uses: docker/metadata-action@v4
    with:
    # An anonymous, emphemeral registry built on ttl.sh
      images: registry.uffizzi.com/${{ env.UUID_RESULT }}
      tags: type=raw,value=24h
  - name: Build and Push Image to Uffizzi Ephemeral Registry
    uses: docker/build-push-action@v3
    with:
      push: true
      tags: ${{ steps.meta.outputs.tags }}
      labels: ${{ steps.meta.outputs.labels }}
      context: ./result

‍

In the above build job first we are going to checkout the code base, then create a UUID for the image. The UUID is important as it helps to obfuscate information regarding which repo or project was the image built for. This image will be pushed to the registry and the final step. This job outputs the tag of the image which has been built and pushed. This will be helpful in the deploy stage to identify the image to be used for the preview.

‍

Once the images are built, the placeholders in the configuration are replaced with actual values and then the ephemeral environment deploy job is triggered. The following is what the relevant jobs looks like:

‍

render-compose-file:
  name: Render Docker Compose File
  runs-on: ubuntu-latest
  needs:
  - build-vote
  - build-worker
  - build-result
  - build-loadbalancer
  outputs:
    compose-file-cache-key: ${{ env.COMPOSE_FILE_HASH }}
    compose-file-cache-path: docker-compose.rendered.yml
  steps:
  - name: Checkout git repo
    uses: actions/checkout@v3
  - name: Render Compose File
    run: |
    VOTE_IMAGE=$(echo ${{ needs.build-vote.outputs.tags }})
    export VOTE_IMAGE
    WORKER_IMAGE=$(echo ${{ needs.build-worker.outputs.tags }})
    export WORKER_IMAGE
    RESULT_IMAGE=$(echo ${{ needs.build-result.outputs.tags }})
    export RESULT_IMAGE
    LOADBALANCER_IMAGE=$(echo ${{ needs.build-loadbalancer.outputs.tags }})
    export LOADBALANCER_IMAGE
    PGUSER=${{ secrets.PGUSER }}
    export PGUSER
    PGPASSWORD=${{ secrets.PGPASSWORD }}
    export PGPASSWORD
    # Render simple template from environment variables.
    envsubst < docker-compose.uffizzi.yml > docker-compose.rendered.yml
    cat docker-compose.rendered.yml
  - name: Hash Rendered Compose File
    id: hash
    run: echo "COMPOSE_FILE_HASH=$(md5sum docker-compose.rendered.yml | awk '{ print $1 }')" >> $GITHUB_ENV
  - name: Cache Rendered Compose File
    uses: actions/cache@v3
    with:
      path: docker-compose.rendered.yml
      key: ${{ env.COMPOSE_FILE_HASH }}

deploy-uffizzi-preview:
  name: Use Remote Workflow to Preview on Uffizzi
  needs: render-compose-file
  uses: UffizziCloud/preview-action/.github/workflows/reusable.yaml@v2
  with:
    compose-file-cache-key: ${{ needs.render-compose-file.outputs.compose-file-cache-key }}
    compose-file-cache-path: ${{ needs.render-compose-file.outputs.compose-file-cache-path }}
    server: https://app.uffizzi.com
    permissions:
    contents: read
    pull-requests: write
    id-token: write

‍

Above you there are two jobs which together help deploy the ephemeral environment:

• render-compose-file : This job which involves rendering the final compose file job depends on all the previous build jobs. It takes in the outputs (the image tags) from the jobs and replaces the placeholders in the Uffizzi configuration to use the latest changes from the
• deploy-uffizzi-preview : Once the final uffizzi configuration is created, it is used in this job to deploy the ephemeral environment

‍

When setting up this reusable workflow for your own project, you will have to update the docker build based on the location of your dockerfile and adjust the number of jobs based on the number of images you need for your compose file.

‍

The reusable workflow in this setup works great for your internal projects where the only people accessing and/or contributing to the project are the ones who have permissions to execute github workflows and create pull requests. If there are outside contributors on this project they would not be able to trigger creation of ephemeral environments as they would not have the necessary credentials to do so. Hence, using just one workflow would mean that every outside contributor to your project would need a Uffizzi account before they could preview their pull request, or else maintainers would need to share credentials with contributors. A two-stage workflow solves this problem by first building the application in the context of the contributor's head branch, then delegating responsibility to the target base branch in the second stage to authenticate with Uffizzi.

‍

The Two-Stage Workflow : Uffizzi + Github Actions for Open Source Project

‍

The reusable workflow can be extended to work in a two stage mode to allow contributors to create pull requests and consume ephemeral environments without needing credentials or accounts on Uffizzi if they are new uses on Uffizzi itself.

‍

The workflow in this case is divided between the build and deploy part where the build part is executed in context of the pull request branch to build the relevant image. The deploy part of the repo runs in context of the main repo. It is triggered when the build is completed. If you are interested in the motivation and working of the two stage workflow, this blog post dives deeper in the same. Let’s look at the triggers for each to understand their relationship, let’s take meilisearch as an example where this has been implemented:

‍

name: Uffizzi - Build PR Image
on:
  pull_request:
    types: [opened,synchronize,reopened,closed]

‍

name: Uffizzi - Deploy Preview
on:
  workflow_run:
    workflows:
      - "Uffizzi - Build PR Image"
  types:
    - completed

‍

In the above excerpts from the two stage workflow files we can see that there is a natural trigger for the uffizzi-build.yml. The build workflow is triggered when a pull request is opened or updated. In the uffizzi-preview-deploy.yml we can see that the workflow is triggered only when the first workflow completes.

‍

Open Source projects using Uffizzi in the wild

Uffizzi has been used in multiple open source projects across github with Github Actions. These projects are all listed here.

‍

Next Steps : getting creative with using ephemeral environments

Once you know how to set up a github actions workflow to trigger ephemeral environments, you can now start evolving them as per your needs. The following are a few ways you can start getting creative with your Ephemeral Environments.

‍

• Running integration tests against Ephemeral Environments : Once the ephemeral environments are up, and if the project uses playwright or selenium for end to end or integration testing, the final UFFIZZI_URL can be passed to the test suite to run tests against a running ephemeral environment instance. This would allow the project to have integration tests to run on every PR.
• Interacting with your Ephemeral Environment using Webterminal : If the application has a CLI environment which would be useful to have access to, the user can build their application with a webterminal installed and then access every ephemeral environment instance using a webterminal.

‍

If you have a project for which you would like to  build ephemeral environments for and discuss the different ways you can use them to better fit your needs, we look forward to speaking with you. You can get in touch with us by going to https://www.uffizzi.com/contact and we will reach out to you !

‍