Guide to Backstage Kubernetes Integration

Backstage is a popular software project developed by the team at Spotify to enable organizations to build a developer portal that serves as a curated interface into the organization’s entire software stack. Backstage enables developers to utilize a service catalog to discover software, view documentation, and launch infrastructure managed by any number of teams. The unified entry point (a “single pane of glass”) for launching an entire software stack enables teams to effectively collaborate and manage software within the organization.

Backstage natively supports integration with Kubernetes, which is a valuable feature for an organization leveraging Kubernetes cluster infrastructure as its compute platform. Kubernetes integration allows developers to view status information for their software, which is deployed to Kubernetes clusters.

Backstage can query cluster data and provide relevant information (like pod status) directly in the Backstage interface, allowing developers to view their software’s state without the complexity of manually connecting to Kubernetes clusters, managing authentication, configuring the Kubectl command line, etc. Additional plugins also provide functionality for creating clusters directly through Backstage. Improving the developer experience is the primary objective of Backstage, and this article discusses how the project achieves this, with an emphasis on the value of Kubernetes integration.

Summary of key concepts

‍

‍What is Backstage?

Backstage is an open-source software project developed by Spotify to provide a unified entry point for an organization’s software ecosystem, including tools, services managed by multiple teams, deployment pipelines, compute infrastructure, and documentation. Backstage aims to combine disparate elements of an organization’s software stack into a single, cohesive developer portal, making it simple for teams to collaborate and manage their software and infrastructure. These concepts are part of a modern strategy called internal developer platforms (IDPs). Adopting an IDP involves implementing tooling and processes that help developers within an organization simplify and scale their development efforts, leveraging tools that include Backstage.

The problem Backstage solves is one where large organizations have many software teams working in silos without a unified developer experience. In such circumstances, developers need to navigate a maze of tools and infrastructure, rely on disparate sources of documentation, and deal with the cognitive overhead of learning new processes to get their software deployed. All of this slows developer velocity and onboarding while creating duplicate effort due to a lack of central organization. Teams in growing organizations have struggled to provide a single pane of glass to developers due to a lack of tooling and difficulty collaborating using a unified interface, a gap that Backstage is plugging.

Backstage helps organizations bring their teams together to serve developers via a single user journey. Backstage primarily operates as a service catalog, where each team in the organization can integrate its software projects, documentation, API schema, and Git repositories in a single location. A developer can then browse through all the software components available in the organization and understand how to use and deploy them entirely via self-service. Backstage includes plugins for various monitoring tools, so developers can even monitor their application via the Backstage interface.

This user journey is a significant improvement over the traditional approach used by most organizations. In classical arrangements, developers are exposed to many tools hosted with separate interfaces, documentation spread across multiple sources, no centralized deployment pipelines, and multiple monitoring tools.

Backstage integration for Kubernetes

Kubernetes is a container orchestration platform and one of the world’s largest open-source projects. It is the platform of choice for organizations developing and deploying containerized workloads at scale.

Infrastructure teams can use Backstage to allow developers to deploy and monitor software on Kubernetes infrastructure easily, without the cognitive overhead of learning about Kubernetes and cloud provider concepts, tooling, and architectures. Developers typically prefer to focus their attention on application development and business logic while expecting the infrastructure implementation details to be abstracted and transparent to them.

Platforms teams have the objective of abstracting this complexity and reducing their developers’ cognitive overhead, so they can focus more on application development while infrastructure is maintained transparently. Implementing Backstage with Kubernetes is an approach to solving this problem.

Backstage supports easy integration with Kubernetes using plugins, allowing third-party vendors or independent developers to add support and extend functionality. Many third parties, like AWS and Weaveworks, develop plugins for Backstage to enable easier integration with their services.

Kubernetes integration allows developers to view the status of their Kubernetes workloads directly from the Backstage interface.

An example of Kubernetes integration with Backstage is the GitOps-Profiles plugin, which lets Backstage present the developer with a simple cluster creation workflow.

The developer selects a “cluster template” predefined by the organization’s infrastructure team and launches the template through the Backstage form. The infrastructure team will take care of ensuring that the cluster templates contain reasonable defaults suitable for software teams in the organization. Deploying the template can trigger cluster creation workflows managed by the infrastructure team using tools like Uffizzi, GitHub Hooks, ClusterAPI, eksctl, etc.

Developers don’t need to be aware of how the cluster is created in the background; they simply expect the Backstage interface to complete their requests for clusters, so they can begin deploying an application. When they select other software in the Backstage service catalog, they will have cluster infrastructure ready to host their software.

The user journey here is far simpler than expecting the developer to learn and understand the complexities of building clusters. Leveraging Backstage to abstract this complexity also enables infrastructure teams to enforce guardrails and best practices in their cluster templates. An example guardrail may be ensuring that multiple Kubernetes worker nodes are deployed in different availability zones, enabling high availability. By enforcing this configuration in the cluster template defaults, infrastructure teams can ensure that every cluster launched via Backstage follows this best practice.

Going a step further, by integrating the “Backstage Kubernetes” plugin, the project can also query clusters to fetch and display application health data to the developer directly in the Backstage interface. The developer who deployed software through the Backstage service catalog to a Kubernetes cluster can now view the application’s health details (such as pod statuses) without manually installing tools like kubectl, dealing with cluster authentication, or learning how to query their resources.

The user journey above is a significant improvement for developers attempting to quickly get their software up and running. Backstage has enabled them to quickly launch a Kubernetes cluster (with guardrails out of the box), launch a software component from a service catalog to the cluster, and monitor the health of the software automatically. All of this is done entirely through the Backstage interface.

The software and infrastructure teams that own these components are responsible for the implementation details of how the software components are configured and deployed or how the Kubernetes clusters are designed. Those teams can now “vend” their software and infrastructure to developers easily through a unified interface, enabling a complete self-service experience. The teams will have control over the configuration of resources listed in Backstage to ensure that everything follows best practices and uses reasonable defaults, which will help ensure that developers can work independently and spend less time troubleshooting.

Ephemeral environments with Backstage and Uffizzi

Ephemeral environments are a crucial technology for software teams deploying applications to Kubernetes clusters. These are sandbox virtual cluster environments (like staging Kubernetes clusters) where software can be tested and validated prior to production deployments.

As part of the developer’s user journey in Backstage, integrating ephemeral environment capabilities through a simple interface enables developers to quickly test their software and validate production readiness. An example approach to integration includes using GitHub Actions integration to deploy containers to the ephemeral environment and output the container’s deployment status to the pull request. This would enable developers to validate that their containers are building, deploying, and running successfully as part of the code review workload, allowing necessary testing to take place prior to merging changes.

Uffizzi is purpose-built to manage these ephemeral environments, which can be used for remote development or as temporary QA or staging environments.  In Dev mode Uffizzi enables Developers to run individual, isolated remote Development environments where code changes are instantly synced.  This capability helps teams rapidly onboard new developers with a standardized set-up and enables Developers to run production-like environments without burning their laptops.

Uffizzi can operate in a self-hosted configuration where teams have complete control over their infrastructure and data plane or it can operate in a managed configuration for teams that want to outsource the underlying infrastructure.

Uffizzi’s ephemeral environments run as virtual clusters so that Developers can quickly spin them up (under 60s), they are completely isolated from other infrastructure for security and compliance, and they use a fraction of the resources of full Kubernetes clusters.  Uffizzi environments can serve as Dev Clusters, Preview or Pull Request Environments, Automated Test Environments, or any other use case that has a finite time period.

GitHub integration

Platform teams can store their configuration files in a GitHub repository and ensure that the Backstage service catalog component points to the GitHub repository. The platform team can provide the template repository containing out-of-the-box integration with GitHub Actions to connect to Uffizzi’s platform. The result will be that when developers launch software through the Backstage catalog, their repositories will already be configured to utilize Uffizzi environments automatically.

The developers can simply open pull requests to their new repositories with their container images, and Uffizzi will take care of providing build statuses. This enables platform teams to automatically enable Kubernetes workload validation for other developers in the organization while abstracting the initial setup steps to connect the repository with Uffizzi.

This screenshot shows a GitHub pull request with automated tests performed by deploying Kubernetes manifests to Uffizzi’s platform. Developers can easily see the result of the build output, which helps validate manifest changes before merging.

Virtual clusters

Uffizzi supports creating virtual clusters using a command-line tool:

uffizzi cluster create quickstart

Platform teams could include the command line steps with their Backstage catalog components and allow developers to run the tool directly to deploy the Uffizzi virtual clusters. This approach can be helpful for developers who are confident with deploying to a Kubernetes cluster directly. They can now deploy and test their Kubernetes manifests with the newly created virtual cluster hosted on the Uffizzi platform.

Other approaches may also be available depending on what plugins have been installed for Backstage. For example, integrating Uffizzi’s ephemeral environments into a continuous integration pipeline like GitLab CI is possible.

Ephemeral environments are a vital aspect of validating modern containerized applications. Allowing developers to test their Kubernetes workloads quickly in managed sandbox environments is a big step in capturing reliable test results prior to production deployments and accelerating the development process. Uffizzi makes it easy to provide this functionality as a standard part of the developer experience.

Getting started with Backstage for Kubernetes

Backstage is a Node.js application that can be installed quite easily. The package is available via npm and only has some minor dependencies, such as SQLite, Docker, npm, and Git. Installation in a Linux-based environment is as easy as:

npx @backstage/create-app@latest

Full installation steps can be found here.

Once installed, the project can be launched by running:

cd my-backstage-app
yarn dev

Navigating to http://localhost:3000 should display the Backstage interface:

‍

And that’s it! Now you can begin experimenting with the Backstage installation and get familiar with how it works. If you’re trying to deploy a production-ready Backstage setup, you’ll want to configure features like authentication and persistent storage.

It is also possible for Backstage to be deployed to Kubernetes clusters. It’ll run using a Kubernetes Deployment resource with the container image for Backstage and another Deployment resource for a Postgres database that is used for persistent storage. The Backstage endpoint can then be served via a Kubernetes Service or Ingress resource, the same way other cluster web applications receive traffic.

Installing the Backstage Kubernetes plugins

The installation steps for plugins can vary depending on whether the plugin is custom or built-in/native. The following steps explain how the native Kubernetes plugins are installed for Backstage.

There are two plugins required for full Kubernetes integration with Backstage. The “frontend” plugin adds the user interface features, like a “Kubernetes” tab in the software catalog. The “backend” plugin provides the functionality for authenticating and connecting to the Kubernetes clusters.

Installing the frontend plugin requires Yarn commands, just like when installing Backstage:

# From your Backstage root directory
yarn add --cwd packages/app @backstage/plugin-kubernetes

This adds the plugin files into the Backstage installation directory. Now we need to ensure that Backstage loads the new plugin by updating the EntityPage file:

# packages/app/src/components/catalog/EntityPage.tsx
import { EntityKubernetesContent } from '@backstage/plugin-kubernetes';
const serviceEntityPage = (
  <EntityLayout>
	{/* other tabs... */}
	<EntityLayout.Route path="/kubernetes" title="Kubernetes">
  	    <EntityKubernetesContent refreshIntervalMs={30000} />
	</EntityLayout.Route>
  </EntityLayout>
); 

The frontend plugin is now installed and will load the next time the service catalog is refreshed. Now we can install the backend plugin:

# From your Backstage root directory
yarn add --cwd packages/backend @backstage/plugin-kubernetes-backend

Next, we create a new file called kubernetes.ts, which stores some configuration parameters that we can leave as default (copy/paste as-is):

# packages/backend/src/plugins/kubernetes.ts
import { KubernetesBuilder } from '@backstage/plugin-kubernetes-backend';
import { Router } from 'express';
import { PluginEnvironment } from '../types';
import { CatalogClient } from '@backstage/catalog-client';

export default async function createPlugin(
  env: PluginEnvironment,
): Promise {
  const catalogApi = new CatalogClient({ discoveryApi: env.discovery });
  const { router } = await KubernetesBuilder.createBuilder({
	logger: env.logger,
	config: env.config,
	catalogApi,
	permissions: env.permissions,
  }).build();
  return router;
}

Now we just import the backend plugin by updating the index.ts file:

#packages/backend/src/index.ts
// ..
import kubernetes from './plugins/kubernetes';

async function main() {
  // ...
  const kubernetesEnv = useHotMemoize(module, () => createEnv('kubernetes'));
  // ...
  apiRouter.use('/kubernetes', await kubernetes(kubernetesEnv));

The plugins are now ready. The final step in ensuring that the Kubernetes cluster functionality is ready is to supply the cluster configuration details, like the API Server endpoint and authentication details. The root directory of the Backstage installation has an “app-config.yaml” file that contains various environment configuration details, which is where we’ll need to add a new section called “kubernetes”:

kubernetes:
  serviceLocatorMethod:
	type: 'multiTenant'
  clusterLocatorMethods:
	- type: 'config'
  	clusters:
    	- url: http://127.0.0.1:9999
      	name: cluster-1
      	authProvider: 'serviceAccount'
      	skipTLSVerify: false
      	skipMetricsLookup: true
      	serviceAccountToken: ${K8S_MINIKUBE_TOKEN}
      	caData: ${K8S_CONFIG_CA_DATA}
      	caFile: '' # local path to CA file
    	- url: http://127.0.0.2:9999
      	name: cluster-2
      	authProvider: 'aws'

This configuration will connect Backstage to two Kubernetes clusters, one running locally and one running on AWS. Details like authentication data can be added to this configuration file. As we can see above, we can configure Backstage to integrate with more than one cluster, allowing developers to deploy software to multiple clusters.

These steps only need to be completed once, and now Kubernetes integration will be available for any software components stored in the Backstage catalog.

Conclusion

Backstage provides a strong foundation to unify the development efforts of software teams in large organizations. The ability to implement a single pane of glass for developers to deploy software and infrastructure entirely through self-service is a powerful step in the field of platform engineering, boosting productivity and lowering the cognitive load placed on developers.

The integration with Kubernetes is a great feature that allows infrastructure teams to supply Kubernetes clusters for developers to run software without burdening developers with the cognitive overhead of becoming Kubernetes experts. The abstraction made possible with Backstage allows developers to deploy containerized applications seamlessly without the operational overhead of creating clusters, setting up local tools, authenticating, etc. Everything happens through the Backstage interface, which allows infrastructure teams to provide guardrails and best practices built into the cluster usage experience.

Extending Backstage’s capabilities further by integrating ephemeral environments is another step towards full self-service capabilities. Developers can now utilize software that comes with automatic integration with the Uffizzi platform, allowing developers to quickly validate their Kubernetes manifests when raising pull requests and receiving feedback.

The overall benefits provided by Backstage are huge for organizations that want to facilitate better collaboration between teams, reduce cognitive overhead for developers, and accelerate the velocity of application deployments. You may want to experiment with Backstage using the setup steps above to determine if it’s right for your use case.  To rapidly get a demo environment for deploying Backstage you can spin up a Uffizzi Virtual Cluster for free on their Starter Tier.