Managing ExpressJs API Kubernetes Deployment with Flux

Flux is a declarative system for managing Kubernetes deployments. This tutorial will guide you through deploying an ExpressJS application to Azure Kubernetes Service (AKS) using Flux, along with managing resources with Terraform. I have created a repository for reference to help follow along with the tutorial.

Prerequisites

• Review my previous tutorial on deploying an Express.js API to AKS.

• Install K3D for a local cluster for testing purposes.

• Install Flux

• Install Terraform CLI

• Pull Latest Project Image

Create Cluster

Startup a fresh AKS and Azure Container Registry (ACR) deployment. We will utilize Terraform files to declaratively deploy these resources to Azure using Terraform CLI .

Create a

.tf

terraform {
  required_version = ">=1.0"

  required_providers {
    azapi = {
      source  = "azure/azapi"
      version = "~>1.5"
    }
    azurerm = {
      source  = "hashicorp/azurerm"
      version = "~>3.0"
    }
    time = {
      source  = "hashicorp/time"
      version = "0.9.1"
    }
  }
}

provider "azurerm" {
  features {}
}

resource "azurerm_resource_group" "rg" {
  name     = "KubernetesTest"
  location = var.location
}

# Container Registry
resource "azurerm_container_registry" "acr" {
  name                = var.container_registry_name
  resource_group_name = azurerm_resource_group.rg.name
  location            = var.location
  sku                 = "Basic"
  admin_enabled       = false

  identity {
    type = "SystemAssigned"
  }

  tags = {
    Environment = "dev"
  }
}

resource "azurerm_kubernetes_cluster" "k8s" {
  name                = var.azurerm_kubernetes_cluster_dns_name
  location            = "canadacentral"
  resource_group_name = azurerm_resource_group.rg.name
  dns_prefix          = var.azurerm_kubernetes_cluster_dns_prefix

  identity {
    type = "SystemAssigned"
  }

  default_node_pool {
    name       = "agentpool"
    vm_size    = "Standard_A2_v2"
    node_count = var.node_count
  }

  tags = {
    Environment = "dev"
  }
}

Deploy resources to Azure

terraform apply

Create new role assignment to give AKS AcrPull access.

az aks show --resource-group KubernetesTest --name devdeveloper-aks-cluster --query "identityProfile.kubeletidentity.objectId" -o tsv

az role assignment create --assignee ed6b1c8a-ca84-4041-9d87-da92420c8565 --role "AcrPull" --scope /subscriptions/59966e90-8185-44af-a00c-13bc237e59cb/resourceGroups/KubernetesTest/providers/Microsoft.ContainerRegistry/registries/devdeveloperregistry

Register aks deployment with Kubectl

az aks get-credentials --resource-group KubernetesTest --name devdeveloper-aks-cluster

Setup Azure Devops Pipeline

Now we need to set up the Azure DevOps Pipeline. As mentioned before, this is a similar CI/CD pipeline to the previous tutorial but with the deployment step to AKS removed. Flux will be handling that going forward. This YAML config will be automatically generated by the Azure Pipelines web application and you'll only have to make slight modifications based on your setup. Take a look inside the code snippet below for comments.

trigger:
  - version_2

resources:
  - repo: self

variables:
  # Container registry service connection established during pipeline creation
  dockerRegistryServiceConnection: "766a7075-56b5-40d1-8d79-2d89a8710f11"  <=== automatically generated
  imageRepository: "node-ts-api"                                           <=== this is going to be the name of the image repository and will be refered to later
  containerRegistry: "devdeveloperregistry.azurecr.io"                     <=== make sure that the container registry matches the resource in Azure
  dockerfilePath: "**/Dockerfile"                                          <=== make sure that your docker file matches the one inside the repository
  tag: "$(Build.BuildId)"                                                  <=== this is a tag which Flux will refer to when changes are made to the image.
  imagePullSecret: "devdeveloperregistry41003ccd-auth"

  # Agent VM image name
  poolName: "Personal Laptop"                                              <=== this is the agent we will use for building the image artifacts.

stages:
  - stage: Build
    displayName: Build stage
    jobs:
      - job: Build
        displayName: Build
        pool:
          name: $(poolName)
        steps:
          - task: Docker@2
            displayName: Build and push an image to container registry
            inputs:
              command: buildAndPush
              repository: $(imageRepository)
              dockerfile: $(dockerfilePath)
              containerRegistry: $(dockerRegistryServiceConnection)
              tags: |
                $(tag)

          - upload: manifests
            artifact: manifests

Test out application on local cluster

Now let's deploy our ExpressJS application in our local cluster.

Kubernetes offers the benefit of running applications inside their own containers, isolated from one another. This containerization allows teams to develop applications in separate, independent environments, which simplifies managing dependencies and configurations. To take advantage of this for local development, we can deploy our ExpressJS application within a local Kubernetes cluster using K3D, a lightweight Kubernetes implementation designed specifically for development purposes.

Create K3D cluster and local registry

k3d cluster create test-cluster --registry-create test-cluster-registry.localhost --port 44397

Upload ExpressJS app image to local registry

docker build -t node-ts-api . -f Dockerfile.dev

docker tag node-ts-api:latest test-cluster-registry.localhost:36741/node-ts-api:local

docker push test-cluster-registry.localhost:36741/node-ts-api:local

Run Application

Run your application inside a local k3d cluster. The

-k

kustomization.yaml

dev

kustomization.yaml

kubectl apply -k ./manifests/overlays/dev

Check that the services and deployments are up and running.

kubectl get services,deployments,pods

// Example Output:
NAME          TYPE           CLUSTER-IP     EXTERNAL-IP   PORT(S)          AGE
kubernetes    ClusterIP      10.43.0.1      <none>        443/TCP          8m12s
node-ts-api   LoadBalancer   10.43.180.22   172.19.0.3    8080:32289/TCP   5m25s

View application running in browser

Now visit your browser and hit the load balancer IP address (172.19.0.3) on port 8080 (mapped to port 3000 on ExpressJS application container).

172.19.0.3:8080

Setup Flux

Setup a new repository for the main Flux Configuration. This is the repository that our Kubernetes cluster will interact with and watch for changes.

The

flux bootstrap

kubectl

flux bootstrap github \
  --owner=$GITHUB_USER \
  --repository=flux-kubernetes-test \
  --branch=main \
  --path=./clusters/test-cluster \
  --personal \
  --components-extra image-reflector-controller,image-automation-controller #<=== add extra components to automate image updates \
  --read-write-key=true

Now clone the respository:

git clone https://github.com/$GITHUB_USER/flux-kubernetes-test

cd flux-kubernetes-test

Add ExpressJS API Repositoy to Flux

The next step is to add a

GitRepository

flux-kubernetes-test-source.yaml

/clusters/test-cluster/

flux create source git flux-kubernetes-test \
  --url=https://github.com/barnacleDevelopments/kubernetes-test
  --branch=version_2 \
  --interval=1m \
  --export > ./clusters/test-cluster/flux-kubernetes-test-source.yaml

View the

flux-kubernetes-test-source.yaml

apiVersion: source.toolkit.fluxcd.io/v1
kind: GitRepository
metadata:
  name: node-ts-api
  namespace: flux-system
spec:
  interval: 1m0s
  ref:
    branch: version_2
  url: https://github.com/barnacleDevelopments/kubernetes-test
  secretRef:
    name: kubernetes-test-auth

Create secret to authenticate with repository. This is different from the personal access token (PAT) provided when bootstraping flux.

kubectl create secret generic flux-git-auth --namespace flux-system --from-literal=username=barnacleDevelopments --from-literal=password=$GITHUB_PAT

the next few objects will go in the same file.

Add Azure Container Registry

The Azure container registry is where the Docker images for our ExpressJS application will be stored after they have been built in our CI/CD Azure Devops Pipeline. This is not to be confused with the

GitRepository

apiVersion: image.toolkit.fluxcd.io/v1beta2
kind: ImageRepository
metadata:
  name: node-ts-api
  namespace: flux-system
spec:
  image: devdeveloperregistry.azurecr.io/node-ts-api #<=== this is the address of our image
  interval: 1h #<=== we are checking every hour
  provider: azure

Add ImagePolicy object

The image policy object will tell Kubernetes how to select the latest image from our container registry.

apiVersion: image.toolkit.fluxcd.io/v1beta2
kind: ImagePolicy
metadata:
  name: node-ts-api
  namespace: flux-system
spec:
  imageRepositoryRef:
    name: node-ts-api # <=== reference to the ImageRepository object.
  policy:
    numerical:
      order: asc # <=== the image we would like to select (the latest image in this case)

Add ImageUpdateAutomation object

This object tells Kubernetes how often to check our ExpressJS repository for changes and where to update the latest image tag.

apiVersion: image.toolkit.fluxcd.io/v1beta2
kind: ImageUpdateAutomation
metadata:
  name: node-ts-api-automation
  namespace: flux-system
spec:
  interval: 5m #<=== check the repository every 5 minutes for changes
  sourceRef:
    kind: GitRepository
    name: node-ts-api #<=== use the node-ts-api GitRepository object
  git:
    checkout:
      ref:
        branch: version_2
    push:
      branch: version_2
    commit:
      author:
        email: devin@mailfence.com
        name: devin
  update:
    path: ./manifests/overlays/prod
    strategy: Setters

Prepare Express JS API for Flux

In order for Flux to automate image deployment later, we will need to update our ExpressJS application manifest files. We are going to update the overlay with a couple of comments. These comments will allow Flux to identify these fields and push updates to the repository whenever the ExpressJS API image changes. It will update the tag and name separately as needed.

resources:
  - ../../base
namePrefix: prod-
images:
  - name: node-ts-api
    newName: devdeveloperregistry.azurecr.io/node-ts-api # {"$imagepolicy": "flux-system:node-ts-api:name"} #<=== comment 1
    newTag: "88" # {"$imagepolicy": "flux-system:node-ts-api:tag"} #<=== comment 2

Complete Config

Here is the complete configuration for the ExpressJS application. We are going to place this all in one file for simplicity because all these objects are highly related to each other.

---
apiVersion: source.toolkit.fluxcd.io/v1
kind: GitRepository
metadata:
  name: node-ts-api
  namespace: flux-system
spec:
  interval: 1m0s
  ref:
    branch: master
  url: https://github.com/barnacleDevelopments/kubernetes-test
  secretRef:
    name: flux-git-auth
---
apiVersion: image.toolkit.fluxcd.io/v1beta2
kind: ImageRepository
metadata:
  name: node-ts-api
  namespace: flux-system
spec:
  image: devdeveloperregistry.azurecr.io/node-ts-api #<=== this is the address of our image
  interval: 1m #<=== we are checking every hour
  provider: azure
---
apiVersion: image.toolkit.fluxcd.io/v1beta2
kind: ImageUpdateAutomation
metadata:
  name: node-ts-api-automation
  namespace: flux-system
spec:
  interval: 5m #<=== check the repository every 5 minutes for changes
  sourceRef:
    kind: GitRepository
    name: node-ts-api #<=== use the node-ts-api GitRepository object
  git:
    checkout:
      ref:
        branch: master
    push:
      branch: master
    commit:
      author:
        email: devin@mailfence.com
        name: devin
  update:
    path: ./manifests/overlays/prod
    strategy: Setters
---
apiVersion: image.toolkit.fluxcd.io/v1beta2
kind: ImagePolicy
metadata:
  name: node-ts-api
  namespace: flux-system
spec:
  imageRepositoryRef:
    name: node-ts-api # <=== reference to the ImageRepository object.
  policy:
    numerical:
      order: asc # <=== the image we would like to select (the latest image in this case)
---

Commit and push changes to github.

git add -A && git commit -m "Add flux-kubernetes-test GitRepository"
git push

Deploy the Expresss JS Application

Next, we are creating a Kustomization object. This object tells Flux where the manifest configurations are in our ExpressJS Application repository and how often to check them for changes. Notice that we are creating a file

flux-kubernetes-test-kustomization.yaml

/clusters/test-cluster/

flux create kustomization flux-kubernetes-test \
  --target-namespace=default \
  --source=node-ts-api \
  --path="./manifests/overlays/prod" \
  --prune=true \
  --wait=true \
  --interval=30m \
  --retry-interval=2m \
  --health-check-timeout=3m \
  --export > ./clusters/test-cluster/flux-kubernetes-test-kustomization.yaml

Commit and push changes to Github.

git add -A && git commit -m "Add flux-kubernetes-test Kustomization"
git push

Watch your Flux Agent

Now let's watch Flux and Kubernetes do their work reconciling this deployment:

flux get kustomizations --watch

In another terminal look at the deployments, pods, and services:

kubectl -n default get deployments,services,pods

Bringing it all Together

In the previous tutorial, I demonstrated uploading new Docker images of our ExpressJS application to our Azure Container Registry. Once this image was uploaded, Azure Pipelines (CI/CD) triggered AKS to pull the latest image from the registry to re-deploy the updated image to Kubernetes pods. Now, because we are utilizing Flux to manage deployments to Kubernetes, our pipeline simply needs to handle the build and upload to the container registry, and Flux will handle the rest.

1. Now let's update our ExpressJS application with a new route.

app.get("/flux", (req: Request, res: Response) => {
  res.send("Welcome to Flux!");
});

2. Then push those changes to GitHub.

git add .

git commit -m "New route after Flux setup"

git push

3. View images being built on Azure and uploaded to container registry.

4. Monitor our pods re-deploying.

kubectl get pods

Resources

• Flux Troubleshooting
• Terraform Resource Registry Documentation
• K3D Website
• Flux Website