RCE Endeavors 😅

May 18, 2021

Creating a multi-language compiler system: Kubernetes (8/11)

Filed under: Programming — admin @ 10:29 PM

Table of Contents:

This post will cover how to add Kubernetes as an orchestration layer for the Dockerfiles that were created in the previous post. Right now, each language has its own dedicated container environment to run it. However, launching these Dockerfiles is a manual process, and these files do not automatically scale with user input. If we have a C++ compiler system running and provide it with a constant stream of input files, the system will eventually back up if the rate of compilation and execution is greater than the rate of input files being added. It would be nice to automatically launch Docker containers to handle this increased load, and alternatively it would be nice to automatically kill off extra Docker containers that are idle for long periods of time. This is where Kubernetes comes in to the picture.

I won’t go into much detail about Kubernetes; their interactive tutorials cover more ground than I ever could in a blog post. All that I will mention is that it is an exceptionally feature rich and complex platform for automating deployments, scaling, and general orchestration of containers. As part of this multi-language compiler system, the only features that we care about are autoscaling and mounting a partition to the host.

Deployment configuration

The Kubernetes deployments for each language is provided in a deployment.yaml file. This file provides a spec for the containers for CPU and memory limits, as well as volume information for where the shared mount volume. This shared mount volume is needed as an easy way to provide input source files to the running containers, as containers cannot have access to the host filesystem. Exposing mounted volumes to containers can be a security risk and was explicitly chosen to keep the system simple, as opposed to fully secure. A deployment.yaml file is shown below:

apiVersion: apps/v1
kind: Deployment
  name: test-compiler-c
  namespace: default
  replicas: 1
      compiler: c
        compiler: c
      - name: compiler-base-c
        image: localhost:32000/compiler-base-c:latest
            memory: "512Mi"
            cpu: "100m"
            memory: "1024Mi"
            cpu: "1000m"
        - mountPath: /home/user/code/share
          name: io-shared-volume
              command: ['sh', '-c', './shutdown.sh']
      - name: io-shared-volume
          path: /home/{{USER}}/Desktop/share
          type: Directory
apiVersion: autoscaling/v2beta2
kind: HorizontalPodAutoscaler
  name: test-compiler-c
    apiVersion: apps/v1
    kind: Deployment
    name: test-compiler-c
  minReplicas: 1
  maxReplicas: 10
  - type: Resource
      name: cpu
        type: Utilization
        averageUtilization: 75
  - type: Resource
      name: memory
        type: AverageValue
        averageValue: 800Mi

This particular file is for the C compiler environment. The container compiler-base-c will be launched under this deployment configuration. This container will have the resources specified under the resources section and have a mounted volume that will be visible as /home/user/code/share on the container. This mounted volume will map to the host at path /home/{{USER}}/Desktop/share where {{USER}} will be replaced by the current username when the deployment script is run. When the container is set to shut down, it will invoke the “sh -c ./shutdown.sh” command to clean up.

The latter half of the deployment.yaml file specifies the autoscaling configurations. These containers will scale based on two resource limmits: CPU and memory. If the CPU utilization averages above 75% then Kubernetes will scale in another container pod. Likewise, if the average memory usage is above 800 MB then the same will happen. The minReplicas and maxReplicas specify the minimum and maximum number of container pods that may be active at any given time. Kubernetes will continue to scale the number of container pods up to a maximum of ten if the memory and/or CPU usage is consistently high. If pods become underutilized then they will be automatically taken out of service and killed until there is one pod remaining.

Deployment script

The script to perform a deployment is shown below. The script will perform a deployment for each supported language. There is some logic to substitute the {{USER}} field in the deployment file, but the rest is a straightforward call to the Kubernetes command-line tool kubectl to do the deployment.


LANGUAGES="c cpp cs java py"

    REPLACE_USER=`cat "Deployments/${LANGUAGE}/deployment.yaml" | sed "s/{{USER}}/$CURRENT_USER/g"`
    echo "Deploying containers for ${LANGUAGE}"
    echo "$REPLACE_USER" > Deployments/${LANGUAGE}/deployment.runtime.yaml
    sudo microk8s.kubectl apply -f Deployments/${LANGUAGE}/deployment.runtime.yaml
    echo "Deployed containers for ${LANGUAGE}"

This fully covers it for the design and implementation of the multi-language compiler system! As described throughout these posts, the system has support for multiple languages, interactive input, multi-threading, and a scalable production deployment in a containerized environment. The next post will show off a demo of the system in action.

No Comments »

No comments yet.

RSS feed for comments on this post. TrackBack URL

Leave a comment


Powered by WordPress