Foreword

The following is an example of integrating Kubernetes with Edgio Delivery CDN service for experienced Kubernetes practitioners. It assumes a level of knowledge in Kubernetes and knowledge of standard internet protocols such as TCP, DNS, and HTTP/HTTPS. The final objective is to demonstrate a way to make Edgio Delivery CDN service appear as standard ingress. The Kubernetes user may maintain the ingress configuration in familiar systems such as Portainer or Terraform.

Kubernetes

Kubernetes.io defines Kubernetes as “an open-source system for automating deployment, scaling, and management of containerized applications.” There are several ways to bring up a cluster. The quickest and easiest way is to leverage one of the many cloud providers that allow “one-click” deployments.

Kubernetes clusters contain many parts. In the demo, we will focus on a subset of the total.

Kubernetes Parts

Node: machine or VM that hosts the workloads and services.
Workload: Application simple or complex
Pod: containers that the application software is running within. One or more pods make up a workload.
Service: a mechanism that exposes a workload to other workloads or ingresses
Ingress: A mechanism to expose services to the outside world

Edgio

Edgio makes connected living faster, safer, and simpler to manage by powering unmatched speed, security and, simplicity at the edge with our seamlessly integrated delivery, applications and, streaming solutions. Our globally-scaled technology and expert services fuel the world’s top brands with the capacity to deliver the fastest, most dynamic and, frictionless education, entertainment, events and, applications to every user. Dedicated to providing unparalleled client care and extending value every step of the way, Edgio is a partner of choice, driving worldwide internet traffic to support the most popular shows, movies, sports, games and music, and instant-loading websites.

Architecture

Edgio Delivery CDN

For this demo, we use the Edgio python SDK at GitHub – llnw/llnw-sdk-python: Limelight Networks Python SDK. The basic configuration will be kept simple.

Configuring the CDN, the following pieces of data (other than authentication) are needed.

Shortname: A unique identifier within Edgio Delivery CDN that contains CDN rules.

Published hostname: The hostname that faces the Internet from the CDN. In the case of this demo, the hostname cstradtman.s.llnwi.net was previously configured.

Published Url Path: The path mapped into the specific rule in question.

Source Hostname: The hostname of the origin. In this case, a round-robin record points to the Kubernetes cluster.

Source Url Path: The destination path associated with the hostname. In this case, it will be blank since we describe it later in this document. In this demo, workloads will be differentiated by port number and not a path.

DNS

The demo uses the Vultr DNS API. Configuring the CDN using IP addresses as the origins is not a best practice. The demo uses an API to create or modify a round-robin A record pointing to the cluster nodes

Cluster

The demo is in Vultr on a three-node cluster.

Namespaces

The demo will exist under the default namespace since it’s the only thing running on this cluster

Nodes

VKE cluster – 3 nodes each with
4G ram
80G disk
2 vCPU
1 G network

Workloads

The workload is a simple Python program that returns basic information about the HTTP connection. We deploy this demo inline, which is not a typical deployment method. However, it allows the demo to be fully self-contained rather than having external dependencies to build the containers and store them. The source code is in Appendix 1. The actual Python code is italicized, and the YAML code describing the deployment is not. The workload utilizes TCP port 8000.

Services

There are several service types available. We are using the “NodePort” service type. It exposes the service on the external IP addresses of the cluster on a randomly selected high port number. Kubernetes takes care of the connection to a node where the workload runs.

Ingress

Kubernetes ingress is a software object that describes the behavior of a collection of resources that make a Kubernetes service available outside of the cluster.

An Ingress controller combines software and/or hardware that instantiates the Ingress software object.

A CDN is a feature-rich distributed reverse proxy, besides the reverse proxy function, CDNs usually have flexible cache algorithms, geographic diversity, security features (ACLs (GEO, method, regex), Web applications Firewalls, watermarking, and other more advanced features not found in simple reverse proxies.

The APIs for Kubernetes object monitoring are well defined, and a number of HTTP reverse proxies and load balancers have been extended to work as Ingress controllers. For example, NGINX and HAProxy are old school proxies that have been extended to work as ingress controllers and ones like Traefik and Itsio were developed in parallel to Kubernetes . A CDN is a natural next progression from simple reverse proxies. In the case of this demo, we will create a custom ingress controller that controls Edgio Delivery CDN. The data typically required for describing an Ingress is

Public Hostname: The hostname that the Kubernetes service will be available on
Public path: the path associated with the Public hostname that the Kubernetes services will be available on
Backend service name: service-name defined by a Kubernetes services stanza
Backend Port: A name matching the port description in the service description

Edgio Delivery CDN asset mappings

Kubernetes	Edgio Delivery CDN
Public Hostname	Published Hostname
Public Path	Published URL Path
Backend Service Name	Unique within the Kubernetes cluster (Not known/available to the outside world)
Source Hostname	Fully qualified domain name (Not actually one-to-one)

The logic behind it is:

Listen for namespace ingress event from the cluster
- If the event type is “added.”
  - Get the IP address of nodes in the cluster
  - Create DNS record in vultr
  - Create CDN rules in Edgio CDN pointing toward the created DNS records and the port allocated by the NodePort service.
- If the event type is “modified.”
  - Find CDN rules in Edgio CDN that match the existing ingress
  - Modify rules in Edgio CDN to reflect new data in the event
- If the event type is “deleted.”
  - Find CDN rules in Edgio CDN that match removed ingress
  - Remove DNS records in Vultr associated with this ingress
  - Remove the CDN rules in Edgio that match removed ingress

Configuration

Objects

Kubernetes objects are described in YAML files. In Kubernetes, every configuration is an object. You could place all the deployment configs in a single file. It’s common practice to break them up based on what portion of the deployment they describe.

Deployment

In Appendix 1, there is a large YAML file that is a combination of Python code and YAML. Everything after “spec.template.spec.containers.command” is the Python code that is turned into a container during the deployment. From the point of view of the Kubernetes object the lines that are important to us for the demo are:

Kind: Deployment – describes the object as a deployment
Metadata.name: name is used by the service object to create the link between the two.
Spec.selector.matchlabels.app: tells what pods the deployment will apply to.
Spec.template.spec.containers.image: points to the docker hub python 3 official image
Spec.template.spec.containers.ports.name: a label for the port to be referenced in other objects service in this case
Spec.template.spec.containers.ports.containerport: lists the TCP port exposed from the running container

Service

Kind: Deployment – describes the object as a deployment
Metadata.name: name is used by the ingress object to create the link between the two.
Spec.selector.app: points to the name of the deployment – pywai-inline-service as named in the deployment object.
Spec.selector.type: Nodeport this defines this service as type Nodeport as described above
Spec.ports.name: the label for the port to be referenced by other objects(ingress in this case)
Spec.ports.protocol: defines IP protocol in use – TCP
Spec.ports.port: defines port exposed by this service
spec.ports.targetPort: points to port exposed by the deployment – pywai-inline-deployment in this case

Ingress

In Appendix 3, you will see the YAML file describing the Kubernetes object for the Edgio ingress controller. The objects to be concerned with are:

Config Maps

Secrets

Kubernetes has a built-in concept of secrets for the storage of sensitive data. This demo leverages this for the usernames and shared keys for both Edgio SDK and the Vultr DNS API

Code examples

The code for this demonstration included in the git repos

Edgio Delivery python SDK
Kubernetes demo deployment
- https://github.com/llnw/edgio-delivery-k8s-demo
- These are a series of Kubernetes object described as yaml that show how to deploy the Edgio Delivery integration
Edgio Delivery ingress controller demo

Future additions and enhancements to the current design

In a future post, we’ll show how to leverage other Edgio solutions for example Applications. We could use the Kubernetes cluster as a way to leverage and sync configurations among multiple Edgio CDN offerings for different workflows.

Appendix 1

pywai-inline-deployment.yaml

apiVersion: apps/v1
kind: Deployment
metadata:
name: pywai-inline-deployment
spec:
  selector:
    matchLabels:
      app: pywai-inline-deployment
  replicas: 1
  template:
    metadata:
      labels:
        app: pywai-inline-deployment
    spec:
      containers:
        - name: python
          image: python:3
          command:
            - /bin/sh
            - "-c"
            - |
              cat > /test.py <<EOF
              #!/usr/bin/env python3
              """
              Very simple HTTP server in python for logging requests
              Usage::
                  ./server.py [<port>]
              """
              from http.server import BaseHTTPRequestHandler, HTTPServer
              import logging
              import pprint

              class S(BaseHTTPRequestHandler):
                  def _set_response(self):
                      self.send_response(200)
                      self.send_header('Content-type', 'text/html')
                      self.end_headers()

                  def do_GET(self):
                      logging.info("GET request,\nPath: %s\nHeaders:\n%s\n", str(self.path), str(pprint.pformat(self.headers)))
                      self._set_response()
                      headerinfo = pprint.pformat(self.headers.items())
                      clientinfo = pprint.pformat(self.client_address)
                      lineinfo = pprint.pformat(self.requestline)
                      versioninfo = pprint.pformat(self.request_version)

                      self.wfile.write("<pre> {} </pre>".format(headerinfo).encode('utf-8'))
                      self.wfile.write("<pre> {} </pre>".format(clientinfo).encode('utf-8'))
                      self.wfile.write("<pre> {} </pre>".format(lineinfo).encode('utf-8'))
                      self.wfile.write("<pre> {} </pre>".format(versioninfo).encode('utf-8'))

              def run(server_class=HTTPServer, handler_class=S, port=8080):
                  logging.basicConfig(level=logging.INFO)
                  server_address = ('', port)
                  httpd = server_class(server_address, handler_class)
                  logging.info('Starting httpd...\n')
                  try:
                      httpd.serve_forever()
                  except KeyboardInterrupt:
                      pass
                  httpd.server_close()
                  logging.info('Stopping httpd...\n')

              if __name__ == '__main__':
                  from sys import argv
                  if len(argv) == 2:
                      run(port=int(argv[1]))
                  else:
                      run()
              EOF
              exec python /test.py 8000
          ports:
            - name: http
              containerPort: 8000

apiVersion: apps/v1
kind: Deployment
metadata:
name: pywai-inline-deployment
spec:
  selector:
    matchLabels:
      app: pywai-inline-deployment
  replicas: 1
  template:
    metadata:
      labels:
        app: pywai-inline-deployment
    spec:
      containers:
        - name: python
          image: python:3
          command:
            - /bin/sh
            - "-c"
            - |
              cat > /test.py <<EOF
              #!/usr/bin/env python3
              """
              Very simple HTTP server in python for logging requests
              Usage::
                  ./server.py [<port>]
              """
              from http.server import BaseHTTPRequestHandler, HTTPServer
              import logging
              import pprint

              class S(BaseHTTPRequestHandler):
                  def _set_response(self):
                      self.send_response(200)
                      self.send_header('Content-type', 'text/html')
                      self.end_headers()

                  def do_GET(self):
                      logging.info("GET request,\nPath: %s\nHeaders:\n%s\n", str(self.path), str(pprint.pformat(self.headers)))
                      self._set_response()
                      headerinfo = pprint.pformat(self.headers.items())
                      clientinfo = pprint.pformat(self.client_address)
                      lineinfo = pprint.pformat(self.requestline)
                      versioninfo = pprint.pformat(self.request_version)

                      self.wfile.write("<pre> {} </pre>".format(headerinfo).encode('utf-8'))
                      self.wfile.write("<pre> {} </pre>".format(clientinfo).encode('utf-8'))
                      self.wfile.write("<pre> {} </pre>".format(lineinfo).encode('utf-8'))
                      self.wfile.write("<pre> {} </pre>".format(versioninfo).encode('utf-8'))

              def run(server_class=HTTPServer, handler_class=S, port=8080):
                  logging.basicConfig(level=logging.INFO)
                  server_address = ('', port)
                  httpd = server_class(server_address, handler_class)
                  logging.info('Starting httpd...\n')
                  try:
                      httpd.serve_forever()
                  except KeyboardInterrupt:
                      pass
                  httpd.server_close()
                  logging.info('Stopping httpd...\n')

              if __name__ == '__main__':
                  from sys import argv
                  if len(argv) == 2:
                      run(port=int(argv[1]))
                  else:
                      run()
              EOF
              exec python /test.py 8000
          ports:
            - name: http
              containerPort: 8000

Appendix 2

pywai-inline-service.yaml

apiVersion: v1
kind: Service
metadata:
  name: pywai-inline-service
spec:
  selector:
    app: pywai-inline-deployment
  type: NodePort
  ports:
    - name: http
      protocol: TCP
      port: 80
      targetPort: 8000

apiVersion: v1
kind: Service
metadata:
  name: pywai-inline-service
spec:
  selector:
    app: pywai-inline-deployment
  type: NodePort
  ports:
    - name: http
      protocol: TCP
      port: 80
      targetPort: 8000

Appendix 3 pywai-edgio-ingress.yaml

apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
  name: external-pywai-inline-ingress
  annotations:
    kubernetes.io/ingress.class: edgio
  labels:
    cdn: edgio
spec:
  rules:
    - host: <Enter unique shortname>.s.llnwi.net
      http:
        paths:
          - path: /fred
            pathType: Exact
            backend:
              service:
                name: external-pywai-inline-service
                port:
                  name: pywai-port

apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
  name: external-pywai-inline-ingress
  annotations:
    kubernetes.io/ingress.class: edgio
  labels:
    cdn: edgio
spec:
  rules:
    - host: <Enter unique shortname>.s.llnwi.net
      http:
        paths:
          - path: /fred
            pathType: Exact
            backend:
              service:
                name: external-pywai-inline-service
                port:
                  name: pywai-port

Implementing Edgio Delivery CDN as an Ingress Provider in Your Kubernetes Cluster

Foreword

Kubernetes

Kubernetes Parts

Edgio

Architecture

Edgio Delivery CDN

DNS

Cluster

Namespaces

Nodes

Workloads

Services

Ingress

Edgio Delivery CDN asset mappings

Configuration

Objects

Deployment

Service

Ingress

Config Maps

Secrets

Code examples

Future additions and enhancements to the current design

Appendix 1

pywai-inline-deployment.yaml

Appendix 2

pywai-inline-service.yaml

Appendix 3

pywai-edgio-ingress.yaml