Deploy to Kubernetes (Helm)

Deploying scala sbt microservice to Kubernetes

SBT Native Packager - Docker

OpenSSL

Helm

Deployment of a sbt-built app on Kubernetes (MiniKube)

Test packaging without Kubernetes first

  • Stage all Play files in a local directory and verify
sbt stage
  • For direct deployment, create a distribution in target/universal
sbt dist

The dist task builds a binary version of your application that you can deploy to a server without any dependency on SBT, the only thing the server needs is a Java installation.

Deploy a Helm chart to Kubernetes

Prerequisites: minikube, kubectl, docker client and helm should be installed

  • Generate the Dockerfile and environment prepared for creating a Docker image
sbt docker:stage

  • Verify the output under target/docker

  • Start minikube

minikube start
  • Enable Ingress
minikube addons list
minikube addons enable ingress

Also consider enabling heapster

  • List available nodes to verify that kubectl is properly configured
kubectl get nodes

It should return one node.

  • Connect the Docker client to the Docker daemon in the K8s VM
eval $(minikube docker-env)

Just make sure you tag your Docker image with something other than ‘latest’ and use that tag while you pull the image. Otherwise, if you do not specify version of your image, it will be assumed as :latest, with pull image policy of Always correspondingly, which may eventually result in ErrImagePull as you may not have any versions of your Docker image out there in the default docker registry (usually DockerHub) yet.

  • If needed, remove previously built images from the local Docker server with sbt docker:clean or docker rmi <image>. To view the list of Docker images, run docker images

  • Build the Docker image and publish it to Kubernetes' Docker server. 

sbt docker:publishLocal
  • Deploy the Helm chart
./helm install --dry-run --debug <helm chart folder> &> output.txt

and if that looks OK

./helm install <helm chart folder>

or specify a release name:

./helm install --name <release name> <helm chart folder>
  • Verify the Helm deployment to minikube
./helm list
./helm status <release name>

More details via:

kubectl get ing
kubectl get service
kubectl get deployment
kubectl get pods
  • Test the deployment by forwarding a local port to a port on the pod
kubectl get pods
kubectl port-forward <pod name> 8080:<target port on pod> 
curl -v http://localhost:8080/api

kubectl port-forward also allows using resource name, such as a service name, to select a matching pod to port forward to

kubectl port-forward svc/<service name>  8080:<service port> 
curl -v http://localhost:8080/
  • When needed, delete the release with 
helm ls
helm delete <release name>

If you want to deploy / not deploy an Ingress

  • Update values.yaml in the Helm chart root folder
ingress:
  enabled: true  # or: false; true by default
  • If true, make sure the minikube Ingress add-on is enabled
minikube addons enable ingress
  • Deploy on Kubernetes as above

See Blog

SSL Termination (TO DO)

IBM Ingress TLS tutorial

  • Generate a x509, pem encoded, RSA 2048 certificate with OpenSSL
openssl req -x509 -nodes -days 365 -newkey rsa:2048 -keyout ${KEY_FILE} -out ${CERT_FILE} -subj "/CN=${HOST}/O=${HOST}"

openssl req -x509 -nodes -days 365 -newkey rsa:2048 -keyout tls.key -out tls.crt -subj "/CN=john-cd.com"

Note: To find myhost.com for minikube, run the following commands:

 $ minikube ssh
 $ echo $HOSTNAME
 minikube
  • Create a Kubernetes secret
kubectl create secret tls ${CERT_NAME} --key ${KEY_FILE} --cert ${CERT_FILE}

kubectl create secret tls my-secret --key tls.key --cert tls.crt

Add under spec: in 

  tls:
  - hosts:
    - myhost.com
    secretName: my-secret

Find and delete all nginx pods to force the nginx.conf to update and reflect the ingress changes. Find the ingress pods with the following:

kubectl get pods --all-namespaces
kubectl delete pods --namespace=kube-system [ingress pod]