Deploy to Kubernetes (Helm)
Useful Links¶
Deploying scala sbt microservice to Kubernetes
Deployment of a sbt
-built app on Kubernetes (MiniKube)¶
Test packaging without Kubernetes first¶
- Stage all Play files in a local directory and verify
sbt stage
- For direct deployment, create a distribution in
target/universal
sbt dist
The dist
task builds a binary version of your application that you can deploy to a server without any dependency on SBT, the only thing the server needs is a Java installation.
Deploy a Helm chart to Kubernetes¶
Prerequisites: minikube
, kubectl
, docker
client and helm
should be installed
- Generate the Dockerfile and environment prepared for creating a Docker image
sbt docker:stage
-
Verify the output under
target/docker
-
Start
minikube
minikube start
- Enable Ingress
minikube addons list
minikube addons enable ingress
Also consider enabling heapster
- List available nodes to verify that
kubectl
is properly configured
kubectl get nodes
It should return one node.
- Connect the Docker client to the Docker daemon in the K8s VM
eval $(minikube docker-env)
Just make sure you tag your Docker image with something other than ‘latest’ and use that tag while you pull the image.
Otherwise, if you do not specify version of your image, it will be assumed as :latest
, with pull image policy of Always
correspondingly, which may eventually result in ErrImagePull as you may not have any versions of your Docker image out there in the default docker registry (usually DockerHub) yet.
-
If needed, remove previously built images from the local Docker server with
sbt docker:clean
ordocker rmi <image>
. To view the list of Docker images, rundocker images
-
Build the Docker image and publish it to Kubernetes' Docker server.
sbt docker:publishLocal
- Deploy the Helm chart
./helm install --dry-run --debug <helm chart folder> &> output.txt
and if that looks OK
./helm install <helm chart folder>
or specify a release name:
./helm install --name <release name> <helm chart folder>
- Verify the Helm deployment to
minikube
./helm list
./helm status <release name>
More details via:
kubectl get ing
kubectl get service
kubectl get deployment
kubectl get pods
- Test the deployment by forwarding a local port to a port on the pod
kubectl get pods
kubectl port-forward <pod name> 8080:<target port on pod>
curl -v http://localhost:8080/api
kubectl port-forward
also allows using resource name, such as a service name, to select a matching pod to port forward to
kubectl port-forward svc/<service name> 8080:<service port>
curl -v http://localhost:8080/
- When needed, delete the release with
helm ls
helm delete <release name>
If you want to deploy / not deploy an Ingress¶
- Update
values.yaml
in the Helm chart root folder
ingress:
enabled: true # or: false; true by default
- If true, make sure the minikube Ingress add-on is enabled
minikube addons enable ingress
- Deploy on Kubernetes as above
See Blog
SSL Termination (TO DO)¶
- Generate a x509, pem encoded, RSA 2048 certificate with OpenSSL
openssl req -x509 -nodes -days 365 -newkey rsa:2048 -keyout ${KEY_FILE} -out ${CERT_FILE} -subj "/CN=${HOST}/O=${HOST}"
openssl req -x509 -nodes -days 365 -newkey rsa:2048 -keyout tls.key -out tls.crt -subj "/CN=john-cd.com"
Note: To find myhost.com for minikube, run the following commands:
$ minikube ssh
$ echo $HOSTNAME
minikube
- Create a Kubernetes secret
kubectl create secret tls ${CERT_NAME} --key ${KEY_FILE} --cert ${CERT_FILE}
kubectl create secret tls my-secret --key tls.key --cert tls.crt
Add under spec:
in
tls:
- hosts:
- myhost.com
secretName: my-secret
Find and delete all nginx pods to force the nginx.conf
to update and reflect the ingress changes. Find the ingress pods with the following:
kubectl get pods --all-namespaces
kubectl delete pods --namespace=kube-system [ingress pod]