Creating namespaces and initial cluster configuration on vSphere 7 with Tanzu Kubernetes Grid Service (TKGS)

  • Tanzu Kubernetes Grid Service (TKGS)
    Deploy and operate Tanzu Kubernetes clusters natively in vSphere with HA Proxy as the load balancer. Without VMware Harbor as a container repository.
    - Deploying and configuring HA Proxy
    - Deploying workloads via the supervisor cluster
    - Creating namespaces and initial cluster configuration (this post)
  • VMware Tanzu Kubernetes Grid Integrated Edition (TKGI)
    Fully featured Tanzu deployment with NSX-T.
    - Deploying and configuring NSX-T
    - Deploying workloads via the supervisor cluster
    - Creating namespaces and initial cluster configuration

Prerequisites

If you haven’t already please read through our first post on TKGS as it provides a lot of detail on what TKGS is and the configuration we’ll be using for our deployment. Further to the first post. Our second post covers the initial setup and configuration of the supervisor cluster. Which is required in order to create your first namespace.

Deploying TKGS Namespaces

No that we’ve stood up and successfully configured and tested both our HA Proxy and supervisor cluster. The next stage is to deploy our first namespace. Each namespace will have its own Kubernetes cluster. With its own subsequent Kubernetes namespaces. Where you’ll be able to deploy Zercurity and any other Kubernetes deployment.

Namespace creation

From the vSphere dashboard. Click again on the Workload Management icon and from there make sure you’re on the Namespaces tab and click, New namespace. This will open the dialog below.

Adding permissions

From here we can our current user to the namespace with edit permissions. This can be done for additional users that require management access to the cluster. This will enable them to add additional nodes and provisioning requirements.

Adding storage

More importantly, we also need to let the namespace know which storage policies are available to it for persistent volume claims (PVC).

Creating our cluster spec

If you’re still logged from our last post. You can use the logout command to close your session to the supervisor cluster.

$ kubectl vsphere logout
$ kubectl vsphere login --server=10.64.32.1 --insecure-skip-tls-verify
$ kubectl config use-context production
kubectl get virtualmachineimages
apiVersion: run.tanzu.vmware.com/v1alpha1
kind: TanzuKubernetesCluster
metadata:
name: zercurity
namespace: production
spec:
distribution:
version: v1.20.3
topology:
controlPlane:
count: 1
class: best-effort-small
storageClass: tanzu-storage-policy
workers:
count: 3
class: best-effort-small
storageClass: tanzu-storage-policy
$ kubectl get virtualmachineclasses
$ kubectl describe virtualmachineclasses best-effort-small
$ kubectl apply -f tkgs-cluster-production.yaml
$ kubectl get tanzukubernetescluster
$ kubectl get cluster

Troubleshooting

Should you run into any issues there are a number of commands you can use to get an idea of what might be going wrong.

$ kubectl get machines
$ kubectl get virtualmachines
$ kubectl get cluster
$ kubectl describe tanzukubernetescluster

Error: ErrImagePull

If you’re getting image pull errors whilst the containers are creating then check that both the VMs being provisioned are route-able to the internet and that DNS is working to resolve domain requests.

Creating a default storage class

By default there is no default storage class provided (kubectl get sc). You’ll have to manually specify the storage class when deploying for example helm apps like so:

helm install harbor bitnami/harbor --set global.storageClass=tanzu-storage-policy ...
kubectl edit tanzukubernetescluster zercurity
spec:
distribution:
fullVersion: v1.18.5+vmware.1-tkg.1.c40d30d
version: v1.18.5
settings:
network:
cni:
name: antrea
pods:
cidrBlocks:
- 192.168.0.0/16
serviceDomain: cluster.local
services:
cidrBlocks:
- 10.96.0.0/12
storage:
defaultClass: tanzu-storage-policy
$ hugh@hugh-ubuntu-dev-2004:~$ kubectl get scNAME                             PROVISIONER              
tanzu-storage-policy (default) csi.vsphere.vmware.com ...

Accessing the cluster

As with before. Logout of the previous session. This time we’re going to add our new cluster name and namespace parameters.

$ kubectl vsphere logout
$ kubectl vsphere login --server=10.64.32.1 --insecure-skip-tls-verify --tanzu-kubernetes-cluster-namespace production --tanzu-kubernetes-cluster-name zercurity
$ kubectl config use-context zercurity
kubectl get pod -A
kubectl create clusterrolebinding default-tkg-admin-privileged-binding --clusterrole=psp:vmware-system-privileged --group=system:authenticated

Deploying harbor

Harbor is an opensource container repository. Not only does it provided permissioned access to pull and push docker images but it also provides vulnerability scanning, webhooks and other functionality.

$ kubectl create ns harbor$ helm install harbor bitnami/harbor \
--set harborAdminPassword='adminpass' \
--set global.storageClass=tanzu-storage-policy \
--set service.type=LoadBalancer \
--set externalURL=harbor.test.corp \
--set service.tls.commonName=harbor.test.corp \
-n harbor
$ helm uninstall harbor -n harbor
$ kubectl get pod -n harbor
NAME READY STATUS RESTARTS AGE
harbor-chartmuseum-657b95d5f7-fxzll 1/1 Running 0 9d
harbor-clair-586d8cf9f6-rhzzd 2/2 Running 0 9d
harbor-core-5cd79cc5f6-2r2sw 1/1 Running 4 9d
harbor-jobservice-b6fff8654-kvnmn 1/1 Running 5 9d
harbor-nginx-55d7d6d846-vfr6c 1/1 Running 0 9d
harbor-notary-server-8695c547f5-hrvft 1/1 Running 0 9d
harbor-notary-signer-5647c4968c-pqwmc 1/1 Running 0 9d
harbor-portal-54cc4dbc8c-dgswz 1/1 Running 0 9d
harbor-postgresql-0 1/1 Running 0 9d
harbor-redis-master-0 1/1 Running 0 9d
harbor-registry-dd67784b8-hbthw 2/2 Running 0 9d
harbor-trivy-0 1/1 Running 0 9d
$ kubectl get svc -n harbor

Its all over!

This is our last part into standing up TKGS on vSphere. We hope you found it helpful. Please feel free to get in touch if you have any questions.

--

--

Get the Medium app

A button that says 'Download on the App Store', and if clicked it will lead you to the iOS App store
A button that says 'Get it on, Google Play', and if clicked it will lead you to the Google Play store
Zercurity

Zercurity

Real-time security and compliance delivered.