Operators hide the complexity of the application and Kubernetes. Instead of dealing with Pods, StatefulSets, tons of YAML manifests, and various configuration files, the user talks to Kubernetes API to provision a ready-to-use application. An Operator automatically provisions all the required resources and exposes the application. Though, there is always a risk that the user would want to do something manual that can negatively affect the application and the Operator logic.
In this blog post, we will explain how to limit access scope for the user to avoid manual changes for database clusters deployed with Percona Operators. To do so, we will rely on Kubernetes Role-based Access Control (RBAC).
The goal
We are going to have two roles: Administrator and Developer. Administrator will deploy the Operator, create necessary roles, and service accounts. Developers will be able to:
- Create, modify, and delete Custom Resources that Percona Operators use
- List all the resources – users might want to debug the issues
Developers will not be able to:
- Create, modify, or delete any other resource
As a result, the Developer will be able to deploy and manage database clusters through a Custom Resource, but will not be able to modify any operator-controlled resources, like Pods, Services, Persistent Volume Claims, etc.
Action
We will provide an example for Percona Operator for MySQL based on Percona XtraDB Cluster (PXC), which just had version 1.12.0 released.
Administrator
Create a dedicated namespace
We will allow users to manage clusters in a single namespace called
prod-dbs
:
$ kubectl create namespace prod-dbs
Deploy the operator
Use any of the ways described in our documentation to deploy the operator into the namespace. My personal favorite will be with simple
kubectl
command:
$ kubectl apply -f https://raw.githubusercontent.com/percona/percona-xtradb-cluster-operator/v1.12.0/deploy/bundle.yaml
Create ClusterRole
ClusterRole resource defines the permissions the user will have for a specific resource in Kubernetes. You can find the YAML in this github repository.
- apiGroups: ["pxc.percona.com"] resources: ["*"] verbs: ["*"] - apiGroups: [""] resources: - pods - pods/exec - pods/log - configmaps - services - persistentvolumeclaims - secrets verbs: - get - list - watch
As you can see we allow any operations for
pxc.percona.com
resources, but restrict others to get, list, and watch.
$ kubectl apply -f https://github.com/spron-in/blog-data/blob/master/rbac-operators/clusterrole.yaml
Create ServiceAccount
We are going to generate a kubeconfig for this service account. This is what the user is going to use to connect to the Kubernetes API.
apiVersion: v1 kind: ServiceAccount metadata: name: database-manager namespace: prod-dbs
$ kubectl apply -f https://raw.githubusercontent.com/spron-in/blog-data/master/rbac-operators/serviceaccount.yaml
Create ClusterRoleBinding
We need to assign the
ClusterRole
to the
ServiceAccount
.
ClusterRoleBinding
acts as a relation between these two.
$ kubectl apply -f https://github.com/spron-in/blog-data/blob/master/rbac-operators/clusterrolebinding.yaml
apiVersion: rbac.authorization.k8s.io/v1 kind: ClusterRoleBinding metadata: name: percona-database-manager-bind namespace: prod-dbs roleRef: apiGroup: rbac.authorization.k8s.io kind: ClusterRole name: percona-pxc-rbac subjects: - kind: ServiceAccount name: database-manager namespace: prod-dbs
Developer
Verify
can-i
command allows you to verify if the service account can really do what we want it to do. Let’s try:
$ kubectl auth can-i create perconaxtradbclusters.pxc.percona.com --as=system:serviceaccount:prod-dbs:database-manager yes $ kubectl auth can-i delete service --as=system:serviceaccount:prod-dbs:database-manager no
All good. I can create Percona XtraDB Clusters, but I can’t delete Services. Please note, that sometimes it might be useful to allow Developers to delete Pods to force the cluster recovery. If you feel that it is needed, please modify the ClusterRole.
Apply
There are multiple ways to use this service account. You can read more about it in Kubernetes documentation. For a quick demonstration, we are going to generate a kubeconfig that we can share with our user.
Run this script to generate the config. What it does:
- Gets the service account secret resource name
- Extracts Certificate Authority (ca.crt) contents from the secret
- Extracts the token from the secret
- Gets the endpoint of the Kubernetes API
- Generates the kubeconfig using all of the above
$ bash generate-kubeconfig.sh > tmp-kube.config
Now let’s see if it works as expected:
$ KUBECONFIG=tmp-kube.config -n prod-dbs apply -f https://raw.githubusercontent.com/percona/percona-xtradb-cluster-operator/v1.12.0/deploy/cr.yaml perconaxtradbcluster.pxc.percona.com/cluster1 created $ KUBECONFIG=tmp-kube.config kubectl -n prod-dbs get pods NAME READY STATUS RESTARTS AGE cluster1-haproxy-0 2/2 Running 0 15m cluster1-haproxy-1 2/2 Running 0 14m cluster1-haproxy-2 2/2 Running 0 14m cluster1-pxc-0 3/3 Running 0 15m cluster1-pxc-1 3/3 Running 0 14m cluster1-pxc-2 3/3 Running 0 13m percona-xtradb-cluster-operator-77bf8b9df5-qglsg 1/1 Running 0 52m
I was able to create the Custom Resource and a cluster. Let’s try to delete the Pod:
$ KUBECONFIG=tmp-kube.config kubectl -n prod-dbs delete pods cluster1-haproxy-0 Error from server (Forbidden): pods "cluster1-haproxy-0" is forbidden: User "system:serviceaccount:prod-dbs:database-manager" cannot delete resource "pods" in API group "" in the namespace "prod-dbs"
What about the Custom Resource?
$ KUBECONFIG=tmp-kube.config kubectl -n prod-dbs delete pxc cluster1 perconaxtradbcluster.pxc.percona.com "cluster1" deleted
Conclusion
Percona Operators automate the deployment and management of the databases in Kubernetes. Least privilege principle should be applied to minimize the ability of inexperienced users to affect availability and data integrity. Kubernetes comes with sophisticated Role-Based Access Control capabilities which allow you to do just that without the need to reinvent it in your platform or application.
