{"id":31199,"date":"2024-03-12T11:23:46","date_gmt":"2024-03-12T10:23:46","guid":{"rendered":"https:\/\/www.dbi-services.com\/blog\/?p=31199"},"modified":"2024-03-12T11:23:49","modified_gmt":"2024-03-12T10:23:49","slug":"rancher-rke2-templates-assign-members-to-clusters","status":"publish","type":"post","link":"https:\/\/www.dbi-services.com\/blog\/rancher-rke2-templates-assign-members-to-clusters\/","title":{"rendered":"Rancher RKE2 templates – Assign members to clusters"},"content":{"rendered":"\n
When testing RKE2 templates, I faced an issue with member assignments. When creating the cluster, a management cluster name is generated with the format c-m-xxxxxxxx, but the ClusterRoleTemplateBinding requires the cluster name to work. After digging into Rancher source code, I found out how to set the management cluster name. So let’s start!<\/p>\n\n\n\n
Force the cluster name with RKE2 templates<\/h2>\n\n\n\n
Investigation<\/h3>\n\n\n\n
When searching how the cluster name is generated when provisioning, I found the following code in the Rancher GitHub repository.<\/p>\n\n\n
From the function mgmtClusterName I was able to find the following code.<\/p>\n\n\n
\nif mgmtClusterNameAnnVal, ok := cluster.Annotations[mgmtClusterNameAnn]; ok && mgmtClusterNameAnnVal != "" && newCluster.Name == "" {\n\t\/\/ If the management cluster name annotation is set to a non-empty value, and the mgmt cluster name has not been set yet, set the cluster name to the mgmt cluster name.\n\tnewCluster.Name = mgmtClusterNameAnnVal\n} else if newCluster.Name == "" {\n\t\/\/ If the management cluster name annotation is not set and the cluster name has not yet been generated, generate and set a new mgmt cluster name.\n\tmgmtName, err := mgmtClusterName()\n\tif err != nil {\n\t\treturn nil, status, err\n\t}\n\tnewCluster.Name = mgmtName\n}\n<\/pre><\/div>\n\n\n
Which finally leads to the following annotation.<\/p>\n\n\n
To avoid using the generated Cluster Name given by mgmtClusterName(), we can add the following annotation “provisioning.cattle.io\/management-cluster-name” into the cluster.provisioning.cattle.io resources.<\/p>\n\n\n\n
We can pick the same code from the Rancher template example in ClusterRoleTemplateBinding and do the following.<\/p>\n\n\n
The template code above will ensure that the management cluster name will always be the one we generated ourselves. Now let’s check the ClusterRoleTemplateBinding resource for automatically assigning users and groups.<\/p>\n\n\n\n
RKE2 ClusterRoleTemplateBinding<\/h2>\n\n\n\n
To predefined users and groups into the cluster, we can use the template clusterroletemplatebinding.yaml.<\/p>\n\n\n
When using only local users, it is easier as you only specify local:\/\/ with the ID of the user. But if you use groups, you may not know what value to provide. The same applies to your custom roles. The easiest way is to manually assign your members, and read the YAML files created.<\/p>\n\n\n\n
For this example, I am adding my GitHub group “teamA” as a cluster member, and a local user “autoscaler” as a cluster owner.<\/p>\n\n\n\n<\/figure>\n\n\n\n
Now go to More Resources > RBAC > ClusterRoleBindings and sort by age. You should find the values to specify for principalName and roleTemplateName.<\/p>\n\n\n\n<\/figure>\n\n\n\n
New issues<\/h2>\n\n\n\n
When the cluster is created for the first time, Rancher automatically creates the namespace. ClusterRoleTemplateBinding needs to be deployed into this namespace. Therefore it cannot be created at the creation of the cluster. In addition, it needs to wait for the Cluster resources to be provisioned by Rancher.<\/p>\n\n\n
Due to that issue with the namespace, setting the management cluster name doesn’t bring much advantage, as we need to manually redeploy the App (cluster template) to assign the users\/groups.<\/p>\n\n\n\n
Therefore we can use Helm function lookup, which can directly read the cluster name from the Kubernetes local cluster. Same as precedent, we need to redeploy the App (cluster template) the first time as it needs multiple resources to be provisioned by Rancher first.<\/p>\n\n\n\n
Here is the code for the clusterroletemplatebinding.yaml<\/p>\n\n\n
It will look into fleet-default for the cluster.provisionning.catte.io\/v1 that is created by the RKE2 templates itself. On the first deployment of the RKE2 templates, it doesn’t exist yet, therefore an “if” condition is added. Once the RKE2 template is deployed, you can like precedently, edit the App to redeploy it, and it will then create the ClusterRoleTemplateBinding.<\/p>\n\n\n\n
The helm install will show no errors as the ClusterRoleTemplateBinding resources are skipped.<\/p>\n\n\n
\nhelm install --generate-name=true --namespace=fleet-default --timeout=10m0s --values=\/home\/shell\/helm\/values-dbiservices-template-ec2-0.0.1.yaml --version=0.0.1 --wait=true \/home\/shell\/helm\/dbiservices-template-ec2-0.0.1.tgz\n2024-02-28T13:18:48.886232477Z creating 5 resource(s)\nbeginning wait for 5 resources with timeout of 10m0s\n2024-02-28T13:18:49.098629894Z NAME: dbiservices-template-ec2-0-1709126327\n2024-02-28T13:18:49.098705783Z LAST DEPLOYED: Wed Feb 28 13:18:47 2024\nNAMESPACE: fleet-default\nSTATUS: deployed\n2024-02-28T13:18:49.098717154Z REVISION: 1\n2024-02-28T13:18:49.098720727Z TEST SUITE: None\n2024-02-28T13:18:49.126871035Z \n2024-02-28T13:18:49.126936065Z ---------------------------------------------------------------------\n2024-02-28T13:18:49.135118662Z SUCCESS: helm install --generate-name=true --namespace=fleet-default --timeout=10m0s --values=\/home\/shell\/helm\/values-dbiservices-template-ec2-0.0.1.yaml --version=0.0.1 --wait=true \/home\/shell\/helm\/dbiservices-template-ec2-0.0.1.tgz\n---------------------------------------------------------------------\n<\/pre><\/div>\n\n\n
And then, when redeploying the App, it does show the creation of the resources as the cluster name now exists.<\/p>\n\n\n
\nhelm upgrade --history-max=5 --install=true --namespace=fleet-default --timeout=10m0s --values=\/home\/shell\/helm\/values-dbiservices-template-ec2-0.0.1.yaml --version=0.0.1 --wait=true dbiservices-template-ec2-0-1709126327 \/home\/shell\/helm\/dbiservices-template-ec2-0.0.1.tgz\nchecking 8 resources for changes\nPatch Amazonec2Config "kke-test-template-controlplane" in namespace fleet-default\nPatch Amazonec2Config "kke-test-template-workers" in namespace fleet-default\nPatch Cluster "kke-test" in namespace fleet-default\nCreated a new ClusterRoleTemplateBinding called "ctrb-96090621" in c-m-58wcfhnl\n2024-02-28T13:24:13.652797054Z \nCreated a new ClusterRoleTemplateBinding called "ctrb-2c866242" in c-m-58wcfhnl\n\nCreated a new ClusterRoleTemplateBinding called "ctrb-d4063a0e" in c-m-58wcfhnl\n2024-02-28T13:24:13.689031588Z \nPatch ManagedChart "monitoring-crd-kke-test" in namespace fleet-default\nPatch ManagedChart "monitoring-kke-test" in namespace fleet-default\nbeginning wait for 8 resources with timeout of 10m0s\nRelease "dbiservices-template-ec2-0-1709126327" has been upgraded. Happy Helming!\n2024-02-28T13:24:13.872941208Z NAME: dbiservices-template-ec2-0-1709126327\n2024-02-28T13:24:13.872946743Z LAST DEPLOYED: Wed Feb 28 13:24:11 2024\n2024-02-28T13:24:13.872951269Z NAMESPACE: fleet-default\n2024-02-28T13:24:13.872955002Z STATUS: deployed\n2024-02-28T13:24:13.872958850Z REVISION: 4\n2024-02-28T13:24:13.872962507Z TEST SUITE: None\n\n---------------------------------------------------------------------\nSUCCESS: helm upgrade --history-max=5 --install=true --namespace=fleet-default --timeout=10m0s --values=\/home\/shell\/helm\/values-dbiservices-template-ec2-0.0.1.yaml --version=0.0.1 --wait=true dbiservices-template-ec2-0-1709126327 \/home\/shell\/helm\/dbiservices-template-ec2-0.0.1.tgz\n---------------------------------------------------------------------\n<\/pre><\/div>\n\n\n<\/figure>\n\n\n\n<\/figure>\n\n\n\n
Conclusion<\/h2>\n\n\n\n
The assignment of users\/groups to a cluster through a template is not simple. My approach might be wrong but It was the first solution I thought of when I encountered the problem. The issue of waiting for the creation (first deployment with Helm) is a little bit annoying but is not much of an issue when you are aware of redeploying the RKE2 template. Also if you use Fleet to continuously deploy\/update managed clusters, you could add the values for member configuration after the first deployment to avoid managing through the UI like above.
Below is the link to the GitHub Repository branch for the RKE2 templates using a fixed management cluster name and Helm function lookup.<\/p>\n\n\n\n
![\"\"](\"http:\/\/www.dbi-services.com\/blog\/wp-content\/uploads\/sites\/2\/2024\/02\/image-41-1024x340.png\")
<\/figure>\n\n\n\n![\"\"](\"http:\/\/www.dbi-services.com\/blog\/wp-content\/uploads\/sites\/2\/2024\/02\/image-42-1024x352.png\")
<\/figure>\n\n\n\n![\"\"](\"https:\/\/www.dbi-services.com\/blog\/wp-content\/uploads\/sites\/2\/2024\/02\/image-105-1024x268.png\")
<\/figure>\n\n\n\n![\"\"](\"https:\/\/www.dbi-services.com\/blog\/wp-content\/uploads\/sites\/2\/2024\/02\/image-106-1024x396.png\")
<\/figure>\n\n\n\n![\"\"](\"https:\/\/www.dbi-services.com\/blog\/wp-content\/uploads\/sites\/2\/2024\/02\/image-108-1024x455.png\")
<\/figure>\n\n\n\n![\"\"](\"https:\/\/www.dbi-services.com\/blog\/wp-content\/uploads\/sites\/2\/2024\/02\/image-107-1024x448.png\")
<\/figure>\n\n\n\n