Grant fine-grained access in control planes

important

ResourceDistribution is an alpha feature. It requires Spaces v1.19.0 or later and is off by default. To enable it, set features.alpha.resourceDistribution.enabled=true when you install or upgrade Spaces:

up space init --token-file="${SPACES_TOKEN_PATH}" "v${SPACES_VERSION}" \
  ...
  --set "features.alpha.resourceDistribution.enabled=true"

Built-in control plane roles such as controlplane-view cover a whole control plane. To give a team less, such as read access to S3 buckets in one namespace, put Kubernetes RBAC objects in a ResourceDistribution. Spaces applies them to every control plane in the group that the ResourceDistribution selects, including control planes you create later. The team needs no built-in role, because Spaces forwards its requests to the control plane and the RBAC there decides.

warning

Anyone who can write a ResourceDistribution can grant controlplane-admin on every control plane in its group. With Upbound RBAC, only group and organization admins can write them. With Kube RBAC, grant write access to resourcedistributions only to users who should be controlplane-admin.

Grant a team access

This example lets team A, the group okta:team-a from the Space's OIDC provider, read S3 buckets in the team-a namespace of every production control plane in the payments group. Every production control plane runs provider-aws-s3 and already has a team-a namespace.

1. In the Space, label the production control planes:

   kubectl label controlplanes prod-east prod-west env=prod -n payments

2. Save this ResourceDistribution as team-a-buckets.yaml:

   apiVersion: spaces.upbound.io/v1alpha1
   kind: ResourceDistribution
   metadata:
     name: team-a-buckets
     namespace: payments
   spec:
     controlPlaneSelector:
       labelSelectors:
       - matchLabels:
           env: prod
     deletionPolicy: Delete
     sources:
     - type: Inline
       inline:
         manifests:
         - apiVersion: rbac.authorization.k8s.io/v1
           kind: ClusterRole
           metadata:
             name: team-a-s3-bucket-viewer
           rules:
           - apiGroups: ["s3.aws.m.upbound.io"]
             resources: ["buckets"]
             verbs: ["get", "list", "watch"]
         - apiVersion: rbac.authorization.k8s.io/v1
           kind: RoleBinding
           metadata:
             name: team-a-buckets
             namespace: team-a
           roleRef:
             apiGroup: rbac.authorization.k8s.io
             kind: ClusterRole
             name: team-a-s3-bucket-viewer
           subjects:
           - apiGroup: rbac.authorization.k8s.io
             kind: Group
             name: okta:team-a

   The RoleBinding grants the ClusterRole to okta:team-a only in the team-a namespace. To bind Upbound users, robots or teams instead, use the subject names in Authorize actions in control planes.

3. Apply it to the Space:

   kubectl apply -f team-a-buckets.yaml

4. Check that every selected control plane has the objects:

   kubectl get resourcedistribution team-a-buckets -n payments

   NAME             READY   PROVISIONED   AGE
   team-a-buckets   True    2/2           1m

   READY is True once Spaces has created the ClusterRole and RoleBinding in every selected control plane. It doesn't check what they grant. If it isn't True, see Troubleshoot a grant.

5. Check the access. Connect to one of the control planes as controlplane-admin, for example with up ctx <org>/<space>/payments/prod-east, and create a SubjectAccessReview:

   kubectl create -o jsonpath='{.status.allowed}{"\n"}' -f - <<EOF
   apiVersion: authorization.k8s.io/v1
   kind: SubjectAccessReview
   spec:
     groups: ["okta:team-a"]
     resourceAttributes:
       group: s3.aws.m.upbound.io
       resource: buckets
       verb: list
       namespace: team-a
   EOF

   It prints true. With namespace: default, it prints false. To check one member, also set user and all their groups. kubectl auth can-i --as doesn't work, because controlplane-admin can't impersonate users.

6. Let the team list its control planes. With Kube RBAC, switch back to the Space with up ctx .. and apply this Role and RoleBinding, so kubectl get controlplanes -n payments shows them to team A:

   apiVersion: rbac.authorization.k8s.io/v1
   kind: Role
   metadata:
     name: team-a-controlplanes
     namespace: payments
   rules:
   - apiGroups: ["spaces.upbound.io"]
     resources: ["controlplanes"]
     resourceNames: ["prod-east", "prod-west"]
     verbs: ["get"]
   ---
   apiVersion: rbac.authorization.k8s.io/v1
   kind: RoleBinding
   metadata:
     name: team-a-controlplanes
     namespace: payments
   roleRef:
     apiGroup: rbac.authorization.k8s.io
     kind: Role
     name: team-a-controlplanes
   subjects:
   - apiGroup: rbac.authorization.k8s.io
     kind: Group
     name: okta:team-a

   With Upbound RBAC, any group role lists the control planes but also grants controlplane-view on the whole group.

Guidelines

• Give each team and namespace its own ResourceDistribution. While any object in a ResourceDistribution fails to apply on a control plane, Spaces doesn't remove any of its objects there. Separate ResourceDistributions keep one team's failure from blocking another team's revocation.
• Set deletionPolicy: Delete, as the example does. With Delete, removing a role or binding from the ResourceDistribution, or deleting the ResourceDistribution, deletes it from the control planes. The default, Orphan, leaves it in place, so access outlives the grant. strategy: ApplyOnce and the annotation resourcedistribution.spaces.upbound.io/prune: "false" also keep objects after you remove them.
• List each role before the bindings that use it, because Spaces applies manifests in order.
• Declare each object in one ResourceDistribution only. To share a role between teams, give the role its own ResourceDistribution.
• Grant a provider's resources only on control planes that run the provider.

Limits

• Deselecting a control plane doesn't revoke access. The control plane keeps what Spaces applied, and the status stops listing it. To revoke access there, delete the objects yourself as controlplane-admin:

  kubectl delete clusterroles,clusterrolebindings,roles,rolebindings -A \
    -l resourcedistribution.spaces.upbound.io/managed-by=team-a-buckets

• A ResourceDistribution only adds access. It can't narrow a built-in role the team already has, such as controlplane-view.

Troubleshoot a grant

Each selected control plane has a ResourceDistributionBinding with its status:

kubectl get resourcedistributionbindings -n payments \
  -l spaces.upbound.io/resource-distribution=team-a-buckets

Creating means a manifest waits for something, such as a missing namespace. Unavailable needs a fix:

Message contains	Cause	Fix
is attempting to grant RBAC permissions not currently held	The role grants something controlplane-admin lacks there, usually a missing provider's resources.	Install the provider, or stop selecting that control plane.
already exists and is not managed by this distribution	Another ResourceDistribution or a user owns that object.	Rename it, or adopt it with the label the message names.