This feature is in preview.
For Connected and Disconnected Spaces, this feature requires Spaces v1.9.0
and is off by default. To enable, set features.alpha.spaceBackup.enabled=true
when installing Spaces:
up space init --token-file="${SPACES_TOKEN_PATH}" "v${SPACES_VERSION}" \
...
--set "features.alpha.spaceBackup.enabled=true"
Upbound’s Space Backups is a built-in Space-wide backup and restore feature. This guide explains how to configure Space Backups and how to restore from one of them in case of disaster recovery.
This feature is meant for Space administrators. Group or Control Plane users can leverage Shared Backups to backup and restore their ControlPlanes.
Benefits
The Space Backups feature provides the following benefits:
- Automatic backups for all resources in a Space and all resources in managed control planes, without any operational overhead.
- Backup schedules.
- Selectors to specify resources to backup.
Prerequisites
Enabled the Space Backups feature in the Space:
- Cloud Spaces: Not accessible to users.
- Connected Spaces: Space administrator must enable this feature.
- Disconnected Spaces: Space administrator must enable this feature.
Configure a Space Backup Config
SpaceBackupConfig is a cluster-scoped resource. This resource configures the storage details and provider. Whenever a backup executes (either by schedule or manually initiated), it references a SpaceBackupConfig to tell it where store the snapshot.
Backup config provider
The spec.objectStorage.provider
and spec.objectStorage.config
fields configures:
- The object storage provider
- The path to the provider
- The credentials needed to communicate with the provider
You can only set one provider. Upbound currently supports AWS, Azure, and GCP as providers.
spec.objectStorage.config
is a freeform map of configuration options for the object storage provider. See Thanos object storage for more information on the formats for each supported cloud provider. spec.bucket
and spec.provider
overrides the required values in the config.
AWS as a storage provider
This example demonstrates how to use AWS as a storage provider for your backups:
apiVersion: admin.spaces.upbound.io/v1alpha1
kind: SpaceBackupConfig
metadata:
name: default
spec:
objectStorage:
provider: AWS
bucket: spaces-backup-bucket
config:
endpoint: s3.eu-west-2.amazonaws.com
region: eu-west-2
credentials:
source: Secret
secretRef:
name: bucket-creds
namespace: upbound-system
key: creds
This example assumes you’ve already created an S3 bucket called
spaces-backup-bucket
in the eu-west-2
AWS region. To access the bucket,
define the account credentials as a Secret in the specified Namespace
(upbound-system
in this example).
Azure as a storage provider
This example demonstrates how to use Azure as a storage provider for your backups:
apiVersion: admin.spaces.upbound.io/v1alpha1
kind: SpaceBackupConfig
metadata:
name: default
namespace: default
spec:
objectStorage:
provider: Azure
bucket: upbound-backups
config:
storage_account: upbackupstore
container: upbound-backups
endpoint: blob.core.windows.net
credentials:
source: Secret
secretRef:
name: bucket-creds
namespace: upbound-system
key: creds
This example assumes you’ve already created an Azure storage account called
upbackupstore
and blob upbound-backups
. To access the blob,
define the account credentials as a Secret in the specified Namespace
(upbound-system
in this example).
GCP as a storage provider
This example demonstrates how to use Google Cloud Storage as a storage provider for your backups:
apiVersion: admin.spaces.upbound.io/v1alpha1
kind: SpaceBackupConfig
metadata:
name: default
namespace: default
spec:
objectStorage:
provider: GCP
bucket: spaces-backup-bucket
credentials:
source: Secret
secretRef:
name: bucket-creds
namespace: upbound-system
key: creds
This example assumes you’ve already created a Cloud bucket called
“spaces-backup-bucket” and a service account with access to this bucket. Define the key file as a Secret in the specified Namespace
(upbound-system
in this example).
Configure a Space Backup Schedule
SpaceBackupSchedule is a cluster-scoped resource. This resource defines a backup schedule for the whole Space.
Below is an example of a Space Backup Schedule running every day. It backs up all groups having environment: production
labels and all managed control planes in those groups having backup: please
labels.
apiVersion: admin.spaces.upbound.io/v1alpha1
kind: SpaceBackupSchedule
metadata:
name: daily-schedule
spec:
schedule: "@daily"
configRef:
kind: SpaceBackupConfig
name: default
match:
groups:
labelSelectors:
- matchLabels:
environment: production
controlPlanes:
labelSelectors:
- matchLabels:
backup: please
Define a schedule
The pec.schedule` field is a Cron-formatted string. Some common examples are below:
Entry | Description |
---|---|
@hourly | Run once an hour. |
@daily | Run once a day. |
@weekly | Run once a week. |
0 0/4 * * * | Run every 4 hours. |
0/15 * * * 1-5 | Run every fifteenth minute on Monday through Friday. |
@every 1h30m10s | Run every 1 hour, 30 minutes, and 10 seconds. Hour is the largest measurement of time for @every. |
Suspend a schedule
Use spec.suspend
field to suspend the schedule. It creates no new backups, but allows running backups to complete.
apiVersion: admin.spaces.upbound.io/v1alpha1
kind: SpaceBackupSchedule
metadata:
name: daily-schedule
spec:
suspend: true
...
Garbage collect backups when the schedule gets deleted
Set the spec.useOwnerReferencesInBackup
to garbage collect associated SpaceBackup
when a SpaceBackupSchedule
gets deleted. If set to true, backups are garbage collected when the schedule gets deleted.
Set the time to live
Set the spec.ttl
field to define the time to live for the backup. After this time, the backup is eligible for garbage collection. If this field isn’t set, the backup isn’t garbage collected.
The time to live is a duration, for example, 168h
for 7 days.
apiVersion: admin.spaces.upbound.io/v1alpha1
kind: SpaceBackupSchedule
metadata:
name: daily-schedule
spec:
ttl: 168h # Backup is garbage collected after 7 days
...
Selecting space resources to backup
By default, a SpaceBackup selects all groups and, for each of them, all managed control planes, secrets, and any other group-scoped resources.
By setting spec.match
, you can include only specific groups, control planes, secrets, or other Space resources in the backup.
By setting spec.exclude
, you can filter out some matched Space API resources from the backup.
Including space resources in a backup
Different fields are available to include resources based on labels or names:
spec.match.groups
to include only some groups in the backup.spec.match.controlPlanes
to include only some control planes in the backup.spec.match.secrets
to include only some secrets in the backup.spec.match.extras
to include only some extra resources in the backup.
apiVersion: admin.spaces.upbound.io/v1alpha1
kind: SpaceBackup
metadata:
name: my-backup
spec:
configRef:
kind: SpaceBackupConfig
name: default
match:
groups:
labelSelectors:
- matchLabels:
environment: production
controlPlanes:
labelSelectors:
- matchLabels:
backup: please
secrets:
names:
- my-secret
extras:
- apiGroup: "spaces.upbound.io"
kind: "SharedBackupConfig"
names:
- my-shared-backup
Excluding Space resources from the backup
Use the spec.exclude
field to exclude matched Space API resources from the backup.
Different fields are available to exclude resources based on labels or names:
spec.exclude.groups
to exclude some groups from the backup.spec.exclude.controlPlanes
to exclude some control planes from the backup.spec.exclude.secrets
to exclude some secrets from the backup.spec.exclude.extras
to exclude some extra resources from the backup.
apiVersion: admin.spaces.upbound.io/v1alpha1
kind: SpaceBackup
metadata:
name: my-backup
spec:
ttl: 168h # Backup is garbage collected after 7 days
configRef:
kind: SpaceBackupConfig
name: default
match:
groups:
labelSelectors:
- matchLabels:
environment: production
exclude:
groups:
names:
- not-this-one-please
Exclude resources in control planes’ backups
By default, it backs up all resources in a selected control plane.
Use the spec.controlPlaneBackups.excludedResources
field to exclude resources from control planes’ backups.
apiVersion: admin.spaces.upbound.io/v1alpha1
kind: SpaceBackup
metadata:
name: my-backup
spec:
ttl: 168h # Backup is garbage collected after 7 days
configRef:
kind: SpaceBackupConfig
name: default
controlPlaneBackups:
excludedResources:
- secrets
- buckets.s3.aws.upbound.io
Create a manual backup
SpaceBackup is a cluster-scoped resource that causes a single backup to occur for the whole Space.
Below is an example of a manual SpaceBackup:
apiVersion: admin.spaces.upbound.io/v1alpha1
kind: SpaceBackup
metadata:
name: my-backup
spec:
configRef:
kind: SpaceBackupConfig
name: default
deletionPolicy: Delete
The backup specification DeletionPolicy
defines backup deletion actions,
including the deletion of the backup file from the bucket. The Deletion Policy
value defaults to Orphan
. Set it to Delete
to remove uploaded files
in the bucket.
For more information on the backup and restore process, review the Spaces API documentation.
Set the time to live
Set the spec.ttl
field to define the time to live for the backup. After this time, the backup is eligible for garbage collection. If this field isn’t set, the backup isn’t garbage collected. The time to live is a duration, for example, 168h
for 7 days.
apiVersion: admin.spaces.upbound.io/v1alpha1
kind: SpaceBackup
metadata:
name: my-backup
spec:
ttl: 168h # Backup is garbage collected after 7 days
...
Restore from a space backup
Space Backup and Restore focuses only on disaster recovery. The restore procedure assumes a new Space installation with no existing resources. The restore procedure is idempotent, so you can run it multiple times without any side effects in case of failures.
To restore a Space from an existing Space Backup, follow these steps:
- Install Spaces from scratch as needed.
- Create a
SpaceBackupConfig
as needed to access the SpaceBackup from the object storage, for example namedmy-backup-config
. - Select the backup you want to restore from, for example
my-backup
. - Run the following command to restore the Space:
export SPACE_BACKUP_CONFIG=my-backup-config
export SPACE_BACKUP=my-backup
kubectl exec -ti -n upbound-system deployments/spaces-controller -c spaces -- hyperspace restore $SPACE_BACKUP $SPACE_BACKUP_CONFIG