Syncing S3 and GCS buckets as Flux sources isn’t about just copying files; it’s about enabling Flux, a GitOps tool, to pull Kubernetes manifests from object storage instead of just Git repositories. This means your infrastructure-as-code can live in S3 or GCS, allowing for different workflows, potentially faster updates, and leveraging existing object storage infrastructure.
Here’s a Flux configuration that uses an S3 bucket as a source. Notice the bucketName, bucketRegion, and authentication details.
apiVersion: source.toolkit.fluxcd.io/v1beta2
kind: Bucket
metadata:
name: my-s3-bucket
namespace: flux-system
spec:
interval: 5m0s
url: s3://my-flux-bucket/path/to/manifests
secretRef:
name: s3-credentials
suspend: false
And here’s a comparable configuration for GCS:
apiVersion: source.toolkit.fluxcd.io/v1beta2
kind: Bucket
metadata:
name: my-gcs-bucket
namespace: flux-system
spec:
interval: 5m0s
url: gs://my-flux-bucket/path/to/manifests
secretRef:
name: gcs-credentials
suspend: false
The core problem this solves is providing an alternative to Git for managing your Kubernetes manifests. While GitOps is traditionally Git-centric, object storage offers benefits like:
- Simpler Workflow for Non-Developers: Teams less familiar with Git branching strategies or PR workflows can manage manifests more directly.
- Leveraging Existing Infrastructure: If your organization already heavily uses S3 or GCS for data, logging, or backups, you can extend that to Kubernetes deployments.
- Potential for Faster Updates: In some scenarios, directly updating an object in a bucket can be quicker than committing and pushing to a Git repository, especially for large numbers of small files.
- Cost-Effectiveness: For certain usage patterns, object storage can be more cost-effective than maintaining a Git server or using managed Git solutions.
Internally, Flux interacts with these object storage services using their respective SDKs. When it’s time to reconcile, Flux fetches the specified object (or objects, if a "directory" is specified in the URL) from the bucket. It then processes these as if they were fetched from a Git repository, looking for Kubernetes manifest files (YAML, JSON, Helm charts). The interval in the spec dictates how often Flux checks the bucket for changes.
The url field is crucial. For S3, it follows the s3://<bucket-name>/<path> format. For GCS, it’s gs://<bucket-name>/<path>. The path can point to a specific file or a "directory" (which is just a prefix in object storage terminology). Flux will recursively fetch all objects within that prefix.
The secretRef points to a Kubernetes Secret object containing the authentication credentials. For S3, this typically includes AWS_ACCESS_KEY_ID and AWS_SECRET_ACCESS_KEY. For GCS, it would be a service account key in JSON format, usually named gcp_credentials or similar. Flux uses these credentials to authenticate with the respective cloud provider’s API.
The suspend field allows you to temporarily halt reconciliation for this source. Setting it to true stops Flux from polling the bucket.
One of the most powerful, yet often overlooked, aspects of using object storage as a Flux source is its ability to work with versioning. If you enable versioning on your S3 or GCS buckets, Flux can, with a specific configuration, fetch older versions of your manifests. This is achieved by appending a version ID to the URL, though Flux’s Bucket controller doesn’t directly support specifying a version ID in the url field. Instead, you’d typically manage this by having different "paths" or "objects" representing different versions, and then updating the url in your Flux Bucket resource to point to the desired version. This offers a form of immutable deployment history directly within your object storage.
The next logical step after setting up object storage sources is to explore how to use them with Helm repositories, which can also be hosted in S3 or GCS.