Skip to content

UDS Packages

A UDS Package is a Zarf Package with two additions:

  1. It is meant to be deployed on top of UDS Core.
  2. It contains the UDS Package Kubernetes custom resource.

These packages include all the OCI images (docker containers), Helm charts, and supplemental Kubernetes manifests required for the app to communicate with UDS Core. The UDS Operator in turn auto-applies appropriate security and network policies to assure a secure and compliant running environment. A UDS package does not include dependencies like databases or object storage*. These external dependencies are deployed next to a UDS Package inside a UDS Bundle.

To move from the theoretical to the concrete, see the next section on the anatomy of a UDS Package repo.

Anatomy of a UDS Package Repo

Disclaimer: the exact file structure of UDS Packages is subject to change. This document will fall out of date but should retain conceptual accuracy. After understanding this point-in-time snapshot of how a UDS package is built, it should be fairly trivial to extend that knowledge to grasp UDS packages as improved in the interim. To aid in it’s utility as a teaching tool, links to source code are pinned to a specific GitLab Package release.

For an in-depth developer-focused treatment of UDS Packages, see the documentation in GitHub. You can also view the UDS package template in GitHub here. This document will go over the main components of a UDS package and their functions at an overview level, and then show specifically how these components are tied together in the case of GitLab.

Anatomy Overview

Directory / Top-level fileRoleFunction
.github/CI/CDDirectives to GitHub, primarily it contains the build, test, and release pipeline(s).
adr/Docs”ADR” stands for Architectural Decision Records. These documents record key architectural decisions and their reasoning.
bundle/Testing & DevelopmentWhen you’re testing a UDS Package, you need to be able to deploy it with other applications such as databases in order to test your configuration. The bundle/ directories in UDS Package repos are for just that. They deploy light-weight databases, key-value stores, and object stores as needed alongside the application to permit testing. They also serve as an example of how to use the UDS Package in a bundle. Nothing in the bundle/ directory ever becomes part of the UDS Application Package.
charts/UDS Package ComponentThis is for helm charts which are created supplementally to the application’s helm chart. This includes at minimum the UDS Package manifest and the SAML/OIDC configuration for automatic integration with our Keycloak SSO application (which is part of UDS Core). Not infrequently it will also include another resource or two as needed to fully integrate into the UDS ecosystem on an app-specific basis.
common/UDS Package ComponentThis directory holds a single zarf.yaml file which is the base Zarf package definition. It is imported by the root-level zarf.yaml. You can think of it like the parent-class object in an object-oriented-programming model. This generally pulls the charts from the charts/ directory and the main application’s helm chart into the Zarf package but leaves flavor specific details out.
docs/DocsDocumentation about the UDS Package.
src/Testing & DevelopmentThis contains additional zarf package source code in each leaf-directory. These are never made into a part of the UDS Package. Rather, they are included in the test bundle to help glue the application to the larger ecosystem. This often includes a zarf package that contains only the application’s namespace resource. By putting this in a separate zarf package and deploying it ahead of time, secrets deployed to the application’s namespace by other packages (such as authentication secrets) do not get deleted when you run zarf package remove <app package>. This will be revisited when discussing the bundle/ directory in the GitLab repo below.
tasks/Testing & DevelopmentThese tasks run via the UDS CLI which uses the maru task runner under the surface to perform workflows like “build, deploy, test” (normally called dev) or “publish built artifact”. These tasks are a mixture of bespoke repo-specific tasks and included tasks from the uds-common repo. The entrypoint to these tasks is the top-level tasks.yaml file which is to the UDS CLI what a Makefile is to GNU make or as a Rakefile is to Ruby. These tasks are also executed as part of the CI/CD pipeline defined in .github/.
tests/Testing & DevelopmentThis contains files related to the playbook tests which are used to verify that the application in a UDS Package appears to be working as configured in the test bundle (in bundle/). These tests are integration-level tests focused on validating connections between the application and the UDS ecosystem.
values/UDS Package ComponentThis directory typically contains four helm values.yaml files which are fed into the main application’s helm chart to configure it. Of note, the common-values.yaml file contains all configuration common to all deployment flavors and the <flavor>-values.yaml files contain the image URLs for the given value and any configuration changes required to make it work with this specific flavor of images.
misc top-level filesLicensing, tool configurations, etc.Most of the top-level files are self-explanatory or largely irrelevant to most users. The remainder of this table will discuss only the most important ones.
tasks.yamlTesting & DevelopmentAs mentioned when discussing the tasks/ directory. This file defines all tasks accessible to the command uds run <task>. View them by running uds run --list.
zarf.yamlUDS Package ComponentThis is the Zarf package which is in this case also a UDS Package. If this were a code repository, this file would be the main function. It defines all top-level Zarf variables, and then includes one component per flavor, each component importing the common/zarf.yaml package. Each component (which stands for a package flavor) adds the values/<flavor>-values.yaml file to set the images to the desired flavor in the helm chart and lists the needed images so Zarf can pull them down and add them to the package at build time. These components are turned on or off by the “flavor” variable value at build time producing only one of the components at any time in the final UDS Package.

These directories may be easiest understood through a detailed example.

GitLab’s UDS Package Anatomy

GitLab is the cornerstone application in the UDS Software Factory. You can view the UDS Package for GitLab on Defense Unicorn’s GitHub. This dive into UDS Package anatomy will use the repo as it existed in release v17.3.6-uds.1. That is, GitLab version 17.3.6, second package release for that version (zero-indexed).

Rather than reviewing the repo according to the alphabetical ordering of it’s directories as in the table above, components of the repo are discussed in terms of how they build to produce a UDS Package, and not all components will receive the same screen-time, rather, what follows is something of a guided tour through the repository.

GitLab’s UDS Package Components

common/zarf.yaml

Starting with common/zarf.yaml we have the base ZarfPackageConfig. It is reprinted in abbreviated form below with comments added for clear in-line explanation.

kind: ZarfPackageConfig # A UDS Package is just a kind of Zarf Package
metadata:
name: gitlab-common
description: "UDS GitLab Common Package"
# Recall later that all three helm charts below are part of a single zarf component, GitLab.
components:
- name: gitlab
required: true
charts: # These charts are deployed in the order listed, use this to your advantage
# Adds SSO, postgre, and redis auth secrets which are expected (required) by the main
# app (if missing, will break the deployment). Also includes the UDS Package resource
# which makes this Zarf Package also a UDS Package.
- name: uds-gitlab-config
namespace: gitlab
version: 0.2.0
localPath: ../charts/config # Note the filepath, this references charts/config
# The GitLab application
- name: gitlab
namespace: gitlab
# The upstream helm chart. Note the repo url given is NOT the git repo which is
# https://gitlab.com/gitlab-org/charts/gitlab. Googling can be required to connect
# a helm chart repo to it's source code repo. Some UDS Packages will use the Big
# Bang helm chart instead, or, rarely, a Defense Unicorns produced helm chart.
url: https://charts.gitlab.io/
gitPath: chart
version: "8.5.1"
valuesFiles:
# Note we pull in the common values file. The top-level zarf.yaml will add overrides
# for the desired flavor
- ../values/common-values.yaml
# If you inspect https://github.com/defenseunicorns/uds-package-gitlab/blob/v17.3.6-uds.1/charts/settings/templates/_settings-pod.tpl
# you will see this chart creates a job that further configures GitLab in ways that
# could not be done via Helm at deploy-time.
- name: uds-gitlab-settings
namespace: gitlab
version: 0.1.0 # These "helper" charts rarely change version. Often, they are
# forever this version, 0.1.0.
localPath: ../charts/settings # Note the filepath, this references charts/settings
actions:
onDeploy:
after:
...
# multiple zarf jobs to ensure each part of GitLab is happy after deployment
./zarf.yaml

If we go now to the root-level zarf.yaml file we can see the Zarf variables and flavors get added in.

kind: ZarfPackageConfig # As before, a UDS Package is just a Zarf Package with made for UDS
metadata:
name: gitlab
description: "UDS GitLab Package"
version: "17.3.6-uds.1" # As mentioned earlier, this document is locked to a specific
# version for consistency.
# These variables are given values within the UDS ecosystem in one of two ways: either they
# are given a value in the UDS bundle definition (if the correct value can be known at bundle
# build time) or in the UDS configuration (use only if the correct value cannot be known
# until deploy time).
variables:
- name: DOMAIN
default: "uds.dev"
- name: BUCKET_PREFIX # Incidentally, buckets refer to object storage such as S3.
default: "uds-"
- name: BUCKET_SUFFIX
default: ""
- name: GITLAB_REDIS_ENDPOINT # Redis is always replaced with valkey in a UDS deployment
default: ""
- name: GITLAB_REDIS_SCHEME
default: "redis"
- name: GITLAB_DB_NAME # This refers to the Postgresql db.
default: "gitlabdb"
- name: GITLAB_DB_USERNAME
default: "gitlab"
- name: GITLAB_DB_ENDPOINT
default: "postgresql"
- name: GITLAB_PAGES_ENABLED # A GitLab documentation feature
default: "false"
- name: GITLAB_SIGNUP_ENABLED # A GitLab feature related account creation
default: "true"
- name: DISABLE_REGISTRY_REDIRECT
default: "false"
description: "If your storage endpoint is not publicly accessible set this to true"
components:
# For each component, notice the "only" keyword and how each component requires a
# different flavor. Consequently, only one of the following components are included.
# Notice too, that each one imports the gitlab component from the common/zarf.yaml
# file discussed above, and so automatically includes both "helper" charts (from charts/).
# Registry1 flavor as shown by the 'only' block.
- name: gitlab
required: true
description: "Deploy gitlab with registry1 images"
import:
path: common # The crucial import
only:
flavor: registry1 # This is the Ironbank flavor
cluster:
architecture: amd64 # The architecture _must_ be amd64 because Ironbank does not have
# all the ARM-based images required for GitLab.
charts:
- name: gitlab
valuesFiles:
- values/registry1-values.yaml # Add the registry1-values.yaml file to further
# configure the chart and override values in the
# values/common-values.yaml file if necessary.
- name: uds-gitlab-settings
valuesFiles:
# Because the gitlab settings setting job runs in an image, also grab the Ironbank
# version of the required image. See:
# https://github.com/defenseunicorns/uds-package-gitlab/blob/v17.3.6-uds.1/charts/settings/values.yaml#L8
- values/registry1-values.yaml
images:
# Here we list all the images referenced in the values.yaml files. This is how Zarf
# knows which images to pull down, package with the helm charts & manifests, and deploy
# to the image registry it will put in-cluster (this is part of how it enables
# air-gapped deployments). List abbreviated for space. For more, see:
# https://docs.zarf.dev/ref/components/#container-images
- "registry1.dso.mil/ironbank/gitlab/gitlab/certificates:17.3.6"
...
- "registry1.dso.mil/ironbank/gitlab/gitlab/gitlab-exporter:17.3.6"
# Upstream flavor as shown by the 'only' block.
- name: gitlab
required: true
description: "Deploy gitlab"
import:
path: common # The crucial import again
only:
flavor: upstream # This component is what we get if flavor is upstream.
# Note no processor architecture restrictions. The upstream Gitlab registry has ARM-
# based images.
charts:
- name: gitlab
valuesFiles:
- values/upstream-values.yaml # Using the upstream values file.
- name: uds-gitlab-settings
valuesFiles:
- values/upstream-values.yaml # Setting upstream image values similarly to the
# settings chart
images:
# The upstream image URLs, again abbreviated for the sake of space.
- "registry.gitlab.com/gitlab-org/build/cng/certificates:v17.3.6"
...
- "registry.gitlab.com/gitlab-org/build/cng/gitlab-exporter:v17.3.6"
# Unicorn flavor as shown by the 'only' block.
- name: gitlab
required: true
description: "Deploy gitlab with chainguard images"
import: # See this
path: common
only:
flavor: unicorn # See this
charts:
- name: gitlab
valuesFiles:
- values/unicorn-values.yaml # See this
- name: uds-gitlab-settings
valuesFiles:
- values/unicorn-values.yaml # And this
images:
# And finally the chainguard OCI image URLs
- "cgr.dev/du-uds-defenseunicorns/gitlab-certificates-fips:17.3.6"
...
- "cgr.dev/du-uds-defenseunicorns/gitlab-exporter-fips:17.3.6"

GitLab’s Testing & Development

As explained in the anatomy overview, the bundle/ directory contains a bundle using the GitLab UDS Package and serves two functions. First, it provides a way to deploy and test GitLab as configured by the UDS Package. Second, like any good test, it is a form of documentation showing how Gitlab may be connected into a bundle.

Bundle files get larger than the zarf.yaml files previously explored so this one will be more severely abbreviated.

uds-bundle.yaml excerpts:

kind: UDSBundle
metadata:
name: gitlab-test
description: A UDS bundle for deploying Gitlab and it's dependencies on a development cluster
version: 17.3.6-uds.1
# A UDS Bundle contains many UDS Packages. UDS Bundles never contain other bundles.
packages:
# This provides object storage with the AWS S3 API in-cluster. While useful for development
# and testing, this service is often provided by an off-cluster service in large deployments.
# Repo behind the package: https://github.com/defenseunicorns/uds-package-minio-operator
- name: dev-minio
repository: ghcr.io/defenseunicorns/packages/uds/dev-minio
ref: 0.0.2
# This is a bit of an odd duck. We create the namespace for GitLab ahead of time so if we run
# `zarf package remove gitlab` we don't lose the namespace. This keeps secrets generated by
# applications like postgres around so the re-deployed GitLab package can still discover how
# to authenticate with these services. This is frequently helpful.
- name: dev-namespace
path: ../ # Note the path, when `uds run dev` is executed, one of the build jobs will put
# the built zarf package defined in src/namespace/zarf.yaml in the root directory.
# This path anticipates this fact as the bundle is built after the zarf packages
# are built.x
ref: 0.1.0
# This provides a postgres database in cluster.
# Repo behind the package: https://github.com/defenseunicorns/uds-package-postgres-operator
- name: postgres-operator
repository: ghcr.io/defenseunicorns/packages/uds/postgres-operator
ref: 1.13.0-uds.2-upstream
overrides:
postgres-operator:
uds-postgres-config:
values:
- path: postgresql
value:
enabled: true
teamId: "uds"
volume:
size: "10Gi"
numberOfInstances: 2
users:
gitlab.gitlab: [] # heads up, usernames are always namespace.app
databases:
gitlabdb: gitlab.gitlab
version: "14"
ingress:
- remoteNamespace: gitlab
# This is our Redis replacement because it's totally open source.
# Repo behind the package: https://github.com/defenseunicorns/uds-package-valkey
- name: valkey
repository: ghcr.io/defenseunicorns/packages/uds/valkey
ref: 7.2.7-uds.0-upstream
overrides:
valkey:
uds-valkey-config:
values:
- path: custom
value:
- direction: Ingress
selector:
app.kubernetes.io/name: valkey
remoteNamespace: gitlab
port: 6379
description: "Ingress from GitLab"
- path: copyPassword # This is how we get the access secret where GitLab expects it.
value:
enabled: true
namespace: gitlab
secretName: gitlab-redis
secretKey: password
# This puts a secret in the gitlab namespace containing access information for MinIO (S3 mimic)
# It's in src/dev-secrets in the GitLab repo.
- name: dev-secrets
path: ../
ref: 0.1.0
# This is where we add the GitLab UDS Package to the bundle. Recall that it had two charts included:
# - https://github.com/defenseunicorns/uds-package-gitlab/tree/v17.3.6-uds.1/charts
# - https://github.com/defenseunicorns/uds-package-gitlab/blob/v17.3.6-uds.1/common/zarf.yaml#L14
# - https://github.com/defenseunicorns/uds-package-gitlab/blob/v17.3.6-uds.1/common/zarf.yaml#L25
- name: gitlab
# Note that the path for the GitLab Zarf package (which is a UDS package) is '../' same as for
# the dev-secret package because all zarf packages are built before the bundle is built, and put
# in the repo's root dir.
path: ../
ref: 17.3.6-uds.1 # FYI: versions tend to be <application version>-uds.<uds release of that version>
overrides:
gitlab:
# This is the chart in `charts/` which we're configuring here.
# Notice how we can hardcode a value to a path (under values:) OR we can add a variable
# and assign it to a path (everything under variables). These variables can be overridden
# at deploy-time via entries in the uds-config.yaml. Creating variables in your bundle
# is a super powerful way to expose parts of the underlying helm chart for deploy-time
# configuration. Full lists and dictionaries can be given as a variable value too, so
# it's very malleable.
uds-gitlab-config:
values:
- path: ssh.enabled
value: true
- path: ssh.port
value: 2223
variables:
- name: GITLAB_SSO_ENABLED
description: "Boolean to enable or disable sso things"
path: "sso.enabled"
...
- name: GITLAB_REQUIRED_GROUPS
description: "Array of group names that are required for GitLab acess."
path: "sso.requiredGroups"
gitlab:
values:
- path: gitlab.gitlab-shell.enabled
value: true
- path: global.shell.port
value: 2223
# The hostname is known after deploying it once and from knowing the namespace
# we're putting the operator in.
- path: global.psql.host
value: pg-cluster.postgres.svc.cluster.local
# The username is known from the yaml above. As warned there too, don't try to get
# away from the app.app usernames, or, if you're having problems, try app.namespace.
# See postgre operator package docs for more here, if you can avoid changing it,
# you'll avoid a set of problems.
- path: "global.psql.username"
value: "gitlab.gitlab"
# These hard-coded values are known via experience combined with what's given to the
# postgres operator above. The name of the secret the operator creates is predictable.
- path: "global.psql.password.secret"
value: "gitlab.gitlab.pg-cluster.credentials.postgresql.acid.zalan.do"
# This hostname is known based on experience (deploying once and seeing what the
# service name was) and knowing the namespace the valkey package is deploying too.
- path: global.redis.host
value: valkey-master.valkey.svc.cluster.local
variables:
- name: GITLAB_SSO_ENABLED
description: "Boolean to enable or disable sso things"
path: "global.appConfig.omniauth.enabled"
...
- name: SHELL_REPLICAS
description: "Gitlab Shell Min Replicas"
path: "gitlab.gitlab-shell.minReplicas"
uds-gitlab-settings:
values:
- path: settingsJob.application.enabled_git_access_protocol
value: all
variables:
- name: BOT_ACCOUNTS
description: "Bot Accounts to Create"
path: "botAccounts"

Footnotes

*object storage is the type of storage most commonly associated with AWS’ S3 service. In Azure it’s called Blob Storage, in Kubernetes it is typically MinIO, and it goes by various other names in other storage platforms.


Feedback

Was this page helpful?
Yes
No