WorkflowGen Helm Chart
Overview
This section presents the WorkflowGen Helm chart including usage, configuration options, and examples. Using the WorkflowGen chart simplifies the deployment of WorkflowGen in a Kubernetes cluster by not having to manage many Kubernetes deployment files. You only have to manage one "values" file.
Prerequisites
You must have a working Kubernetes cluster with Windows Server 2019 nodes.
You must have installed the kubectl command line tool and it must be connected to the cluster.
You must have installed the helm command line tool. See the Installing Helm section on the Helm website for instructions on how to install it. Only the Helm version 3.0+ is supported.
Helm chart primer
From the Helm website:
A chart is a collection of files that describe a related set of Kubernetes resources.
These resources are written in YAML in the chart with the help of the Go templating language. This enables the chart to generate valid Kubernetes resource definition files based on values provided by the user of the chart. Therefore, when you use the chart to install or upgrade WorkflowGen, you can provide some values that will help you deploy the correct resources for WorkflowGen.
A chart has a manifest file called Chart.yaml
. It has a version for the chart and a version for the application. For example, the chart version of WorkflowGen could be 0.0.3
and its application version 7.18.3
. A chart also has a kubeVersion
that tells which versions of Kubernetes are supported. In the case of WorkflowGen, only versions 1.14 and higher are supported since it is the first version to include Windows containers support.
You install a chart using the command line. For example:
This will install the chart that is at the path ./chart-path
in the cluster with the release name release-name
. It also sets a the value image.tag
to 7.15.5-win-ltsc2019
. More information about the install command including its --set
parameter can be found in Helm Install on the Helm website.
While useful for a few parameters, setting values from the command line can be cumbersome and prone to errors, which is why you can also set values from a YAML or JSON file. The first thing you need to do is to create the file:
Then, you can pass that file to the install command:
WorkflowGen values
The following are all of the value names supported by the WorkflowGen chart and their default values, accompanied by brief descriptions.
The chart is divided in groups of configuration:
Global
WorkflowGen
WorkflowGen Windows services
Database
Ingress
Hooks
Global values
Global values control the overall deployment that results from the Helm generation. For example, the scalable
value indicates if the resulting deployment should be scalable or not. If true
, the resulting deployment will have a separate pod for the WorkflowGen Windows services that will be deployed using the Singleton deployment pattern. The WorkflowGen web services will be deployed with the number of replicas indicated by the replicaCount
global value.If false
, WorkflowGen web and Windows services will be deployed in a single pod using the Singleton pattern.
To deploy multiple WorkflowGen instances that must be isolated, the use of the tenantName
value is recommended because it will prefix each object created by the installation with the name of the tenant and add a selector label named workflowgen.com/tenant=tenantName
.
WorkflowGen & Windows services
The WorkflowGen part groups the configuration options related to the WorkflowGen web and Windows services pods when configuring the WorkflowGen product. For infrastructure-related configuration, configuration options are in separate groups in order to be able to properly deploy each part individually but still having the same WorkflowGen-specific configuration.
WorkflowGen configuration
There are two main ways to configure WorkflowGen with the chart: with the values file or by providing your own ConfigMap or Secret.
In your values file, you can use the sub YAML object workflowgen.config
and workflowgen.secret
to configure WorkflowGen's environment variables. Let's begin with an example:
Each key and value in the config
part will be added as in a ConfigMap object and will be used by the WorkflowGen deployments. Concretely, the keys and values of the ConfigMap will be injected as environment variables in WorkflowGen's container. Each value must be of the string type. Therefore, YAML boolean values such as true
, yes
, and Y
will be rejected. Use single or double quotes when needed, as in the example.
For secret values, you can do exactly like the config
part. You only have to provide the name of the secret and its concrete value. The chart will add them to a Secret object and encode the value in Base64. The Secret object will then get injected as a volume inside WorkflowGen's container. Each key will become a file with its concrete value written inside it. Therefore, you need a corresponding config
value that will tell the container to pick up the value in a specific file using the _FILE
suffix. For more information about how to configure WorkflowGen including secrets, see the Configuration page in the WorkflowGen Image section. The default location of the secrets inside the container is C:\secrets
. You can customize this path by providing the value workflowgen.secretMountPath
.
File storage
The chart generates a PersistentVolumeClaim (PVC) object based on values that you've provided. As with the WorkflowGen configuration, you can specify your own PVC outside of the chart and reference it.
In your values file, you can use the sub YAML object workflowgen.dataPvcSpec
to configure the PersistentVolumeClaim for WorkflowGen's data (App_Data
and wfapps
). Let's begin with an example:
The content of the object is exactly the Kubernetes PersistentVolumeClaim specification. What you write in there will be taken as-is in the object definition.
In this example, the storage class azurefile
is specific to Azure Kubernetes Service.
License
The license must be stored on the Kubernetes cluster before installing the chart. This is because it's more efficient to store it in a secret and inject it as a volume in the WorkflowGen pods instead of provisioning a file share and connecting it to the pods. For the chart to handle it, you need to specify the secret name where the license is stored and the license item name in the secret. Here's an example:
Step 1: Store your license in the cluster
The secret object must be in the same namespace as the WorkflowGen pods.
This command will create a secret named wfgen-license-secret
with an item named WFG.lic
and its value will be the content of the file.
Step 2: Reference this secret in the values file
For more information about how to inject files from a Secret object into pods, see Secrets in the Kubernetes documentation.
Custom image reference
To use your own WorkflowGen image, you can change the default reference like this:
Service
There is a service that is created by the chart with the WorkflowGen pod for its discovery. By default, the chart will create a ClusterIP
service that provides a cluster-wide IP address and domain name that you can reference from anywhere in the cluster. This works best with an Ingress controller to route external traffic to it. For more information about Ingress controllers, see Ingress Controllers in the Kubernetes documentation.
You can customize this to automatically create a load balancer instead by providing the following value:
The LoadBalancer
type only works with cloud providers. For on-premise clusters, you should use another technique.
Security
There are many security features that aren't yet supported, or don't work in Windows but do work in Linux. It's important to plan the security of the deployments. To find out more about unsupported security features on Windows containers in Kubernetes, see the Security section in the Windows in Kubernetes documentation.
The web applications in the WorkflowGen container run as a user that is part of the IIS_IUSRS
group. This is important for setting permissions for file storage. If this group doesn't have MODIFY permission on the files volume, the container will fail to write to the volume. Generally, you can set mount options for a persistent volume depending on the storage provider or you can run an init container to set the permissions. To get the mount options for Azure Files, see the Mount options section in the Create and use a volume with Azure Files in Azure Kubernetes Service (AKS) Microsoft documentation.
Infrastructure-related recommendations
There are many more options for customizing your Helm release. The majority of them are dependent on your cluster environment. They are all Kubernetes terms, so you can search for them in a search engine and get useful information. The only one that this section will discuss is resources
. The resources
YAML object allows you to limit and request resource consumption for a given pod. Requests are to help the scheduler to assign pods to nodes. Limits are to limit the amount of resources the pod can use. The following requests and limits are known to work well for WorkflowGen pods:
Keep in mind that Windows containers are a lot bigger in terms of storage, CPU, and memory consumption. Therefore, you'll probably need bigger Windows nodes than your Linux ones.
Database
The database part contains the values used to generate the deployment files for the WorkflowGen database StatefulSet and its related objects. It is an optional feature to deploy a database pod along with a WorkflowGen pod. You can disable the creation of the StatefulSet by setting the following values:
Configuration
There are two main ways to configure the database with the chart: with the values file or by providing your own ConfigMap or Secret.
In your values file, you can use the sub YAML object database.config
and database.secret
to configure the environment variables of the database. Let's begin with an example:
Each key and value in the config
part will be added as is in a ConfigMap object. It will be used by the StatefulSet. Concretely, the keys and values of the ConfigMap will get injected as environment variables in the database container. Each value must be of the string type. Therefore, YAML boolean values such as true
, yes
and Y
will be rejected. Use single or double quotes when needed just like in the example.
For secret values, you can do exactly like the config
part. You only have to provide the name of the secret and its concrete value. The chart will add them to a Secret object and encode the value in Base64. The Secret object will then get injected as a volume inside the database container. Each key will become a file with its concrete value written inside it. Therefore, you need a corresponding config
value that will tell the container to pickup the value in a specific file using the _FILE
suffix. For more information about how to configure WorkflowGen including secrets, see the Configuration page of the WorkflowGen database section. The default location of the secrets inside the container is /mnt/secrets
. You can customize this path by providing the value database.secretMountPath
.
The WorkflowGen database image is based on the SQL Server Linux image. It is the recommended image to use for production workloads. The Windows version of the image should only be used for development and test environments.
File storage
A StatefulSet needs a PersistentVolumeClaim template to generate a volume claim for each of its replicas. You can't use your own PVC this time because it is a template, not a concrete object. This template is part of the StatefulSet specification. Here's an example:
The content of the template is exactly the Kubernetes PersistentVolumeClaim spec. Anything you write will be added to the StatefulSet specification as-is. In this example, the access mode is ReadWriteOnly
because the default
storage class refers to an Azure Disk. Azure disks can only be bound to a single node and pod at a time. Physical disks are the preferred way to store database files for better performance. For more information about Azure Disks, see the Create and use a volume with Azure Disks in Azure Kubernetes Service (AKS) Microsoft documentation.
Custom image reference
To use your own WorkflowGen database image, you can change the default reference like this:
Service
There is a service that is created by the chart with the database pod for its discovery. By default, the chart will create a ClusterIP
service without a cluster IP address (clusterIP: None
). This is called a headless service. It provides a way to get a cluster wide domain name to each pod of the StatefulSet. Therefore, you can refer directly to the database instance of your choice. It is recommended to leave the default values untouched. They are available in case you want to customize the service further. For more information about headless services in Kubernetes, see Service in its documentation.
Additionally, to make domain names predictable, you might want to override the full name of the StatefulSet, because the name is generated based on the chart's name and the release name, and you might not know the release name in advance. In this case, you can do the following:
This will ensure that each pod in the StatefulSet has a unique domain name based on this name. If this release is in the default
namespace, the first pod will have the domain name my-database-0.my-database.default.svc.cluster.local
. You then reference this domain name in the connection string of the WorkflowGen configuration at port 1433
and it should connect successfully.
Security
Since the database image used is a Linux image by default, all of the security features are available. You can run the database with a non-root user and group. For more information about running the database container as a non-root user, see Configure SQL Server container images on Docker in the SQL Server documentation. Keep in mind that you also have to setup the permissions for the storage to be able to read and write into it.
For more information about Docker security features, see Docker security in the Docker documentation. To find out how to configure these security features in Kubernetes, see Configure a Security Context for a Pod or Container in the Kubernetes documentation.
Infrastructure-related recommendations
There are many more options you can customize your Helm release with. The majority of them are dependent on your cluster environment. They are all Kubernetes terms, so you can search for them in a search engine and get useful information. The only one that this section will discuss is resources
. The resources
YAML object allows you to limit and request resource consumption for a given pod. Requests are to help the scheduler to assign pods to nodes. Limits are to limit the amount of resources the pod can use. The following requests and limits are known to work well for database pods:
Since WorkflowGen can only handle a main connection string and a read-only replica, you may want to limit the number of pods in the StatefulSet to two and scale vertically if you need more performance.
Ingress
The ingress section of the values is a simplified view of the complete Ingress object specification. It is optional to generate the Ingress rule object. To disable it, you can add the following values:
Otherwise, here's an example of how to use the Ingress section:
This is a complete example on how to use it. In this example, there are annotations set to add some information on how to handle them. The ingress.class
tells Kubernetes to use the Nginx ingress controller to handle this Ingress rule. The Nginx ingress controller must be installed in your cluster for the routing to work. For more information about the Nginx ingress controller, see the open source community project page or the commercial page. The two have different feature sets and are not developed by the same entity. The cert-manager.io/cluster-issuer
annotation is a Cert-Manager specific annotation that tells it to use the letsencrypt
cluster issuer for the TLS certificate. See the TLS/SSL page of this section for more information about cert-manager.
The hosts
section is a list of domain names and where they lead in the container. In this case, when a user goes to myinstance.example.com
, it will be routed to /
on the WorkflowGen container which IIS will handle. The tls
section tells what secret to use for what host. TLS certificates are stored in this secret.
Hooks
In Helm, there is a concept of hooks that enables the developer to deploy some resources (temporary or permanently) at different event in the lifecycle of a chart.
Pre-upgrade hook
The pre-upgrade hook deploys a Kubernetes Job that will add possible missing files and templates to your WorkflowGen database volume and migrate the database automatically. This deployment only occurs when you use the helm upgrade
command. It will wait for the job to finish successfully before upgrading the WorkflowGen and database pods. This hook is optional and you can disable it by using the following values:
This pre-upgrade hook uses the WorkflowGen upgrade image to perform migrations. Here's a complete example to how to configure the hook:
There is no default tag specified for the image because both Windows and Linux versions of it are production-ready. For a better experience and performance, it is recommended to use the Linux version. The secret
section of this example allows you to specify secret values to be put in a Secret object that will be deployed with the job. It will get deleted when the job is terminated. The name of the secret is generated from the release name. If your release name is wfgen
, then the secret name will be wfgen-migrations-secret
. The WFGEN_DATABASE_CONNECTION_STRING
secret is required. It is automatically injected as an environment variable in the container. To customize the name of the secret to use for the upgrade container'sWFGEN_DATABASE_CONNECTION_STRING
environment variable, you can populate the hooks.preupgrade.connectionStringSecretKey
value. You can add your own environment variables as well.
The last part consists of the arguments to pass to the container. See Configuration for more information about configuring the WorkflowGen upgrade container.
Common scenarios
Deploying a simple WorkflowGen pod
Overview
This deployment is meant for a simple installation with a database deployed outside of the cluster. It will deploy a single WorkflowGen pod with all of its services including WorkflowGen Windows services. This diagram shows a high-level view of the objects that will be created when installing the release with the values given in the next part. This architecture is only vertically scalable. You can only scale up the limits of the pod to have a more performant instance.
How to deploy
First, create the values file:
MY-WFG-LIC-NUMBER
is a placeholder value. You should replace it with your own serial number.These values will create a LoadBalancer-type service. If you are on a cloud provider, it will deploy a resource in the cloud provider's specific load balancer service (e.g. Azure Load Balancer, AWS Elastic Load Balancer, etc).
These values assume that you own the
example.com
domain name and that it points to the load balancer's public IP address. In this particular case, the HTTPS means that the load balancer should work at network layer 7 and provide TLS termination. For more information about TLS handling in Kubernetes see the TLS/SSL page of this section.The persistent volume claim that will be created assumes that you have a storage class already present in the cluster named
azurefile
. It is present by default on a Azure Kubernetes Service cluster. See the Create and use a volume with Azure Files in Azure Kubernetes Service (AKS) Microsoft article for more information.
Before installing a release of the chart, you have to create the WorkflowGen license secret object in your cluster. For this particular example, the name of the license file is WFG.lic
and the name of the secret object is wfgen-license-secret
. Execute the following command to create it:
With that done, you can now install the release:
The last argument is the path (or URL) to the WorkflowGen chart. You can use the URL directly or download it and use a local path. From this point, you should have a working WorkflowGen pod in your cluster.
Deploying a scalable WorkflowGen architecture
This architecture is best suited for production workloads that have an external database. This deployment enables you to scale WorkflowGen web applications horizontally (by adding replicas), which has many benefits such as increased availability and performance. The WorkflowGen Windows services must be scaled vertically and cannot be scaled horizontally. Make sure to deploy that pod to a node that has sufficient resources.
How to deploy
MY-WFG-LIC-NUMBER
is a placeholder value. You should replace it with your own serial number.These values will create a LoadBalancer-type service. If you are on a cloud provider, it will deploy a resource in the cloud provider's specific load balancer service (e.g. Azure Load Balancer, AWS Elastic Load Balancer, etc).
These values assume that you own the
example.com
domain name and that it points to the load balancer's public IP address. In this particular case, the HTTPS means that the load balancer should work at network layer 7 and provide TLS termination. For more information about TLS handling in Kubernetes see the TLS/SSL page of this section.The persistent volume claim that will be created assumes that you have a storage class already present in the cluster named
azurefile
. It is present by default on a Azure Kubernetes Service cluster. See the Create and use a volume with Azure Files in Azure Kubernetes Service (AKS) Microsoft article for more information.
This is the same example as for the simple deployment, except that the scalable
property is gone (true
by default), the number of replicas is three, and there are resource requests and limits for the Windows services.
Before installing a release of the chart, you have to create the WorkflowGen license secret object in your cluster. For this particular example, the name of the license file is WFG.lic
and the name of the secret object is wfgen-license-secret
. Execute the following command to create it:
With this done, you can now install the release:
The last argument is the path (or URL) to the WorkflowGen chart. You can use the URL directly or download it and use a local path. From this point, you should have a working WorkflowGen pod in your cluster.
Deploying a scalable WorkflowGen architecture with a database container
This architecture is best suited for the complete automation experience and can help reduce costs by having the container inside the cluster instead of in an external environment. It is a scalable architecture where the WorkflowGen web applications can be scaled horizontally and vertically, the Windows Services can be scaled vertically only, and the WorkflowGen database can be scaled horizontally up to two replicas (read/write and read-only) and vertically.
How to deploy
MY-WFG-LIC-NUMBER
is a placeholder value. You should replace it your own serial number.These values will create a LoadBalancer-typer service. If you are on a cloud provider, it will deploy a resource in the cloud provider's specific load balancer service (e.g. Azure Load Balancer, AWS Elastic Load Balancer, etc).
These values assume that you own the
example.com
domain name and that it points to the public IP address of the load balancer. In this particular case, the HTTPS means that the load balancer should work at network layer 7 and provide TLS termination. For more information about TLS handling in Kubernetes see the TLS/SSL page of this section.The persistent volume claim that will be created assumes that you have a storage class already present in the cluster named
azurefile
. It is present by default on a Azure Kubernetes Service cluster. See Dynamically create and use a persistent volume with Azure Files in Azure Kubernetes Service (AKS) for more information.The persistent volume claim template that is used in the database part uses a storage class called
default
. It is present by default on a Azure Kubernetes Service cluster and represents the Azure Disks service. See the Create and use a volume with Azure Files in Azure Kubernetes Service (AKS) Microsoft article for more information.
This is the same example as before, except that a database section has been added. The security context specifies that the container should run as root. This should be avoided as a general security good practice. It is there for simplicity. You should always use a different user than root and check the permissions on the volumes, for example with an init container.
Before installing a release of the chart, you have to create the WorkflowGen license secret object in your cluster. For this particular example, the name of the license file is WFG.lic
and the name of the secret object is wfgen-license-secret
. Execute the following command to create it:
With this done, you can now install the release:
The last argument is the path (or URL) to the WorkflowGen chart. You can use the URL directly or download it and use a local path. From this point, you should have a working WorkflowGen pod in your cluster.
Last updated