BWS On-Premises Installation

Running BWS in Kubernetes

Of course there are many ways to install BWS in a Kubernetes cluster. Here we give an example using an Azure Kubernetes service cluster. Before you begin, please read through the prerequisites for BWS on-premises installations.

In the example we start with an existing Azure Kubernetes service cluster, that has some additional controllers installed:

the cert-manager together with a cluster-issuer (named letsencrypt, that we use to issue TLS certificates via Let's Encrypt).
the Kubernetes community NGINX ingress controller, that we use as proxy server to handle TLS and route requests, especially gRPC calls, to the specific services.

Secrets

For this example we need to configure two secrets, containing the MongoDB connection string and the Base64 encoded byte array used to encrypt/decrypt the internally used RSA private key. The YAML file (secrets.yaml) defining these secrets might look as follows:

apiVersion: v1
kind: Secret
metadata:
  name: bws-secrets
  namespace: bws
type: Opaque
stringData:
  mongodb_connectionstring: "mongodb://username:password@mongodb.mydomain.com:27017/bws?ssl=true"
  key_secret: "key encryption secret - Base64 encoded byte array"

Note: You define the secret (Base64 encoded byte array) used to encrypt your RSA private key. If no secret is configured, BWS uses a default secret. Feel free to use our BioID Playground API to generate Base64 encoded secrets.

To apply the secret configuration:

kubectl create namespace bws
kubectl apply -f secrets.yaml

Logging Volume

The BWS Service opionally logs images and some generated output to a mounted volume (/mnt/bwslogging by default). To mount this volume later, we specify a PersistentVolumeClaim (here we use a Azure File store):

cat <<EOF | kubectl apply -f -
apiVersion: v1
kind: PersistentVolumeClaim
metadata:
  name: bwslogging
  namespace: bws
spec:
  storageClassName: azurefile-csi
  accessModes: 
    - ReadWriteOnce
  resources:
    requests:
      storage: 100Gi
EOF

BWS Portal

Now you need access to the BWS Portal container image as described in the prerequisites. To deploy the image, apply the following YAML configuration, after you have changed the image container registry and version to your registry and the latest version:

apiVersion: v1
kind: Service
metadata:
  name: bws-portal-service
  namespace: bws
spec:
  type: ClusterIP
  selector:
    app: bws-portal
  ports:
    - name: bws-portal-port
      protocol: TCP
      port: 80
      targetPort: bws-portal-svc
---
apiVersion: apps/v1
kind: Deployment
metadata:
  name: bws-portal
  namespace: bws
spec:
  replicas: 1
  selector:
    matchLabels:
      app: bws-portal
  template:
    metadata:
      labels:
        app: bws-portal
    spec:
      containers:
        - name: bwsportal
          image: bioidcr.azurecr.io/bws/bwsportal-onprem:3.0.24045.1
          ports:
            - containerPort: 5001
              name: bws-portal-svc
          env:
            - name: ASPNETCORE_URLS
              value: "http://+:5001"
            - name: MongoDB__ConnectionString
              valueFrom:
                secretKeyRef:
                  name: bws-secrets
                  key: mongodb_connectionstring
            - name: KeyEncryption__Secret
              valueFrom:
                secretKeyRef:
                  name: bws-secrets
                  key: key_secret
          volumeMounts:
            - mountPath: /mnt/bwslogging
              name: logging
      volumes:
        - name: logging
          persistentVolumeClaim:
            claimName: bwslogging

As we now have created the BWS Portal workload and service, we can create the ingress and route external traffic to the BWS Portal. In this example we use a TLS certificate as issued by Let's Encrypt for the host bws-portal.yourdomain.com. Of course you can create a route without TLS if desired (by removing the annotations and the tls section).

The generated external IP address for this host needs to be added to the yourdomain.com DNS server of course.

apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
  annotations:
    nginx.ingress.kubernetes.io/ssl-redirect: "true"
    cert-manager.io/cluster-issuer: letsencrypt
  name: portal-tls
  namespace: bws
spec:
  ingressClassName: nginx
  rules:
  - host: bws-portal.yourdomain.com
    http:
      paths:
      - backend:
          service:
            name: bws-portal-service
            port:
              name: bws-portal-port
        path: /
        pathType: Prefix
  tls:
  - hosts:
    - bws-portal.yourdomain.com
    secretName: portal-cert

Now we should have the BWS Portal up and running. Go ahead and configure your BWS installation by using a browser and navigating to bws-portal.yourdomain.com.

Start BWS configuration

When the BWS Portal detects that the database does not contain a BWS configuration yet, an initialization process is started.

Step 1: Enter your company name

The user enters a name for the new BWS on-prem installation, e.g. the company name and click on 'Create On-Prem version'.

The following steps are now carried out automatically:

With this name, an installation, a subscription and a first BWS client is configured in the database.
The user creating these first entries automatically gets Admin privileges.

Step 2: Copy your Public Key to bwsportal.bioid.com

For the communication with BioID, the public key needs to be exported/copied and imported/pasted into the pre-configured on-prem installation client at the BioID BWS Portal (Cloud).

Import your public key by clicking on the 'Create initial protection key' button. The dialog opens, put your public key here and click on the button 'Upload Public RSA key'. After a short moment your protection key is generated. Click the button 'Copy Protection Key' and go back to your local BWS Portal.

Step 3: Enter the Protection Key from bwsportal.bioid.com

Copy your Protection Key and finalize the BWS Configuration inside the BWS Portal.

The initial version of BWS should be configured now and you can continue to configure and start the BWSgRPC service.

If you have skipped the previous steps of Public/Protection keys, you can create/renew the license as follows.

BWS gRPC service

To install the BWS gRPC service, you need access to the BWS gRPC service container image as described in the prerequisites. To deploy the image, apply the following YAML configuration, after you have changed the image container registry and version to your registry and the latest version:

apiVersion: v1
kind: Service
metadata:
  name: bws-service
  namespace: bws
spec:
  type: ClusterIP
  selector:
    app: bws-grpc
  ports:
    - name: bws-port
      protocol: TCP
      port: 80
      targetPort: bws-svc
---
apiVersion: apps/v1
kind: Deployment
metadata:
  name: bws-grpc
  namespace: bws
spec:
  replicas: 3
  selector:
    matchLabels:
      app: bws-grpc
  template:
    metadata:
      labels:
        app: bws-grpc
    spec:
      containers:
        - name: bwsonprem
          image: bioidcr.azurecr.io/bws/bwsgrpc-onprem:3.0.24045.1
          resources: # TODO
            requests:
              memory: "4Gi"
              cpu: "3"
            limits:
              memory: "6Gi"
              cpu: "4"
          ports:
            - containerPort: 50051
              name: bws-svc
          livenessProbe:
            grpc:
              port: 50051
              service: liveness
            initialDelaySeconds: 10   
            periodSeconds: 20
          readinessProbe:
            grpc:
              port: 50051
              service: readiness
            initialDelaySeconds: 10   
            periodSeconds: 5
          env:
            - name: ASPNETCORE_URLS
              value: "http://+:50051"
            - name: Serilog__MinimumLevel
              value: "Information"
            - name: Serilog__WriteTo__0__Name
              value: "Console"
            - name: MongoDB__ConnectionString
              valueFrom:
                secretKeyRef:
                  name: bws-secrets
                  key: mongodb_connectionstring
            - name: ClientService__Secret
              valueFrom:
                secretKeyRef:
                  name: bws-secrets
                  key: key_secret
          volumeMounts:
            - mountPath: /mnt/bwslogging
              name: logging
      volumes:
        - name: logging
          persistentVolumeClaim:
            claimName: bwslogging

Finally, we can create the ingress and route external traffic to the BWS gRPC service. Please take notice of the first two annotations, which instructs NGINX to communicate with the BWS backend service using gRPC via HTTP2 and sets the maximum allowed size of the client request to the 50MB as also used by the BWS. In this example again we use a TLS certificate as issued by Let's Encrypt for the host bws.yourdomain.com.

The generated external IP address for this host needs to be added to the yourdomain.com DNS server of course.

apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
  annotations:
    nginx.ingress.kubernetes.io/backend-protocol: GRPC
    nginx.ingress.kubernetes.io/proxy-body-size: 50M
    nginx.ingress.kubernetes.io/ssl-redirect: "true"
    cert-manager.io/cluster-issuer: letsencrypt
  name: bws-tls
  namespace: bws
spec:
  ingressClassName: nginx
  rules:
  - host: bws.yourdomain.com
    http:
      paths:
      - backend:
          service:
            name: bws-service
            port:
              name: bws-port
        path: /
        pathType: Prefix
  tls:
  - hosts:
    - bws.yourdomain.com
    secretName: bws-cert

HTTP/2

As the BWS gRPC service utilizes the gRPC framework, which uses HTTP/2 based transport, the service supports the HTTP/2 protocol and only HTTP/2. This is due to clients that might use unsecure gRPC (plain http), that cannot negotiate the connection protocol, in contrast to TLS clients that run ALPN (Application-Layer Protocol Negotiation).

Note: We currently have some issues using the NGINX ingress controller with unsecure gRPC connections, where we receive HTTP/2 protocol errors! We recommend to use TLS anyway.

Health Checks

The BWS gRPC service supports the gRPC health checking protocol (health/v1) and also some classic Docker health checks:

gRPC liveness health check against the server by using the service name liveness
gRPC readiness health check against the BWS API service by using the service name readiness
Docker liveness health check at endpoint /livez
Docker readiness health check at endpoint /readyz
classic Docker health check at endpoint /healthz

Testing

When everything went fine, we can now test our BWS installation using the BWS 3 command line interface (as downloadable here), e.g. bws healthcheck --host https://bws.yourdomain.com

Browse docs