BWS On-Premises Installation

Running BWS in Kubernetes

Of course there are many ways to install BWS in a Kubernetes cluster. Here we give an example using an Azure Kubernetes service cluster. Before you begin, please read through the prerequisites for BWS on-premises installations.

In the example we start with an existing Azure Kubernetes service cluster, that has some additional controllers installed (only required for secure connections):

the cert-manager together with a cluster-issuer (named letsencrypt, that we use to issue TLS certificates via Let's Encrypt).
the Kubernetes community NGINX ingress controller, that we use as proxy server to handle TLS and route requests, especially gRPC calls, to the specific services.

Secrets

For this example we need to configure two secrets, containing the MongoDB connection string and the Base64 encoded byte array used to encrypt/decrypt the internally used RSA private key. The YAML file (secrets.yaml) defining these secrets might look as follows:

apiVersion: v1
kind: Secret
metadata:
  name: bws-secrets
  namespace: bws
type: Opaque
stringData:
  mongodb_connectionstring: "mongodb://username:password@mongodb.mydomain.com:27017/bws?ssl=true"
  key_secret: "key encryption secret - Base64 encoded byte array"

Note: You define the secret (Base64 encoded byte array) used to encrypt your RSA private key. If no secret is configured, BWS uses a default secret. Feel free to use our BioID Playground API to generate Base64 encoded secrets.

To apply the secret configuration:

kubectl create namespace bws
kubectl apply -f secrets.yaml

Logging Volume

The BWS Service opionally logs images and some generated output to a mounted volume (/mnt/bwslogging by default). To mount this volume later, we specify a PersistentVolumeClaim (here we use a Azure File store):

cat <<EOF | kubectl apply -f -
apiVersion: v1
kind: PersistentVolumeClaim
metadata:
  name: bwslogging
  namespace: bws
spec:
  storageClassName: azurefile-csi
  accessModes: 
    - ReadWriteOnce
  resources:
    requests:
      storage: 100Gi
EOF

BWS Portal

Now you need access to the BWS Portal container image as described in the prerequisites. To deploy the image, apply the following YAML configuration, after you have changed the image container registry and version to your registry and the latest version:

apiVersion: apps/v1
kind: Deployment
metadata:
  name: bws-portal
  namespace: bws
spec:
  replicas: 1
  selector:
    matchLabels:
      app: bws-portal
  template:
    metadata:
      labels:
        app: bws-portal
    spec:
      containers:
        - name: bwsportal
          image: bioidcr.azurecr.io/bws/bwsportal-onprem:3.4.24309.2
          ports:
            - containerPort: 5001
              name: bws-portal-svc
          env:
            - name: ASPNETCORE_URLS
              value: "http://+:5001"
            - name: MongoDB__ConnectionString
              valueFrom:
                secretKeyRef:
                  name: bws-secrets
                  key: mongodb_connectionstring
            - name: KeyEncryption__Secret
              valueFrom:
                secretKeyRef:
                  name: bws-secrets
                  key: key_secret
          resources:
            requests:
              cpu: 10m
              memory: 128Mi
            limits:
              cpu: 500m
              memory: 256Mi
          volumeMounts:
            - mountPath: /mnt/bwslogging
              name: logging
      volumes:
        - name: logging
          persistentVolumeClaim:
            claimName: bwslogging
---
apiVersion: v1
kind: Service
metadata:
  name: bws-portal-service
  namespace: bws
spec:
  selector:
    app: bws-portal
  ports:
    - name: bws-portal-port
      protocol: TCP
      port: 80
      targetPort: bws-portal-svc
  type: LoadBalancer

If secure connections are not required in your setup, you can use LoadBalancer as service type above to expose the service externally and you should now be able to access the on-prem BWS Portal using the assigned IP address: $ip = $(kubectl get services bws-portal-service -n bws --output 'jsonpath={..status.loadBalancer.ingress[0].ip}')

Secure connection to the BWS Portal

Otherwise, if you want to secure the connection to the BWS Portal with TLS, use the ClusterIP service type and create an ingress to route external traffic to the BWS Portal. In this example we use a TLS certificate as issued by Let's Encrypt for the host bws-portal.yourdomain.com. Of course you can create a route without TLS if desired (by removing the annotations and the tls section).

The generated external IP address for this host needs to be added to the yourdomain.com DNS server of course.

apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
  annotations:
    nginx.ingress.kubernetes.io/ssl-redirect: "true"
    cert-manager.io/cluster-issuer: letsencrypt
  name: portal-tls
  namespace: bws
spec:
  ingressClassName: nginx
  rules:
  - host: bws-portal.yourdomain.com
    http:
      paths:
      - backend:
          service:
            name: bws-portal-service
            port:
              name: bws-portal-port
        path: /
        pathType: Prefix
  tls:
  - hosts:
    - bws-portal.yourdomain.com
    secretName: portal-cert

Now, as you have the BWS Portal up and running, go ahead and configure your BWS installation by using a browser and navigating to bws-portal.yourdomain.com.

Start BWS configuration

When you navigate to the BWS Portal web site for the first time, you have to register as a new user, or, if you have configured an OpenID Connect provider, you can login with your OpenID Connect account.

If the BWS Portal detects that the database does not contain a BWS configuration yet, an initialization process is started.

Step 1: Enter your company name

The user enters a name for the new BWS on-prem installation, e.g. the company name and click on 'Create On-Prem version'.

The following steps are now carried out automatically:

With this name, an installation, a subscription and a first BWS client is configured in the database.
The user creating these first entries automatically gets Admin privileges.

Step 2: Copy your Public Key to bwsportal.bioid.com

For the communication with BioID, the public key needs to be exported/copied and imported/pasted into the pre-configured on-prem installation client at the BioID BWS Portal (Cloud).

Import your public key by clicking on the 'Create initial protection key' button. The dialog opens, put your public key here and click on the button 'Send Public RSA Key to BioID …'. After a short moment your protection key is generated. Click the button 'Copy Protection Key' and go back to your local BWS Portal.

Step 3: Enter the Protection Key from bwsportal.bioid.com

Copy your Protection Key and finalize the BWS Configuration inside the BWS Portal.

The initial version of BWS should be configured now and you can continue to configure and start the BWSgRPC service.

If you have skipped the previous steps of Public/Protection keys, you can create/renew the license as follows.

Please note, that the newly created subscription has no permissions assigned yet. At least one user needs to have owner permissions to this subscription. Therefore you have to add yourself or another responsible person (which has to register at the on-prem BWS Portal first) as Owner to the subscription permissions.

BWS gRPC service

To install the BWS gRPC service, you need access to the BWS gRPC service container image as described in the prerequisites. To deploy the image, apply the following YAML configuration, after you have changed the image container registry and version to your registry and the latest version:

apiVersion: apps/v1
kind: Deployment
metadata:
  name: bws-grpc
  namespace: bws
spec:
  replicas: 2
  selector:
    matchLabels:
      app: bws-grpc
  template:
    metadata:
      labels:
        app: bws-grpc
    spec:
      containers:
        - name: bws
          image: bioidcr.azurecr.io/bws/bwsgrpc-onprem:3.4.24309.2
          resources: # TODO
            requests:
              memory: "2Gi"
              cpu: "1"
            limits:
              memory: "4Gi"
              cpu: "4"
          ports:
            - containerPort: 50051
              name: bws-grpc-svc
            - containerPort: 8080
              name: bws-http1-svc
          livenessProbe:
            grpc:
              port: 50051
              service: liveness
            initialDelaySeconds: 10   
            periodSeconds: 20
          readinessProbe:
            grpc:
              port: 50051
              service: readiness
            initialDelaySeconds: 10   
            periodSeconds: 5
          env:
            - name: Kestrel__Endpoints__gRPC__Protocols
              value: Http2
            - name: Kestrel__Endpoints__gRPC__Url
              value: http://*:50051
            - name: Kestrel__Endpoints__RESTful__Protocols
              value: Http1
            - name: Kestrel__Endpoints__RESTful__Url
              value: http://*:8080
            - name: MongoDB__ConnectionString
              valueFrom:
                secretKeyRef:
                  name: bws-secrets
                  key: mongodb_connectionstring
            - name: ClientService__Secret
              valueFrom:
                secretKeyRef:
                  name: bws-secrets
                  key: key_secret
          volumeMounts:
            - mountPath: /mnt/bwslogging
              name: logging
      volumes:
        - name: logging
          persistentVolumeClaim:
            claimName: bwslogging
---
apiVersion: v1
kind: Service
metadata:
  name: bws-service
  namespace: bws
spec:
  selector:
    app: bws-grpc
  ports:
    - name: bws-grpc-port
      protocol: TCP
      port: 50051
      targetPort: bws-grpc-svc
    - name: bws-http1-port
      protocol: TCP
      port: 80
      targetPort: bws-http1-svc
  type: ClusterIP

Secure connection to BWS

Again, if a secure connection to the BWS on-prem service is required, you can create an ingress and route external traffic to the BWS gRPC service. Please take notice of the first two annotations, which instructs NGINX to communicate with the BWS backend service using gRPC via HTTP/2 and sets the maximum allowed size of the client request to the 50MB as also used by the BWS. If you plan to use the RESTful API only, you can remove these annotations and use the bws-http1-port.

In this example again we use a TLS certificate as issued by Let's Encrypt for the host bws.yourdomain.com. The generated external IP address for this host needs to be added to the yourdomain.com DNS server of course.

apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
  annotations:
    nginx.ingress.kubernetes.io/backend-protocol: GRPC
    nginx.ingress.kubernetes.io/proxy-body-size: 50M
    nginx.ingress.kubernetes.io/ssl-redirect: "true"
    cert-manager.io/cluster-issuer: letsencrypt
  name: bws-tls
  namespace: bws
spec:
  ingressClassName: nginx
  rules:
  - host: bws.yourdomain.com
    http:
      paths:
      - backend:
          service:
            name: bws-service
            port:
              name: bws-grpc-port
        path: /
        pathType: Prefix
  tls:
  - hosts:
    - bws.yourdomain.com
    secretName: bws-cert

HTTP/2 and HTTP/1.1

As the BWS service allows RESTful API calls and also utilizes the gRPC framework, which uses HTTP/2 based transport, the service supports the HTTP/2 and the HTTP/1.1 protocols by default. Therefore it might come to problems when using unsecure connections (plain http), that cannot negotiate the connection protocol, in contrast to TLS clients that run ALPN (Application-Layer Protocol Negotiation).

When you come across those kind of problems, consider to explicitely use the ports configured for the specific protocols (bws-grpc-svc and bws-http1-svc above).

Health Checks

The BWS gRPC service supports the gRPC health checking protocol (health/v1) and also some classic Docker health checks:

gRPC liveness health check against the server by using the service name liveness
gRPC readiness health check against the BWS API service by using the service name readiness
Docker liveness health check at endpoint /livez
Docker readiness health check at endpoint /readyz
classic Docker health check at endpoint /healthz

Testing

When everything went fine, we can now test our BWS installation using the BWS 3 command line interface (as downloadable here), e.g. bws healthcheck --host https://bws.yourdomain.com

Browse docs