Kubernetes / Helm Security Hardening

Development
1

Sup Seb,

I am in the process of writing a pull request to update your helm charts/templates to include the option to harden the security policy when deployed to a k8s cluster and I wanted to open this thread so I could ask you questions regarding the implementation of the product.

Background

I am currently in the process of hardening our AKS clusters ahead of a penetration test in the coming weeks and when trying to apply the Restricted Pod Security Standards I am finding that the squidex and squidex-mongodb pods are in breach of the policy.

Warning: existing pods in namespace "squidex" violate the new PodSecurity enforce level "restricted:latest"
Warning: squidex-6ccd696cd-lxvvf (and 2 other pods): allowPrivilegeEscalation != false, unrestricted capabilities, runAsNonRoot != true, seccompProfile
Warning: squidex-mongodb-replicaset-0 (and 2 other pods): allowPrivilegeEscalation != false, unrestricted capabilities, seccompProfile
namespace/squidex labeled

The key restrictions here that we need to apply are:

  • allowPrivilegeEscalation: false
  • unrestricted capabilities: all capabilities need to be dropped and only NET_BIND_SERVICE should be allowed
  • runAsNonRoot: true
  • secCompProfile: while can be set quite granularly, I have found that RuntimeDefault to be sufficient for now

The good news

I have managed to harden the squidex pod to follow the Restricted policy and no longer get a warning for that pod for any of the above reasons, however, the MongoDB replicaset is still reporting to be an issue.

Warning: existing pods in namespace "squidex" violate the new PodSecurity enforce level "restricted:latest"
Warning: squidex-mongodb-replicaset-0 (and 2 other pods): allowPrivilegeEscalation != false, unrestricted capabilities, seccompProfile
namespace/squidex labeled

My actual (potentially stupid) question

How do we go about changing the implementation of the MongoDB chart included in the solution?

I am finding it currently included in the solution as a .tgz file which I could unzip and make the necessary changes but I would not want to be making changes to something that is automatically included as part of a separate pipeline elsewhere and to have the changes overwritten at a later date.

If the case is to simply unzip the folder, make the required changes and zip the files back up again I am more than happy to do so but I just wanted to check what the best practice would be for this particular part of the system.

Another question

Also, as part of my testing of the increased security, I will need to make changes to the Dockerfile which may affect other deployment methods which will need to be tested and accounted for (like running as a non-root user account and exposing only non-privileged ports (1-1024).

Would you be available to help test these deployment methods once the pull request is ready and up on GitHub?

2

To be honest I have to read the details. So far I do not understand what all this is about.

3

No worries, is there any way I could help, possibly get on a call at some point to expand further and show my work in progress?

4

Sure, we can do that. But please create a PR first or something to work with.

5

Alright, I will clean up the code and get something submitted tomorrow.

6

Sounds great :slight_smile: … if you have any good resources for me to read or watch about this topic, let me know.

7

Hi Sebastian,

I have created a pull request: https://github.com/Squidex/squidex/pull/941

Would be more than happy to arrange a conversation to discuss the changes :slight_smile:

In principle, I am trying to reduce the attack scope of a compromised container/pod in a Kubernetes cluster.

Without making changes to your Dockerfile, the Linux container will likely be running as root. When a would-be attacker gains a foothold into one of these containers they would have the ability to install software and potentially expand their attack into other pods on the cluster or worse compromise the entire cluster’s hosts.

The pull request I have raised offers a method for users of the product to increase the security profile of the pods/containers running in a Kubernetes cluster to prevent things like installing applications, attempting to escalate privilege using sudo or su and to limit what damage to the filesystem of a pod they can do.

As for resources: