CERN Computer Security Information

<!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0 Strict//EN" "http://www.w3.org/TR/xhtml1/DTD/xhtml1-strict.dtd"> <html xmlns="http://www.w3.org/1999/xhtml" xml:lang="en" lang="en"> <head> <meta http-equiv="Content-Type" content="text/html; charset=UTF-8" /> <link rel="stylesheet" href="/style.css" type="text/css" /> <script type="text/javascript" src="/jquery.min.js"></script> <title>CERN Computer Security Information</title> <script type="text/javascript"> $(document).ready(function(){ // Menu highlight var path = location.pathname.split("/"); if ( path ) { $('#main_menu a[href*="' + path[1] + '"][class!="noselect"]').addClass('selected'); // path[3] = /security/<xxxxx>/ $('#sidebar ul.sidemenu li[class!="noselect"]:has(a[href$="' + path.reverse()[0] + '"])').addClass('selected'); } // Add icon to external links $('a[id!=logo-img]').filter(function() { return this.hostname && this.hostname !== location.hostname; }).after(' <img src="/images/external_link.png" alt="external link" title="external link"/>'); }); </script> </head> <body> <div id="wrap"> <div id="top-bg"></div>  <div id="header"> <div id="logo-text"> <a id="logo-img" href="https://home.cern/"><img src="/images/CERNLogo2.png" width="59" height="59" style="margin: 10px" alt="CERN Logo"/></a><div id="logo-text-big"><a href="/home/en/index.shtml" title="">CERN Computer Security</a></div> </div> <div id="header-logo"><a href="/services/en/emergency.shtml"><img width=335 src="/images/emergency.png" alt="Computer Emergencies"/></a></div> </div>  <div id="header-photo"></div>  <div id="nav"> <ul id="main_menu"> <li><a class="noselect" href="/home/fr/index.shtml"><img src="/images/fr.png" alt="FR"/></a></li> <li><a href="/home/en/index.shtml">Home</a></li> <li><a href="/rules/en/index.shtml">Computing Rules</a></li> <li><a href="/recommendations/en/index.shtml">Recommendations</a></li> <li><a href="/training/en/index.shtml">Training</a></li> <li><a href="/services/en/index.shtml">Services</a></li> <li><a class="secured" href="/reports/en/index.shtml">Reports & Presentations</a></li> </ul> </div>   <div id="content-wrap"> <div id="main"> <h2>Securing Containers & Pods</h2> <p> Often people who are new to container and orchestration technologies are under the impression that these products are inherently secure and suitable for production use. While that might not be true, these solutions offer a lot of configuration options for strengthening the security of the containerized environment in order to get protections similar to that of a virtual machine. This way you can ensure that, even if an attacker manages to break into an application, they will have a hard time pivoting to other machines on the network or the host system. </p><p> In the below, you find some strategies to improve the security of your <a href="#l1">Kubernets</a> and <a href="#l2">Docker</a> environments. </p> <h4><a name="l1"></a><a href="http://clouddocs.web.cern.ch/clouddocs/containers/">Kubernetes Environments</a></h4> <p> </p> <table class="align-left"> <tbody> <tr><th class="align-center">Requirement</th><th class="align-center">Description</th><th class="align-center">Add'l info</th></tr> <tr> <td>Use central services instead of spinning up your own clusters or containers.</td> <td>Using central services allows you to delegate the responsibility for maintaining a secure configuration.</td> <td> <a href="https://openshift.cern.ch/console/catalog" target="_blank">CERN Openshift</a><br><a href="https://openstack.cern.ch/project/clusters/" target="_blank">CERN OpenStack</a> </td> </tr> <tr> <td>Enable CERN SSO for any websites that will be exposed to the Internet.</td> <td>CERN SSO allows users to authenticate to your application using their CERN credentials.</td> <td> <a href="https://sso-management.web.cern.ch" target="_blank">Manage your SSO applications</a><br><a href="https://espace.cern.ch/authentication/CERN%20Authentication/Home.aspx" target="_blank">Authentication documentation</a> </td> </tr> <tr> <td>Enable HTTPS on your web applications.</td> <td>Encryption makes sure somebody who is listening to your communications can't see sensitive data that is transmitted.</td> <td>In Openshift you can set the route that exposes your application to have <a href="https://docs.openshift.com/enterprise/3.0/architecture/core_concepts/routes.html#edge-termination" target="_blank">Edge termination</a><br>In OpenStack you can <a href="https://clouddocs.web.cern.ch/clouddocs/containers/tutorials/lb.html" target="_blank">define an ingress controller with SSL termination</a> </td> </tr> <tr> <td>Run trusted, official images.</td> <td>Official images are maintained and are usually better secured than alternatives, even if you might find an unofficial image that fits your purposes better you should make the effort to use an official one. CERN runs its own container registry where you can find trusted images.<br>Also, we suggest that you base your images off of CERN's cc7-base image.</td> <td> <a href="https://gitlab.cern.ch">CERN Gitlab</a><br><a href="https://gitlab.cern.ch/linuxsupport/cc7-base/container_registry">cc7-base image</a> </td> </tr> <tr> <td>Run <tt>kube-bench</tt> for checking your Kubernetes deployment.</td> <td> <tt>kube-bench</tt> checks your setup against the <a href="https://www.cisecurity.org/benchmark/kubernetes/" target="_blank">CIS Kubernetes Benchmark</a>. This tool is already ran by the OpenStack team, so there is no need to run it if your cluster resides on OpenStack.</td> <td><a href="https://github.com/aquasecurity/kube-bench" target="_blank">kube-bench</a></td> </tr> <tr> <td>Set up Pod Security Policies.</td> <td>Pod Security Policies allows you to specify rules about sensitive aspects of your pods. Any pods that do not follow those rules are not allowed into the system.</td> <td><a href="https://kubernetes.io/docs/concepts/policy/pod-security-policy/" target="_blank"><tt>PodSecurityPolicy</tt> specification</a></td> </tr> <tr> <td>Do not expose your pods to the network if not necessary.</td> <td>Running a container in the <tt>host</tt> network or exposing ports without the need allows anyone in the network to connect to them.</td> <td> <a href="https://kubernetes.io/docs/concepts/policy/pod-security-policy/#host-namespaces" target="_blank"><tt>PodSecurityPolicy</tt> - Host namespaces options</a> can help ensure that you don't accidentally expose something that you don't want to.</td> </tr> <tr> <td>Do not run pods as the root user.</td> <td>An attacker who is running as root inside a pod is in a much better position to attach the network.</td> <td> <a href="https://kubernetes.io/docs/concepts/policy/pod-security-policy/#users-and-groups" target="_blank"><tt>PodSecurityPolicy</tt> - Users and groups options</a> can help ensure that all your pods run as allowed users.</td> </tr> <tr> <td>Scan images before deploying them.</td> <td></td> <td>We can provide a <a href="https://github.com/coreos/clair" target="_blank">Clair</a> instance. <strong>TODO</strong> </td> </tr> <tr> <td>Do not hard-code secrets (e.g. login details) in your images.</td> <td>Sensitive data such as username-password combos should not be stored in an image, as then anybody with access to the image will be able to read it.</td> <td> <a href="https://kubernetes.io/docs/concepts/configuration/secret/" target="_blank">Kubernetes secrets</a>. If running on OpenStack Magnum, you can use <a href="http://clouddocs.web.cern.ch/clouddocs/containers/tutorials/helm.html#managing-secrets" target="_blank">Helm managed Secrets with Barbican</a> </td> </tr> <tr> <td>Delete secrets after reading them.</td> <td>You can delete the files containing your secrets to prevent a possible attacker from reading them and using them to access other CERN services that your application uses, if they manage to break into your application. However your secrets will still possibly be found in memory so a skilled attacker can circumvent this mitigation.</td> <td>In Kubernetes, you can have an <a href="https://kubernetes.io/docs/concepts/workloads/pods/init-containers/" target="_blank">Init Container</a> which runs before your main container runs. You can also create an <a href="https://kubernetes.io/docs/concepts/storage/volumes/#emptydir" target="_blank"><tt>emptyDir</tt> volume</a> which is initially empty when created a pod is created. You can combine the two by reading your secret files in the <tt>initContainer</tt> and copying them over to your volume. Then, in your main container, you can mount the same volume, read your secrets in your application and delete the files from the volume.</td> </tr> <tr> <td>Restrict outgoing connections to only the necessary hosts.</td> <td>This way, if your application is broken into, an attacker cannot use it to pivot to the rest of the CERN network.</td> <td>Create an egress <a href="https://kubernetes.io/docs/concepts/services-networking/network-policies/#the-networkpolicy-resource" target="_blank"><tt>NetworkPolicy</tt></a> to only allow connections from your pods to the services that they need. You can specify either pods or CIDRs to allow for egress traffic, as well as ports.<br>For this to work in OpenStack pods, the cluster has to be using a network driver that allows this, such as <tt>calico</tt>. To use that, you first have to create a cluster template (e.g. with <tt>coe cluster template create --floating-ip-disabled --master-flavor m2.medium --server-type vm --external-network CERN_NETWORK --image 2886e9bf-7702-4c06-99c1-39a9e7bd5951 --docker-storage-driver overlay2 --network-driver calico --fixed-network CERN_NETWORK --coe kubernetes --flavor m2.medium --dns-nameserver 137.138.17.5 --labels="calico_backend=vxlan" calico-kubernetes</tt>) and then create your cluster using that template. When doing this procedure please use the parameters from an existing template to ensure there are no issues, changing only the <tt>labels</tt> and <tt>network-driver</tt> parameters.<br>It should also work in Openshift without any changes when Openshift 4 is deployed (est. end of 2019).</td> </tr> <tr> <td>Use role-based access control (RBAC).</td> <td>RBAC allows you to manage which users or applications can access which resources in what ways. By creating roles and assigning them to users or applications, you can control which namespaces they can access. You can also specify the types of access in the form of verbs (e.g. <tt>list</tt>, <tt>create</tt>, <tt>update</tt>).</td> <td><a href="https://kubernetes.io/docs/reference/access-authn-authz/rbac/" target="_blank">Roles and Role Bindings reference</a></td> </tr> </tbody> </table> <h4><a name="l2"></a><a href="https://information-technology.web.cern.ch/services/PaaS-Web-App">Docker Environments</a></h4> <p> </p> <table class="align-left"> <tbody> <tr><th class="align-center">Requirement</th><th class="align-center">Description</th><th class="align-center">Add'l info</th></tr> <tr> <td>Do not expose your containers (forward ports) to the network if not necessary.</td> <td>Running a container in the <tt>host</tt> network or exposing ports without the need allows anyone in the network to connect to them.</td> <td></td> </tr> <tr> <td>Do not run containers as the root user.</td> <td>An attacker who is running as root inside the container is in a much better position to escape the container and attack the host.</td> <td><a href="https://docs.docker.com/engine/reference/builder/#user" target="_blank">Docker USER directive</a></td> </tr> <tr> <td>Run trusted, official images.</td> <td>Official images are maintained and are usually better secured than alternatives, even if you might find an unofficial image that fits your purposes better you should make the effort to use an official one. CERN runs its own container registry where you can find trusted images.<br>Also, we suggest that you base your images off of CERN's cc7-base image.</td> <td> <a href="https://gitlab.cern.ch">CERN Gitlab</a><br><a href="https://gitlab.cern.ch/linuxsupport/cc7-base/container_registry">cc7-base image</a> </td> </tr> <tr> <td>Run Docker Bench for Security for ensuring your Docker service runs with a good security setup.</td> <td>Docker Bench for Security checks your Docker daemon configuration for common best practices for deploying containers.</td> <td> <a href="https://github.com/docker/docker-bench-security" target="_blank">Docker Bench for Security</a><br>See <a href="https://www.digitalocean.com/community/tutorials/how-to-audit-docker-host-security-with-docker-bench-for-security-on-ubuntu-16-04" target="_blank">here</a> for solutions to possible warnings.</td> </tr> <tr> <td>Isolate containers with User namespaces.</td> <td>User namespaces limit privilege-escalation attacks to the host from within the container, if an attacker manages to get root access inside the container.</td> <td><a href="https://docs.docker.com/v17.09/engine/security/userns-remap/" target="_blank">User namespaces</a></td> </tr> <tr> <td>Use the host firewall to restrict outgoing connections.</td> <td>You can use <tt>iptables</tt> to restrict the hosts a container can connect to by creating a network interface for it and applying rules to it. This way, an attacker cannot easily use a vulnerability to pivot to the rest of the CERN network.</td> <td>For example:<br><tt>dk network create -o com.docker.network.bridge.name=testbridge testbridge</tt><br><tt>sudo iptables -N BEFORE_DOCKER</tt><br><tt>sudo iptables -I BEFORE_DOCKER -d 172.217.168.4 -j RETURN</tt><br><tt>sudo iptables -I BEFORE_DOCKER -d 104.27.202.92 -j RETURN</tt><br><tt>sudo iptables -I FORWARD -i testbridge -m state --state NEW -j BEFORE_DOCKER</tt><br><tt>dk run --rm -it --net testbridge centos:7</tt> </td> </tr> <tr> <td>Create a seccomp profile for your image.</td> <td>A seccomp profile can restrict the system calls a container has access to to only the necessary, reducing the attack surface. This is not trivial to enable, as it sometimes leads to crashes.</td> <td><a href="https://github.com/antitree/syscall2seccomp" target="_blank">syscall2seccomp</a></td> </tr> <tr> <td>Do not expose your Docker daemon socket to the network.</td> <td>The Docker daemon can be configured to listen to the network for commands. By default, if enabled, Docker listens to port 2375 and allows users to run containers without any authentication. This is very dangerous and might be abused to gain root privileged on the host.</td> <td></td> </tr> </tbody> </table> </div>    <div id="sidebar"> <h3>For All Users<br/> (Experts or Not)</h3> <ul class="sidemenu"> <li><a href="/recommendations/en/good_practises.shtml">Seven easy good practises</a></li> <li><a href="/recommendations/en/how_to_secure_your_pc.shtml">How to secure your PC or Mac</a></li> <li><a href="/recommendations/en/passwords.shtml">Passwords & toothbrushes</a></li> <li><a href="/recommendations/en/2FA.shtml">Starting with multi-factor authentication</a></li> <li><a href="/recommendations/en/bad_mails.shtml">Bad mails for you:<br/>"Phishing", "SPAM" & fraud</a></li> <li><a href="/recommendations/en/malicious_email.shtml">How to identify malicious e-mails and attachments</a></li> <li><a href="/recommendations/en/how_to_remove_malicious_browser_notifications.shtml">How to remove malicious browser notifications</a></li> <li><a href="/recommendations/en/working_remotely.shtml">Working remotely</a></li> <li><a href="/recommendations/en/connecting_to_cern.shtml">Connecting to CERN</a></li> <li><a href="/recommendations/en/ssh.shtml">Connecting using SSH</a></li> </ul> <h3>For Software Developers</h3> <ul class="sidemenu"> <li>Good programming in <a href="/recommendations/en/program_c.shtml">C/C++</a>, <a href="/recommendations/en/program_java.shtml">Java</a>, <a href="/recommendations/en/program_perl.shtml">Perl</a>, <a href="/recommendations/en/program_php.shtml">PHP</a>, and <a href="/recommendations/en/program_python.shtml">Python</a></li> <li><a href="/recommendations/en/password_alternatives.shtml">How to keep secrets secret<br/> (alternatives to passwords)</a></li> <li><a href="/recommendations/en/checklist_for_coders.shtml">Security checklist</a></li> <li><a href="https://gitlab.docs.cern.ch/docs/Secure%20your%20application/">GitLab CI Security Tools</a></li> <li><a href="/recommendations/en/web_applications.shtml">Securing Web applications</a></li> <li><a href="/recommendations/en/code_tools.shtml">Static code analysis tools</a></li> <li><a href="/recommendations/en/more_on_software.shtml">Further reading</a></li> </ul> <h3>For System Owners</h3> <ul class="sidemenu"> <li><a href="/recommendations/en/rootkits.shtml">Checking for rootkits</a></li> <li><a href="https://twiki.cern.ch/twiki/bin/viewauth/CNIC/WebHome">Securing Control Systems (CNIC)</a></li> <li><a href="/recommendations/en/containers.shtml">Securing Containers & Pods</a></li> <li><a href="/rules/en/baselines.shtml">Security baselines</a></li> <li><a href="http://linux.web.cern.ch/linux/docs/linux_exploit_faq.shtml"> The CERN Linux vulnerability FAQ</a></li> </ul> </div>   </div>  <div id="footer-wrap"> <div id="footer-bottom"> © Copyright 2024<strong> <a href="https://cern.ch/security">CERN Computer Security Office</a></strong> <table> <tr> <td id="footer-info-left"> e-mail: <a href="mailto:Computer.Security@cern.ch">Computer.Security@cern.ch</a><br/> Please use the following PGP key to encrypt your messages:<br/> ID: 0x954CE234B4C6ED84<br/> <a href="https://keys.openpgp.org/vks/v1/by-fingerprint/429D60460EBE8006B04CDF02954CE234B4C6ED84">429D 6046 0EBE 8006 B04C DF02 954C E234 B4C6 ED84</a> </td> <td id="footer-info-right"> Phone: +41 22 767 0500<br/> Please listen to the recorded instructions. </td> </tr> </table> </div> </div>  </div>   </body> </html>

CINXE.COM

CERN Computer Security Information