Collaboration yields results. At the ONUG Digital Live event this past spring, leading contributors to the ONUG Security Working Group presented a reference architecture for container security, which was then followed by an open reference solution that matched the outlined architecture. Years of work and input from industry leaders yielded a solution that ensures confidentiality, integrity and availability for container-based workloads. Forrest Bennett of FedEx served as moderator, while Bob Wysocki and Anmol Kulkarni of Microland, Michael Clark of Renaissance Technology, and Adam Hughes of Sylabs explained the architecture and the solution. Here’s a summary of their discussion. 

Context Around the Need for Reference Architecture

The ONUG Security Working Group focused their attention on developing reference architecture that addresses flaws in the confidentiality, integrity and availability of container-based workloads. While not going so far as to say the reference solution is ONUG-approved, it does provide a viable security approach in the complex multi-cloud, open source world. Michael Clark opened the discussion by providing an overview of the vulnerabilities of container security.

Clark emphasized that container security is only as secure as its underlying infrastructure. The container workload itself has not provided protection against runtime, side channel attacks that come from compromised hardware, microarchitecture defects, subverted microcode or implants. Additionally, there is no protection from runtime memory introspection attacks that come from host operating environments. Attackers pose a threat to both the confidentiality and integrity of a container workload. 

The Linux Kernel is not exempt from security flaws. Vulnerabilities exist related to namespace handling, which can cause privilege escalation and/or container breakout. Container runtime patching does not fix any of these issues. Adding to the complexity, is the fact that security models vary across cloud security providers (CSP). There is no comprehensive set of standards that all CSPs must adhere to. In a nutshell, private cloud security does not address container security adequately. 

The Goal of the Reference Architecture

The main goal of the reference architecture is to address the above challenges. Through use cases and collaboration the ONUG group’s aim was to “identify what could be done to secure the confidentiality and integrity of a given workload as it resides and transits through a container environment,” explained Clark. 

Productivity was given priority over security when container technology was being developed. To fill the security gap, we must identify the trustworthiness of the overall environment, monitoring for changes during runtime, both within the container and the larger environment. At the same time, Clark explained we must see the workload as a discrete unit that we are trying to protect. To boost the confidentiality and integrity of container workloads, the reference architecture has three requirements.

• It must bind an operational security policy to the workload.
• It must ensure the security policy cannot be interfered with or decoupled from the workload.
• The reference architecture must ensure the workload, its security policy, and policy enforcement mechanisms cannot be tampered with. 

In order for the reference work to function as intended, Clark identified these assumptions that must be made. 

• The hardware is trustworthy. 
• The container environment platform, as well as the environment used to construct the container image must be trustworthy.
• The certificate of authority and encryption algorithm implementations must be trustworthy.
• The trustworthiness of the environment must be determined at or before runtime.
• The container construction environment must be trusted and secure. 

From Reference Architecture to Solution

Next, the group discussed how Microland and Sylabs worked together to take the reference architecture and turn it into a solution for container security that was easily reproducible, and required little cost and minimal integration. Bob Wysocki and Adam Hughes explained these five pillars of the solution. 

1. Host: First, the solution needed a runtime environment. The team chose Virtual Box for the demo since it was freely available and had a low learning curve. 
2. Container Solution: After much thought, the team settled on Singularity for its container solution since it has many valuable native security features. Hughes explained that Singularity is a container runtime that was developed in high performance computing environments. It is often used for sensitive workloads for academic institutions, government departments, as well as in corporate environments. This solution leverages all available security components to maintain security and block privilege escalation. Users expect to run workloads as unprivileged users, a best practice that Sylabs promotes. 
3. Singularity Image Format (SIF) is used to handle single files. Within each file, users can encapsulate many items, including software and all the dependencies that make up the workload. Full image encryption is the most valuable feature of SIF. Unlike other container runtimes, encrypted file systems pass directly into the Kernel along with associated key material. The decrypted contents of the image only exist in memory, not in storage. This makes SIF images ideal for companies that have policy or regulatory requirements, or the need to protect sensitive data and algorithms. SIFs can also hold arbitrary data, including the policy document. When combined with a digital signature, the policy is cryptographically bound to the workload. 
4. Supervisor Binary (AB): This is the open source wrapping around Singularity that validates the environment, testing its trustworthiness, and ensuring that Singularity only runs if the right policy is met. 
5. Open Policy Agent (OPA): The Open Policy Agent talks to the Supervisor Binary to receive the latest state of the environment from the runtime, comparing the state to the allowed workload policy. It only returns the decrypted key when there is a match, allowing Singularity to run. 

Wysocki explained that Singularity and its SIF ensure the confidentiality and integrity of the workload, while the OPA and SB ensure the confidentiality and integrity of the runtime environment. 

Putting Into Practice

Bob Wysocki next explained how to put the solution into practice using three operating modes. 

• In Setup Mode: The SB establishes a “fingerprint” of the environment, including the allowed policy. Next, the SB stores the policy and the decryption key in the OPA. 
• Upon Startup: Now you want to launch your runtime. The SB fingerprints the environment. The data is forwarded to the OPA. The OPA compares the current state to the allowed state, and only returns the encryption key if it matches. When a match is made, Singularity will start, decrypt the container and launch runtime. 
• During Runtime: At this point, SB switches to a watch mode. If the running policy continues to match the intended policy, everything continues to run fine. If there is a mismatch, the SB will take action to stop the runtime. 

Learn More

View the full discussion that includes a live demo of the container security solution here. Leverage the opportunity to give your input to one of ONUG’s working groups by registering here. 

Don’t forget to mark your calendar for the Fall ONUG conference on October 14th and 15th. 

Register here