ONUG Cloud Native Security Working Group Blog Series #2

“For those of you with an interest in data-driven DevSecOps, the ONUG Cloud Native Security Working Group is kicking off a deeper-dive article series on the role of data in modern applications, discussing some best practices for translating an application’s operational data “exhaust” into deep insights and actionable feedback.  The first piece in this series introduces the three pillars of the data-driven approach: collecting data across the breadth of the distributed infrastructure, ensuring data inputs also span the full stack, and then adopting an AI-friendly mindset regarding the collected data. Subsequent articles will describe each pillar in more detail.” – Ken Arora, Distinguished Engineer with F5’s Office of the CTO

The ONUG Cloud Native Security Working Group recently published a column that provided an examination of the architecture patterns of modern applications, set within the context of the journey that has brought us to where we are today.  In that overview, we also announced the group would present the community with deeper discussions around the key themes and challenges that have emerged from this journey.  This article kicks off one of those focus areas – namely, the DevSecOps practice and its role with respect to securing modern applications.

One key understanding that those of us in the position of having to keep modern applications simultaneously available, scalable, and secure – often referred to as DevSecOps practitioners – quickly develop is that leveraging operational data is foundational to our success.  In fact, this principle is increasingly essential in light of emerging trends around modern applications.  Therefore, we begin this article series focused on the DevSecOps practice by highlighting some of the most noteworthy changes in the area of modern application development, illustrating why data telemetry is crucial.  Later articles in this series will follow up with recommendations for best practices on the collection, curation, and analysis of the data.

A good starting point is to consider how applications are constructed and deployed, and how that has evolved over the last decade.  Application architectures were for over two decades expressed in monolithic or 3-tier stacks; however, application architecture is now a composition of multiple building block containers and services. Meanwhile, application deployments are increasingly manifested in a diverse set of environments: privately managed data centers, dispersed over public cloud, edge, CDN, and even client footprints. Application infrastructures have transitioned from physical network appliances to abstracted services like databases and identity management, to application development platforms running on compute servers in public or private clouds; indeed, infrastructure today abstracts all layers below the core application business logic, including transport, database, and identity. Application development was once performed by a centralized, in-house team of specialists; application development today highly leverages open source code and outsourced functionality-as-a-service, and even in-house development is often distributed across the enterprise.

So, what does this all mean to a DevSecOps practitioner?  The two most important implications are that these trends have simultaneously both exposed new threat surfaces, while – as an unfortunate implication of abstracting the services – reducing visibility into the operations of the application’s infrastructure components.  Exacerbating the consequences of these two factors is the ever-increasing scale of modern applications as they endeavor to address business-to-consumer and even IoT scale solutions.

The key pillars of a DevSecOps strategy are therefore:

• Building systems that increase visibility into their internal operations to counter the effect of decreased visibility into application infrastructure and construction.
• Collection of operational data across the multiple layers of the holistic application stack. Note that because so much of the infrastructure supporting the application is from third-party service providers (e.g. cloud logs, identity events, API gateway processing), the consequence is that the collected telemetry must not be merely ingested, but also curated and normalized before being warehoused and analyzed.
• The use of AI-assistance in sorting through the firehose of data that is collected, looking for both anomalies and correlating patterns across multiple data sources.

Future articles will go into each of these aspects of the solution strategy in more detail, but today we can give a taste of each of these 3 pillars of the approach.

First, we start with the axiom “you can’t fix what you can’t see.” Even if the portion of the application developed in-house within your enterprise – typically, that is the core business logic that is your company’s value-add – is wonderfully designed, undergoes a threat model review to consider and address the myriad threats against that code, and is instrumented to provide operational telemetry, what happens when an open-source package that your application leverages is found to have a vulnerability?  Or a debug tool in the cloud deployment environment is improperly secured, and allows reverse shell access?  Clearly, because the application is a composed of a medley of multiple components, many of which are third party, you need visibility into the operation of all of these components.

A corollary to this is the second pillar: threat surfaces exist at multiple levels – network, application support services, protocol, API, and business logic.  Increasingly, many of those levels are abstracted away, but the threats still remain.  For example, your cloud provider may supply a native API gateway, but what happens if an attacker attempts to abuse your application’s public-facing APIs – specifically, what does the application infrastructure do if the inputs to the API are malformed, or excessively large?  Analogous concerns exist for all other outsourced aspects of infrastructure, such as databases or identity services.

The third pillar arises as a repercussion of doing a good job on the first two items – namely, you may end up drowning in a sea of data that no reasonable group of humans can consume, let alone analyze, in any timely manner.  Consider modern Advanced Persistent Threats (APT’s): they are quick hitters, patiently lying dormant and waiting for long periods of time, followed by a short burst of activity to exfiltrate data.  Finding this short burst is like looking for the proverbial needle in the haystack, but with the added pressure of needing to find the needle within seconds.  To meet this challenge, the solution must leverage AI to find the needle and, in some cases, perform automated remediation to stop the attacker in a timely manner.  An interesting wrinkle is that the data collected – the realm of the first two pillars – must be curated in an AI-friendly manner to make the automated response feasible.

In conclusion, the goal of this article is to set the stage for a set of deeper dives and more detailed recommendations in articles that follow.  At the overview level, the key concepts to take away as a DevSecOps practitioner are first, the evolving nature of modern applications – their compositional architecture strategy, with much of the underlying infrastructure abstracted away from the application developer, deployed in a distributed manner across cloud, edge, and browser – and, second, the implications around new threat surfaces coupled to the lack of development visibility into many of the building blocks and foundational elements of the infrastructure.  From there, the need for greater operational visibility – in the form of real-time data telemetry across the entire application stack, coupled to AI-assisted data analysis – becomes clear as a key component of DevSecOps of the future.  Future articles will dive into how best to collect the data telemetry exhaust, collate and transform it into more machine-consumable formats that are leveraged by automated and AI-assisted workflows, all in an integrated and agile manner.