Comparing The Cloud Foundry and Kubernetes Vision of DevOps

Published on July 19, 2018

---

It’s interesting to me to see how aligned people are around DevOps as a movement, and yet how varied the actual implementations end up being.

The best tools often encourage a highly opinionated philosophy of not only how to use them, but by extension, how a team should operate. This is well illustrated by comparing Kubernetes and Cloud Foundry, how they encourage DevOps inside an organization, and what form of it they enable.

Cloud Foundry

Cloud Foundry is a 12-Factor platform that takes most of it’s inspiration from Heroku.

There are two distinct personas who interact with the Cloud Foundry platform: The application developer and the platform operator. Each has unique responsibilities and the platform presents them with completely different tools and interfaces.

The Application Developer

The developer is the software engineer who’s responsible for writing the application and managing it in production. They interact with Cloud Foundry primarily through the cf command line tool. This tool gives them easy access to higher level commands focused on manipulating deployed applications.

Their view is entirely app-centric. They have no visibility into the underlying virtual machines or servers that the platform is running on. They also need no more than a basic familiarity with the contents of the container image that houses their application, or indeed of the fact that there are containers at all.

It’s entirely possible to run a massive application in production without diving into the details of the containers. The Cloud Foundry platform does an admirable job of hiding those details from the application developer through buildpacks. It hides other components as well – components it believes are the responsibility of the platform, and not something application developers should manage directly. These include how the ingress routing system works, how to mount persistent volumes, how to implement container-to-container networking, how to gather metrics and logs, how to manage distributed databases and so on.

Instead of these low-level concepts, the cf command provides higher level workflows. It provides simple commands for deploying your application, listing running applications, scaling them, tailing logs, creating databases, adding ingress routes and domains, and so on.

The application developer isn’t responsible for configuring servers and running the underlying platform, but they are responsible for the security and performance, and availability of their application. The platform doesn’t carry the pager for them.

The Cloud Foundry philosophy is that by providing a more tailored experience, and by automating as much of the stack away as possible, it makes it easier for application developers to practice DevOps without expecting them to become experts in every component from hypervisor to browser.

The Platform Operator

Of course, someone needs to worry about how to provision VMs, how many IOPS are left on the SAN and how to manage and upgrade the platform. This is the role of the platform operator, who interacts with the Cloud Foundry platform through a tool called BOSH.

BOSH is often referred to as “the platform under the platform.” It provides operators with the workflow and tooling necessary to provision, configure and manage large-scale clusters of virtual machines. It continually monitors the servers and the software deployed on them, keeping everything running and healthy.

By the way, BOSH is a general deployment tool. It’s not at all specific to Cloud Foundry, and it’s gaining some popularity in the overall operations world.

Kubernetes

Kubernetes is a container orchestration platform, where the primary unit of deployment is a set of containers, rather than an application codebase. The distinction is subtle, but has far reaching implications throughout the system.

In Kubernetes, there is one persona and one interface into the system. Instead of providing higher level workflows, Kubernetes focuses on exposing the entire system as building blocks that a team uses to construct their ideal architecture.

This flexibility and configurability comes at a price. Users must understand the intricacies of dozens of different domain concepts, how to configure, secure and maintain them. Deploying a single application consists of configuring a Deployment, it’s Pods and Containers, a ConfigMap, Secrets, a Loadbalancer or Ingress, and NetworkPolicies. The user must also configure and manage the database themselves, which is a complicated affair consisting of StatefulSets, PersistentVolumes, StorageClasses and PersistentVolumeClaims.

The fact that Kubernetes exposes the underlying domain directly has a democratizing effect. It’s possible to draw lines of responsibility in a Kubernetes system. For example, between interacting with the platform via kubectl and administering the underlying servers and Kubernetes processes, or by making use of RBAC and namespaces to restrict what different users can do inside the platform. But the platform doesn’t do this by default, instead leaving it as an exercise to the end user.

By omitting a default level of isolation of concerns, Kubernetes encourages a more platonic version of DevOps. In the Kubernetes world, everyone is part of one team and jointly responsible for all aspects of the platform.

The Philosophies

The Cloud Foundry philosophy on DevOps is that the modern stack is getting larger and larger, and that it isn’t realistic to expect any one person to be responsible for the entire spectrum. It believes that by targeting specific personas, the interface into this stack can be simplified at the expense of flexibility.

Cloud Foundry is in effect a wall between developers and operations. It allows operations teams to provide more self-management and control to application developers, while guiding them toward best practices.

The Kubernetes philosophy, on the other hand, believes in the DevOps commons. It breaks down the developer/operator wall entirely, and provides the user with a collection of Lego bricks to construct an architecture that suits their needs. Kubernetes works best when everyone intermingles, writing and deploying code as a single team.

Each Path in Context

The Cloud Foundry model is best suited on the two ends of the organizational size spectrum: On the large end are enterprises with many teams wanting to consolidate their platform to control security, decrease complexity and make better use of server resources. They already have a large operations team with complex infrastructure and virtualization needs.

On the smallest end of the spectrum are the pre-investment startups that are able to use a hosted offering such as Heroku, Bluemix or Pivotal Web Services, a creation of Pivotal Software, Inc. and now part of VMware. They have simple technology needs and value a quick path to market above all else.

Kubernetes, on the other hand, finds the sweet spot right in-between the two – The large swath of funded startups with complex microservices needs, for example, benefit from the high configurability, as do smaller departments inside enterprise organizations with similar needs.

Some interesting projects are attempting to bridge the gap between Cloud Foundry and Kubernetes.

From the Cloud Foundry Foundation, we now have the CF Container Runtime, a BOSH-powered Kubernetes as a service. This deploys Kubernetes clusters on demand using the Cloud Foundry Services interface.

Taking a different approach is Eirini (a CF incubated project with support from SUSE, IBM and SAP), which modifies Cloud Foundry to deploy applications into an existing Kubernetes installation. SUSE has also produced SCF, which deploys Cloud Foundry itself onto Kubernetes. Combine both if you’re daring.

However, these attempt to sidestep the important question of what type of DevOps organization is right for you. The answer will change as the size and needs of your company changes, but it’s important to be mindful of which directions the tools you choose will send you. Attempting to graft an opposing DevOps philosophy onto the wrong tool will result in wasted effort and a workflow full of friction.

Reach out to learn how we can help your team through engineering or tailored, hands-on workshops on Kubernetes and Cloud Foundry.