Gemini’s DevOps Transformation: Building a Culture of Full Automation and Self Service

DevOps — a portmanteau of “development” and “operations” — is a software development philosophy that advocates for certain cultural norms, technical practices, and architecture choices to drive the performance of software development. Highly evolved DevOps cultures are able to ship software faster and more reliably. While the DevOps methodology has been understood for over a decade, applying it to a highly regulated space in a compliant way is a novel challenge.

In order to achieve the unlock of DevOps, engineers must be able to build self-sufficiently and independently. This requires providing them with end-to-end ownership of service creation, development, deployment, and operations. The journey to DevOps maturity, however, is loaded with obstacles, both culturally and technically.

In this post, we explore the evolution of our DevOps transformation and outline the decisive impact it has had thus far on our engineering teams and ultimately our customers.

Our Transformation

A year ago, Gemini, like many startups, had a rather monolithic system architecture with a traditional operating model. This included typical roles and teams covering Engineering, Q&A, Infrastructure, and Operations, leading to many interdepartmental handoffs, long release cycles, and slow change and innovation velocity.

Since then, we’ve taken various steps as an engineering organization to streamline our processes across departments and implement a DevOps philosophy, ultimately increasing our deployments by more than 8x without losing our focus on security and compliance.

Here’s how it's going.

Changing Mindset

Early on, we realized that the key to providing a truly enjoyable customer experience was rapid iteration and experimentation in our products and our underlying platform to enable a higher frequency of innovation.

In August 2021, we rechartered our Infrastructure team as our Platform team and seeded it with a blend of engineers from various industry backgrounds who had expertise in fostering a DevOps culture and building cloud-native infrastructure.

Our Platform team oversees a host of products that include tooling, services, and abstractions, which allow our engineering teams to be self-sufficient, while operating in a compliant, secure, and safe environment.

We took a 4-pronged approach to implement this new way of thinking:

1. Decouple monolithic deployments into sub-component deployments
2. Automate deployments and enable a self-service “gitops”-based release process (gDeploy)
3. Build out a next-generation Kubernetes-based continuous integration/continuous delivery and deployment (CI/CD) platform
4. Build out a engineering-focused observability/telemetry system

Our Platform team is implementing this new approach within our engineering teams and also providing the guidance and thought leadership to drive the cultural aspect of DevOps across Gemini as a whole.

Our Transformation is Starting to Pay Off

Full automation and self-service are important principles of DevOps. Since implementing them, we have seen extraordinary results:

• We’ve automated manual deployment tasks allowing different sub-components to be deployed independently and safely
• We’ve increased our release velocity by over 8x compared to our pre-DevOps days

We made these improvements to our software development process by launching gDeploy, which is a Golang command line interface (CLI) that integrates GitHub, Concourse, and Ansible using artifacts built by Jenkins and distributing them across our virtual machine-based fleet.

Aerospike*, our next generation development platform, will bring our DevOps transformation to the next level by truly delivering on our goal of instilling a “you build it, you run it” culture. Aerospike is now reaching maturity, and while still in beta, we have already seen a significant increase in release and experimentation velocity for the services that currently leverage it.

Aerospike employs a modern infrastructure approach by unifying three main services:

1. Broad CI/CD capabilities
2. A modern Elastic Kubernetes Service (EKS) based compute infrastructure
3. A well-integrated observability/telemetry solution

We are one year into our DevOps journey. The first six months were about planning and laying the foundation. During the last six months, we have started reaping the benefits, achieving close to a 10x increase in our deployment velocity.

With this continuous integration, delivery, and validation approach, we have proven that we can build an operation model founded in DevOps principles that is secure and compliant. Next stop? ... 1,000 deploys per month and beyond!

Onward and Upward!

Nils Pommerien
VP of Platform

*On Aerospike Engines

The aerospike engine is a type of rocket engine that maintains its aerodynamic efficiency across a wide range of altitudes. Aerospikes are the baseline engines for many single-stage-to-orbit (SSTO) designs and were also a strong contender for the Space Shuttle main engine.

The aerospike engine aligns with the goals we are trying to pursue. Reaching escape velocity with a single engine without complicated stage separations and various propulsion systems — the equivalent of expensive handoffs and gates in our previous software development life cycle (SDLC) — is exactly what we are striving for with our DevOps transformation.