Demystifying the control plane: the easy upgrade path from GitOps with BigConfig
For many engineering teams, GitOps has been a game-changer, providing a declarative way to manage infrastructure and applications. But as complexity grows, you may find your processes hitting a ceiling. The natural next step? Upgrading to a control plane.
Often, teams hesitate, believing this shift requires a costly, painful rewrite of their entire configuration management solution when it is written in Terraform or Ansible. They fear increased cognitive load and massive sunk costs.
This is where BigConfig changes the narrative.
With BigConfig, you can upgrade your existing GitOps solution to a full-fledged control plane without rewriting everything from scratch or overwhelming your team.
Control plane components
Section titled “Control plane components”A control plane is essentially a brain that manages your entire system. It operates on three fundamental components:
- The API: The interface where users (or other systems) declare what they want.
- The desired state: The single source of truth—the definitive record of how the system should look.
- The control loop: The tireless worker that continuously observes the actual state and makes changes to match the desired state.
Desired state persistence
Section titled “Desired state persistence”The desired state is the heart of the control plane, and it needs to be persisted in a reliable, flexible way.
When dealing with complex, nested configuration data—like a complete picture of your developers, teams, environments, policies, and services—a traditional SQL data model can quickly become cumbersome. Instead, storing the state as a flexible, nested map is far more efficient and manageable.
Audit log
Section titled “Audit log”A key requirement for a robust control plane is a detailed audit log. We don’t just want the final state; we want to know how it got there like we do in GitOps.
BigConfig uses a pattern often referred to as Event Sourcing (though we’ll keep it simple here). Instead of storing only the current state, we store all the events that generated that state. This gives you an immutable, perfect audit trail and allows you to rebuild the state at any point in time.
BigConfig Store
Section titled “BigConfig Store”BigConfig tackles this persistence challenge by leveraging Redis Sorted Sets to store both the events and periodic snapshots of the state. This provides high performance and structure for our time-series data.
Let’s look at a conceptual example in code (using Clojure) that demonstrates how BigConfig manages concurrent updates without inconsistency:
(deftest inc-test (testing "inc from 2 instances works" (let [port (-> (system-state) :redis/server :port) default-opts {:store-key (-> (random-uuid) str) :initial-state {:cnt 0} :wcar-opts {:spec {:port port} :pool :none} :snapshot-every 2 :business-fn (fn [state _event _timestamp] (update state :cnt inc))} store1 (sut/store! default-opts) store2 (sut/store! default-opts) times 10] (dotimes [_ times] (sut/handle! store1 {:op :inc}) (sut/handle! store2 {:op :inc})) (is (= [{:cnt (- (* times 2) 1)} {:cnt (* times 2)}] [@store1 @store2])))))In this counter test, we define key parameters:
:store-key: The key used in Redis.:initial-state: The starting Desired State (e.g.,{:cnt 0}).:snapshot-every: Defines how frequently a state snapshot is saved to Redis, dramatically accelerating recovery.:business-fn: The pure function that defines how the state changes based on an event. Purity is critical; it guarantees that replaying events (for recovery or auditing) yields the exact same Desired State.
Notice how two concurrent writers (store1 and store2) process events via the handle! function, yet the final state remains consistent. This is the power of a well-architected control plane store. The desired state can grow to encompass everything: your list of developers, teams, services, policies—all in one place.
BigConfig Workflow
Section titled “BigConfig Workflow”The final component is the control loop. It is a dedicated reader of the desired state. Its sole purpose is to observe the actual state of your infrastructure and ensure it matches the desired state specified in the store.
BigConfig seamlessly integrates here.
BigConfig workflow takes the comprehensive desired state, renders it into the necessary configuration files (reused from an existing Terraform or Ansible GitOps project), and applies them to the target environment. You keep your existing configuration logic while BigConfig provides the intelligence and state management above it.
The result? You upgrade from configuration automation to a more dynamic, auditable, and consistent control plane without the nightmare of a complete rewrite.
Example code
Section titled “Example code”I’m working on the code right now. Stay tuned, I will update the blog post as soon as I have the first version.
Conclusion
Section titled “Conclusion”The shift from GitOps to a full-fledged control plane is the inevitable evolution for engineering teams grappling with growing complexity, but it doesn’t have to be a painful, expensive overhaul. BigConfig provides the crucial bridge, allowing you to move beyond simple configuration automation without sacrificing your investment in existing Terraform or Ansible logic. By introducing a robust control plane built on a flexible desired state, an immutable audit log (stored efficiently in Redis Sorted Sets), and a workflow engine, BigConfig ensures consistency and concurrency even as your system scales. This model separates the declaration of what you want from the execution of how to get it, enabling you to gain the dynamic, auditable, and resilient desired state persistence of a modern control plane while retaining your established configuration workflows.
Would you like to have a follow-up on this topic? What are your thoughts? I’d love to hear your experiences.