This is the multi-page printable view of this section. Click here to print.

Return to the regular view of this page.

Concepts

Learn about Kowabunga conceptual architecture

Conceptual Architecture

Simply put, Kowabunga allows you to control and manage low-level infrastructure at your local on-premises data-centers and spin up various virtual resources on top, as to leverage your applications on top.

Local data centers consist of a bunch of physical machines (can range from personal computers, commodity hardware to high-end enterprise-grade servers) providing raw networking, computing and storage resources. Physical assests plainly sit in your basement. They don’t need to be connected to other data-centers, they don’t even need to know about others data-centers’ existence and more than anything, they don’t need to be exposed to public Internet.

From an IT and assets management’s perspective, one simply needs to ensure they run and, capacity planning in mind, that they do offer enough physical resources to sustain future applications hosting needs.

On each data-center, some physical machines (usually lightweight) will be dedicated to providing networking kind of services, through Kowabunga’s Kiwi agents, while others will providing computing and storage capabilities, thanks to Kowabunga’s Kaktus agents.

The Kowabunga project then come with Kahuna, its orchestration engine. This is the masterpiece cornerstone of your architecture. Kahuna act as a maestro, providing API servicess for admins and end-users, and provising and controlling virtual resources on the various data-centers through Kowabunga connected agents.

Ultimately, DevOps consumers will only ever interface with Kahuna.

So, how does magic happen ?

Kahuna has a triple role exposure:

  • Public REST API: implements and operates the API calls to manage resources, DevOps-orchestrated, manually (not recommended) or through automation tools such as Terraform, OpenTofu or Ansible.
  • Public WebSocket endoint: agent connection manager, where the various Kowabunga agents (from managed data-centers) establish secure WebSocket tunnels to, for being further controlled, bypassing on-premises firewall constraints and preventing the need of any public service exposure.
  • Metadata endpoint: where managed virtual instances and services can retrieve information services and self-configure themselves.

Core Components

So, let’s rewind, the Kowabunga projects consists of multiple core components:

  • Kahuna: the core orchestration system. Remotely controls every resource and maintains ecosystem consistent. Gateway to the Kowabunga REST API.
  • Kaktus: the HCI node(s). Provides KVM-based virtual computing hypervisor with Ceph-based distributed storage services.
  • Kiwi: the SD-WAN node(s). Provides various network services like routing, firewall, DHCP, DNS, VPN, peering (with active-passive failover).
  • Koala: the WebUI. Allows for day-to-day supervision and operation of the various projects and services.

Aside from these, Kowabunga introduces the concept of:

  • Region: basically a physical location, which can be assimilated to a data-center.
  • Zone: a specific subset of a region, where all underlying resources are guaranteed to be self-autonomous (in terms of Internet connectivity, power-supply, cooling …). As with other Cloud providers, the zones allow for application workload distribution within a single region, offering resilience and high-availability.

Topology Uses Cases

This illustrates what a Kowabunga Multi-Zones and Regions topology would looks like:

On the left side, one would have a multi-zones region. Divided into 3 Zones (i.e. 3 physically isolated data-centers, physically inter-connected by a network link), the region features 11 servers instances:

  • 2 Kiwi instances, providing networking capabilities
  • 3x3 Kaktus instances, providing computing and storage capabilities.

Zones can be pictured in different ways:

  • several floors from your personal home basement (ok … useless … but for the sake of example).
  • several IT rooms from your company’s office.
  • several buildings from your company’s office.

Should a Kowabunga user request for a virtual machine creation in this dedicated region, he could specifically request it to be assigned to one of the 3 zones (the underlying hypervisor from each zone will be automatically picked), or request some -as-a-service feature, which would be seamlessly spawned in multiple zones, as to provide service redundancy.

Sharing the same L2/L3 network across the same region, disk instances will be distributed and replicating across zones, allowing for fast instance relocation in the event of one zone’s failure.

On the right side, one would have a single-zone region, with just a couple of physical instances.

What Makes it Different ?

Cloud providers aside, what makes Kowabunga different from other on-premises infrastructure and virtualization providers (such as VMware, Nutanix, OpenStack …).

Well … 0 licensing costs. Kowabunga is Open Source with no paywalled features. There’s no per-CPU or per-GB or memory kind of license. Whether you’d like to set your private small-sized copamy’s data-center with 3 servers or full fleet of 200+, your cost of operation will remain flat.

But aside from cost, Kowabunga has been developed by and for DevOps, the ones who:

  • need to orchestrate, deploy and maintain heterogenous applications on heterogenous infrastructures.
  • use Infrastructure-as-Code principles to ensure reliability, durability and traceability.
  • bear security in mind, ensuring than nothing more than what’s required must be publicly exposed.
  • believe than smaller and simpler is better.

1 - Kahuna

Learn about Kahuna orchestrator.

Kahuna is Kowabunga’s orchestration system. Its name takes root from Hawaiian’s (Big) Kahuna word, meaning “the expert, the most dominant thing”.

Kahuna remotely controls every resource and maintains ecosystem consistent. It’s the gateway to Kowabunga REST API.

From a technological stack perspective, Kahuna features:

  • a Caddy public HTTPS frontend, reverse-proxying requests to:
    • Koala Web application, or
    • Kahuna orchestrator daemon
  • a MongoDB database backend.

The Kahuna orchestrator features:

  • Public REST API handler: implements and operates the API calls to manage resources,interacting with rightful local agents through JSON-RPC over WSS.
  • Public WebSocket handler: agent connection manager, where the various agents establish secure WebSocket tunnels to, for being further controlled, bypassing on-premises firewall constraints and preventing the need of any public service exposure.
  • Metadata endpoint: where managed virtual instances and services can retrieve information services and self-configure themselves.

Kowabunga API folds into 2 type of assets:

  • admin ones, used to handle objects like region, zone, kaktus and kiwi hosts, agents, networks …
  • user ones, used to handle objects such as Kompute, Kawaii, Konvey

Kahuna implements robust RABC and segregation of duty as to ensure access boundaries, such as:

  • Nominative RBAC capabilities and per-organization and team user management.
  • Per-project teams associationfor per-resource access control.
  • Support for both JWT bearer (human-to-server) and API-Key token-based (server-to-server) authentication mechanisms.
  • Support for 2-steps account creation/validation and enforced robust passwords/tokens usage(server-generated, user-input is prohibited).
  • Nominative robust HMAC ID+token credentials over secured WebSocket agent connections.

This ensures that:

  • only rightful designated agents are able to establish WSS connections with Kahuna
  • created virtual instances can only retrieve the metadata profile they belong to (and self configure or update themselves at boot or runtime).
  • users can only see and manage resources for the projects they belong to.

2 - Koala

Learn about Koala Web application.

Koala is Kowabunga’s WebUI. It allows for day-to-day supervision and operation of the various projects and services.

Koala

But should you ask a senior DevOps / SRE / IT admin, fully automation-driven, he’d damn anyone who’d have used the Web client to manually create/edit resources and messes around his perfecly maintained CasC.

We’ve all been there !!

That’s why Koala has been designed to be read-only. While using Kowabunga’s API, the project’s directive is to enforce infrastructure and configuration as code, and such, prevents any means to do harm.

Koala is AngularJS based and usually located next to Kahuna’s instance. It provides users with capability to connect, check for the various projects (they belong to) resources, optionnally start/reboot/stop them and/or see various piece of information and … that’s it ;-)

3 - Kiwi

Learn about Kiwi SD-WAN node.

Kiwi is Kowabunga SD-WAN node in your local data-center. It provides various network services like routing, firewall, DHCP, DNS, VPN and peering, all with active-passive failover (ideally over multiple zones).

Kiwi is central to our regional infrastructure to operate smoothly and internal gateway to all your projects Kawaii private network instances. It controls the local network configuration and creates/updates VLANs, subnets and DNS entries per API requests.

Kiwi offers a Kowabunga project’s network isolation feature by enabling VLAN-bound, cross-zones, project-attributed, VPC L3 networking range. Created virtual instances and services are bound to VPC by default and never publicly exposed unless requested.

Access to project’s VPC resources is managed either through:

  • Kiwi-managed region-global VPN tunnels.
  • Kawaii-managed project-local VPN tunnels.

Decision to do or another depends on private Kowabunga IT policy.

4 - Kaktus

Learn about Kaktus HCI node.

Kaktus stands for Kowabunga Amazing KVM and TUrnkey Storage (!!), basically, our Hyper-Converged Infrastructure (HCI) node.

While large virtualization systems such as VMware usually requires you to dedicate servers as computing hypervisors (with plenty of CPU and memory) and associate them with remote, extensive NAS or vSAN, providing storage, Kowabunga follows the opposite approach. Modern hardware is powerful enough to handle both computing and storage.

This approach allows you to:

  • use commodity hardware, if needed
  • use heterogenous hardware, each member of the pool featuring more or less computing and storage resources.

If you’re already ordering a heavy computing rackable server, extending it with 4-8 SSDs is always going to be cheaper than adding an extra enterprise SAN.

Kaktus nodes will then consists of

  • a KVM/QEMU + libvirt virtualization computing stack. Featuring all possible VT-x and VT-d assistance on x86_64 architectures, it’ll provide near passthrough virtualization capabilities.
  • several local disks, to be part of a region-global Ceph distributed storage cluster.
  • the Kowabunga Kaktus agent, connected to Kahuna

From a pure low-level software perspective, our virtualization stack relies on 3 stacks:

  • Linux Network Bridging driver, for virtual interfaces access to host raw network interfaces and physical network.
  • Linux KVM driver, for CPU VT-X extension support and improved virtualization performances.
  • RBD (Rados Block Device) driver, for storing virtual block devices under distributed Ceph storage engine. QEMU drives these different backends to virtualize resources on to.

Kaktus Topology

Now QEMU being a local host process to be spawned, we need some kind of orchestration layer on top of that. Here comes libvirt. libvirt provides an API over TCP/TLS/SSH that wraps virtual machines definition over an XML representation that can be fully created/updated/destroyed remotely, controlling QEMU underneath. Kaktus agent controls the local KVM hypervisor through libvirt backend and the local-network distributed Ceph storage, allowing management of virtual machines and disks.