Ugrás a fő tartalomhoz

Hybrid Cloud and OpenStack Private Cloud

Basic Advanced

Service: Hybrid Cloud Architecture · OpenStack Private Cloud · Typical shape: design sprint (2 to 4 weeks), pilot region (6 to 10 weeks), then migration waves and managed day-2 operations

What we deliver​

  • OpenStack private cloud on commodity hardware and Ubuntu: compute, software-defined networking, block/object storage on Ceph, load balancing, and Kubernetes-as-a-service, deployed and upgraded by automation.
  • VMware exit path: discovery, wave planning, and automated VM conversion onto OpenStack.
  • Hybrid architecture: sensitive data and steady workloads on-premise, burst and managed services in STACKIT, AWS, or OCI, connected by private, encrypted networking.
  • Unified management: one IaC codebase, one Kubernetes lifecycle model (Cluster API), one observability plane across all environments.
  • GPU capacity for private AI via PCI passthrough or vGPU flavors.

Reference architecture​

Core OpenStack services​

ServiceRoleOur default
KeystoneIdentity, projects, app credentialsFederated to client IdP via OIDC/SAML
NovaCompute (KVM/libvirt)Host aggregates per hardware class
Neutron + OVNSDN, routers, security groups, floating IPsML2/OVN, distributed routing
CinderBlock volumesCeph RBD backend, volume types per pool
GlanceImagesCeph RBD backend (copy-on-write clones)
Swift API / Ceph RGWObject storage, S3 APICeph RGW with Keystone auth
OctaviaLoad balancers (L4/L7)Amphora or OVN provider
BarbicanSecrets, certificatesNeeded by Octavia TLS and Magnum CAPI
Magnum + Cluster APIKubernetes as a servicemagnum-capi-helm driver, or CAPO directly
DesignateDNS as a serviceOptional, for tenant self-service
Horizon / SkylineWeb UIHorizon; API and IaC preferred

Release landscape (as of 2026-10)​

SeriesReleasedSLURPStatus
2026.2 Hibiscus2026-09-30NoMaintained
2026.1 Gazpacho2026-04-01YesMaintained
2025.2 Flamingo2025-10-01NoMaintained
2025.1 Epoxy2025-04-02YesMaintained
2027.1 Indri2027-03-24 (est.)YesIn development

SLURP (Skip Level Upgrade Release Process) releases can be upgraded directly to the next SLURP, skipping the intermediate .2 release. For most clients we deploy SLURP releases only and upgrade yearly: 2025.1 → 2026.1 → 2027.1.

Choosing a deployment method​

OptionWhat it isChoose whenNotes
Canonical OpenStack (Sunbeam)Snap-based, OpenStack control plane on Kubernetes managed by JujuUbuntu shop, wants vendor support (Ubuntu Pro), small to mid-size cloudsLTS releases 2024.1 and 2026.1; 2024.1 runs to Apr 2029 (standard), Apr 2034 (Ubuntu Pro), Apr 2036 (Legacy Support). 2024.1/stable is the snap stable channel (as of 2026-10); the 2026.1 snap channel is not listed on the snap page, so check channels before committing
Kolla-AnsibleUpstream, OpenStack services in containers, deployed with AnsibleTeams that want upstream, full control, any sizekolla-ansible 22.x is the current stable series (22.2.0, 2026-09-08); check its release notes for the Ansible versions it needs
OpenStack-HelmUpstream Helm charts for OpenStack on KubernetesClient already runs Kubernetes well and wants one operational modelMost Kubernetes-native; most moving parts
Red Hat OpenStack Services on OpenShift (RHOSO) 18OpenStack control plane as operators on OpenShiftRed Hat subscription estateRequires an OpenShift cluster and an external Ceph cluster; check the RHOSO 18 planning guide for supported versions

Our default for "enterprise OpenStack on Ubuntu" is Canonical OpenStack when the client wants a commercial support contract, and Kolla-Ansible on Ubuntu 24.04 when the client wants upstream releases on day one or a feature not yet in the Canonical stable channel. Check the Canonical snap channel for the target release before committing to a date.

Implementation steps​

1. Discovery and sizing​

Collect: VM inventory (vCPU, RAM, disk, IOPS, growth), network segments, storage tiers, GPU needs, RPO/RTO, compliance zones. Export from vCenter with RVTools or govc.

Sizing rules of thumb we start from, then validate against the inventory:

ItemStarting rule
Control plane3 nodes (odd number for Galera/RabbitMQ/OVN DB quorum), 16+ cores, 128 GB RAM, NVMe
CPU overcommit4:1 general purpose, 1:1 for latency-sensitive and GPU flavors (hw:cpu_policy=dedicated)
RAM overcommit1:1 (no memory overcommit in production)
Ceph raw capacityusable needed × 3 (replica 3) ÷ 0.75 (stay below ~75 % full for recovery headroom)
Ceph OSD nodesat least 4 for replica 3 with one-node failure headroom; NVMe for RBD pools
Network2 × 25 GbE bonded per node minimum; separate VLANs for management, storage public, storage cluster, tenant overlay, provider
Failure domainsat least 3 racks or power zones mapped to Ceph CRUSH and Nova availability zones

2. Bare metal and OS​

  • Provision with MAAS (Canonical) or Ironic/Bifrost; Ubuntu 24.04 LTS on all nodes.
  • Netplan bonds with LACP, jumbo frames on storage and overlay networks, NTP (chrony) on every node.
  • Firmware, BIOS (VT-d/IOMMU on for GPU nodes), and RAID settings captured as code.

3a. Deploy with Canonical OpenStack (Sunbeam)​

# On the first node (Ubuntu 24.04). Pick the channel that matches the target release.
sudo snap install openstack --channel 2024.1/stable
sunbeam prepare-node-script --bootstrap | bash -x && newgrp snap_daemon
sunbeam cluster bootstrap --role control,compute,storage # interactive or with --manifest
sunbeam cluster add <node-fqdn> # prints a join token
# on the new node:
sunbeam cluster join --role compute,storage <token>
sunbeam configure --openrc demo-openrc
# cluster add/join and enable subcommands not confirmed against the snap docs; check `sunbeam <cmd> --help`
sunbeam enable orchestration
sunbeam enable loadbalancer
sunbeam enable vault && sunbeam enable secrets
sunbeam enable dns "ns1.example.com."
sunbeam enable observability embedded

Keep the deployment manifest (manifest.yaml) in Git; it carries network ranges, Ceph disks, and feature settings.

3b. Deploy with Kolla-Ansible​

python3 -m venv ~/kolla && . ~/kolla/bin/activate
pip install 'kolla-ansible==22.*' # 2026.1 Gazpacho series
kolla-ansible install-deps
cp -r ~/kolla/share/kolla-ansible/etc_examples/kolla /etc/kolla
cp ~/kolla/share/kolla-ansible/ansible/inventory/multinode .
kolla-genpwd # then move passwords.yml into SOPS/OpenBao
kolla-ansible bootstrap-servers -i multinode
kolla-ansible prechecks -i multinode
kolla-ansible deploy -i multinode
kolla-ansible post-deploy -i multinode
/etc/kolla/globals.yml (excerpt)
kolla_base_distro: "ubuntu"
openstack_release: "2026.1"
kolla_internal_vip_address: "10.10.0.250"
kolla_external_vip_address: "203.0.113.10"
network_interface: "bond0.100"
neutron_external_interface: "bond1"
neutron_plugin_agent: "ovn"
enable_cinder: "yes"
enable_octavia: "yes"
enable_barbican: "yes"
enable_magnum: "yes"
enable_designate: "yes"
glance_backend_ceph: "yes"
cinder_backend_ceph: "yes"
nova_backend_ceph: "yes"
kolla_enable_tls_internal: "yes"
kolla_enable_tls_external: "yes"

4. Ceph storage​

Run Ceph Tentacle (20.2.x, latest 20.2.4 as of 2026-10) deployed by cephadm, MicroCeph (Sunbeam), or Rook (OpenStack-Helm).

cephadm bootstrap --mon-ip 10.20.0.11 --cluster-network 10.30.0.0/24
ceph orch host add stor-02 10.20.0.12
ceph orch apply osd --all-available-devices
ceph osd pool create volumes 256 && ceph osd pool application enable volumes rbd
ceph osd pool create images 64 && ceph osd pool application enable images rbd
ceph osd pool create vms 256 && ceph osd pool application enable vms rbd
ceph osd pool set volumes pg_autoscale_mode on
ceph auth get-or-create client.cinder mon 'profile rbd' osd 'profile rbd pool=volumes, profile rbd pool=vms, profile rbd-read-only pool=images'
  • Map CRUSH failure domain to rack (or host on small clusters).
  • Use erasure coding for RGW/object and backup pools; keep RBD for VMs on replica 3.
  • Apply Ceph point releases promptly; 20.2.4 (2026-08-19) is the latest Tentacle point release.

5. Tenancy, quotas, and IaC​

Everything after the platform install is OpenTofu with the openstack provider: domains, projects, quotas, flavors, networks, images.

infra/openstack/project.tf
resource "openstack_identity_project_v3" "team_a" {
name = "team-a-prod"
domain_id = var.domain_id
}

resource "openstack_compute_quotaset_v2" "team_a" {
project_id = openstack_identity_project_v3.team_a.id
cores = 256
ram = 1048576
instances = 100
}

resource "openstack_compute_flavor_v2" "gp_8x32" {
name = "gp.8x32"
vcpus = 8
ram = 32768
disk = 0 # boot from Cinder volume
is_public = true
}

resource "openstack_networking_network_v2" "app" {
name = "app-net"
tenant_id = openstack_identity_project_v3.team_a.id
}

6. Kubernetes on OpenStack​

Two supported paths, both based on Cluster API:

  • Magnum with magnum-capi-helm: tenants request clusters through the Magnum API; the driver renders Helm charts that create Cluster API resources on a management cluster. Needs Octavia (API load balancers) and Barbican (certificates).
  • Cluster API Provider OpenStack (CAPO) directly: platform team manages clusters as YAML in Git. Same model works for AWS (CAPA) and other providers, which is what makes hybrid management uniform. CAPO v0.15 (released 2026-09-17) is the current line: it adds the v1beta2 API types alongside v1beta1 and implements the Cluster API v1beta2 contract; v0.14 still gets patch releases (as of 2026-10). Core Cluster API v1beta1 is on track to be unserved in CAPI v1.16, so write new manifests against v1beta2.
clusters/onprem-prod/cluster.yaml (excerpt)
apiVersion: cluster.x-k8s.io/v1beta2
kind: Cluster
metadata:
name: onprem-prod
spec:
clusterNetwork:
pods: { cidrBlocks: ["10.244.0.0/16"] }
infrastructureRef:
apiGroup: infrastructure.cluster.x-k8s.io
kind: OpenStackCluster
name: onprem-prod
controlPlaneRef:
apiGroup: controlplane.cluster.x-k8s.io
kind: KubeadmControlPlane
name: onprem-prod-cp

Inside each cluster: OpenStack Cloud Controller Manager, Cinder CSI, and Octavia ingress or a Gateway API implementation (see Cloud-native Development).

7. GPU flavors for private AI​

ModeUse forNova mechanism
PCI passthroughFull GPU per VM: LLM inference, fine-tuning[pci] device_spec + alias, flavor pci_passthrough:alias
vGPU (mdev)Shared GPU slices: notebooks, small models, VDI[devices] enabled_mdev_types, flavor resources:VGPU=1
SR-IOV vGPU (Ampere and newer NVIDIA)Shared slices on modern GPUsNot covered here; follow the current Nova vGPU docs (unverified)
nova.conf on GPU compute nodes (passthrough)
[pci]
device_spec = { "vendor_id": "10de", "product_id": "2330" }
alias = { "vendor_id": "10de", "product_id": "2330", "device_type": "type-PCI", "name": "h100", "numa_policy": "preferred" }

[filter_scheduler]
enabled_filters = ComputeFilter,ImagePropertiesFilter,ServerGroupAntiAffinityFilter,ServerGroupAffinityFilter,PciPassthroughFilter,NUMATopologyFilter
openstack flavor create gpu.h100x1 --vcpus 24 --ram 196608 --disk 0 \
--property "pci_passthrough:alias"="h100:1" \
--property hw:cpu_policy=dedicated \
--property hw:mem_page_size=large

Enable IOMMU on the host kernel command line (intel_iommu=on iommu=pt or amd_iommu=on), bind the GPU to vfio-pci, and put GPU hosts in their own aggregate. The serving stack on top is covered in Private and Sovereign AI and Serving at Scale.

8. VMware exit migration​

Tooling (open source first):

ToolNotes
os-migrate vmware-migration-kitAnsible collection (os_migrate.vmware_migration_kit), maintained by the Red Hat OpenStack migration team; default path is nbdkit on a conversion host with optional VMware CBT for near-zero-downtime two-phase cutover, alternatively virt-v2v with or without a conversion host; result lands as Cinder volumes or Glance images
virt-v2v + nbdkitConverts guests (drivers, bootloader, VirtIO) and streams VMDK data without full download first
Coriolis, VEXXHOST MigrateKit, vJailbreakListed in the OpenInfra VMware migration guide; evaluate per estate size and support needs
migrate/wave-01.vars.yml
# Variable names from the collection's vars.yaml example; check them against the pinned collection version.
vcenter_hostname: vcenter.corp.local
vcenter_datacenter: DC1
vddk_libdir: /usr/lib/vmware-vix-disklib
already_deploy_conversion_host: true
os_migrate_vmw_data_dir: /opt/os-migrate
use_existing_flavor: true
vms_list: [app-db-01, app-web-01, app-web-02]
ansible-galaxy collection install os_migrate.vmware_migration_kit
ansible-playbook -i inventory.yml os_migrate.vmware_migration_kit.migration \
-e @secrets.yml -e @migrate/wave-01.vars.yml # vCenter + dst_cloud credentials in secrets.yml (SOPS)

Per-wave checklist: VirtIO drivers present (Windows: install virtio-win before conversion), static IPs mapped to Neutron ports, MAC addresses preserved where licences depend on them, application owner sign-off after smoke tests, rollback (power the source VM back on) rehearsed. Use incremental/warm migration for large disks to keep cutover windows short.

9. Hybrid connectivity​

NeedPattern
Site-to-site private network to a hyperscalerIPsec VPN (strongSwan or a VPN appliance on OpenStack) to the cloud VPN gateway, BGP for route exchange, two tunnels for redundancy
Low-overhead links between our own sites and clustersWireGuard (kernel module), routed with BGP (FRR or BIRD)
Dedicated bandwidthProvider interconnect (AWS Direct Connect, STACKIT and OCI interconnect offerings) with BGP
Publishing internal apps without inbound firewall holesCloudflare Tunnel (cloudflared) plus Cloudflare Access for identity-aware access
Kubernetes service-to-service across cloudsCilium Cluster Mesh or a service mesh with multi-cluster support
/etc/wireguard/wg0.conf (on-prem router)
[Interface]
Address = 169.254.10.1/30
ListenPort = 51820
PrivateKey = <from OpenBao>

[Peer]
PublicKey = <peer public key>
Endpoint = 198.51.100.20:51820
AllowedIPs = 169.254.10.2/32, 10.60.0.0/16
PersistentKeepalive = 25
frr.conf (BGP over the tunnel)
router bgp 65010
neighbor 169.254.10.2 remote-as 65020
neighbor 169.254.10.2 bfd
address-family ipv4 unicast
network 10.10.0.0/16
neighbor 169.254.10.2 prefix-list FROM-CLOUD in
exit-address-family
cloudflared config.yml
tunnel: 6f1c... # tunnel UUID
credentials-file: /etc/cloudflared/6f1c.json
ingress:
- hostname: grafana.example.com
service: http://grafana.monitoring.svc:3000
- service: http_status:404

Plan IP space for all environments up front (non-overlapping RFC 1918 ranges per site and cloud); renumbering later is the most expensive hybrid change.

10. Unified management​

  • One IaC repo, one root module per environment, providers for openstack, stackit, aws, and cloudflare. Shared modules expose the same inputs (network, cluster, DNS) regardless of the target.
  • One cluster lifecycle: a Cluster API management cluster with CAPO for on-prem and the matching provider (or the managed service's API via OpenTofu) for public clouds; Argo CD ApplicationSets deploy the same add-ons (CNI, CSI, cert-manager, ESO, OTel Collector) to every cluster.
  • One observability plane: each site runs Prometheus/OTel Collector agents that remote-write to a central Mimir (or Thanos) and ship logs to Loki and traces to Tempo; Grafana is the single pane. OpenStack itself is monitored with openstack-exporter, node_exporter, Ceph's Prometheus module, libvirt and OVN exporters.
  • Placement policy in an ADR: which data classes may run where (for example personal data on-prem or STACKIT only; stateless burst in public cloud).

11. Day-2 operations​

TaskPractice
UpgradesSLURP to SLURP yearly; rehearse on a staging region built from the same code; upgrade control plane, then computes in batches with live migration
PatchingMonthly OS patching with rolling host evacuation (openstack compute service set --disable, live-migrate, reboot)
CapacityDashboards for allocation vs. physical per aggregate; alert at 70 % CPU/RAM allocation and 70 % Ceph raw usage
BackupsDatabase dumps (Galera/MariaDB) and config nightly; tenant backups via Cinder backup to RGW or an external backup tool
HealthRally/Tempest smoke tests nightly; alert on failed API probes, RabbitMQ queue growth, OVN DB leader changes, Ceph HEALTH_WARN
Certificatescert-manager or the deployer's TLS automation; alert 21 days before expiry
Example alerts
# Ceph nearing full
ceph_cluster_total_used_bytes / ceph_cluster_total_bytes > 0.70
# Nova compute service down
openstack_nova_agent_state{adminState="enabled"} == 0

Best practices​

  • Deploy only SLURP releases unless a client needs a feature from a .2 release.
  • Keep the platform reproducible: if a staging region cannot be rebuilt from Git in a day, fix that before production.
  • Separate control plane and data plane networks physically or by VLAN with QoS; storage traffic should never compete with tenant traffic.
  • Federate Keystone to the client IdP and use application credentials for automation instead of user passwords.
  • Treat GPU hosts as a separate failure and maintenance domain (no live migration with passthrough devices; drain by rescheduling).
  • Document data placement and connectivity in ADRs and C4 diagrams per How We Build.

Pitfalls to design around​

  • Quorum: run three control nodes and three Ceph monitors so a single failure keeps the cloud writeable.
  • MTU: set the physical MTU (9000) high enough that the Geneve overlay inside still offers 1500 to tenants, and test it end to end.
  • Ceph headroom: plan to stay below roughly 75 % raw usage so a node failure can re-replicate without hitting the full ratio.
  • Release availability: confirm the target release exists in the chosen distribution's stable channel before scheduling the install.
  • Windows guests: install VirtIO drivers and check licence activation behaviour before the migration wave, not during it.
  • Overlapping IP ranges: agree an address plan across all clouds during design.

Definition of done​

  • Hardware, network, and IP plan documented; failure domains mapped to Ceph CRUSH and Nova AZs
  • OpenStack deployed from Git (Sunbeam manifest or Kolla globals.yml + inventory); secrets in OpenBao or SOPS
  • Release is a SLURP release (or an ADR records why not); upgrade path documented
  • Keystone federated to the client IdP; RBAC roles and quotas in OpenTofu
  • Ceph healthy, pools sized, alerts at 70 % raw; restore from backup tested
  • Octavia, Barbican, and Kubernetes-as-a-service working; a test cluster created and deleted via the API
  • GPU flavors (if in scope) schedule correctly and pass a CUDA smoke test in a guest
  • VMware migration: pilot wave converted, validated, signed off; runbook for later waves
  • Hybrid links redundant (two tunnels or paths), BGP with BFD, failover tested
  • Central observability shows every site and cloud; SLOs for API availability and VM provisioning time
  • Rally/Tempest smoke tests scheduled; on-call runbooks written for the top alerts
  • Day-2 handbook (patching, upgrades, capacity, backup) handed over and walked through with the operations team

Sources​