From c5f8b0b26b8380c056f9f0014c0c54f850f147ef Mon Sep 17 00:00:00 2001 From: Benjamin Ritter Date: Wed, 10 Jun 2026 15:19:24 +0000 Subject: [PATCH 01/12] feat: add firewall explanation Signed-off-by: Benjamin Ritter --- docs/05-Concepts/03-Network/02-firewalls.md | 48 +++++++++++++++++++++ 1 file changed, 48 insertions(+) diff --git a/docs/05-Concepts/03-Network/02-firewalls.md b/docs/05-Concepts/03-Network/02-firewalls.md index e07cbd11..1bbe5dda 100644 --- a/docs/05-Concepts/03-Network/02-firewalls.md +++ b/docs/05-Concepts/03-Network/02-firewalls.md @@ -7,4 +7,52 @@ draft: true # Firewalls +Firewalls in metal-stack are a kind of machine that connects networks to each other or the internet and allows traffic filtering. + +Only firewalls can have multiple networks attached. + +Firewalls do not require specialized hardware. For most scenarios any of your Machine SKUs can be used as a Firewall. + +:::important +metal-stack does not support using existing firewall appliances for east-west traffic within metal-stack. + +It is technically possible to use an existing firewall appliance to filter north-south traffic. + +Nevertheless we encourage you to use metal-stack firewalls for north-south traffic as well to get the benefits of lower TCO due to use of commodity hardware and end-to-end configurability using metal-api. +::: + +## Why do we use Machines as Firewalls? + +### Why not integrate existing firewall appliances? + +We see limited benefit in using firewall appliances for the following reasons: + +- Firewall appliances are a fragmented product category without common APIs or common behaviour, which would bind development time supporting different vendor specific implementations +- Appliances and bandwidth would be shared between tenants, increasing management complexity and reducing tenant isolation + + +### Why not use the ToR Leaf switches? + +SONiC can provide basic L3/L4 packet filtering capabilities using its ACLs, which can be offloaded to the switching silicon. But unfortunately that is not enough for our requirements: + +- Switch ASICs vary widely in supported maximum number of active rules and supported ACL features +- We require NAT, which is not supported when offloading to ASICs +- Going past stateless filtering to stateful tracking requires specialized DPU hardware, increasing cost and reducing accessibility + +- We offer Intrusion Detection as a feature, which is also not possible while utilizing offloading +- Using dedicated machines allows us to give you the same great isolation guarantees as with regular machines. If you provision the Firewall, you do not share resources or access with other tenants. + +In short, to offer comparable features to our current solution, we would need to disable ASIC offloading and either punt all traffic to either the weak main switch CPU, causing unpredictable performance or use specialized DPUs. + +## Creating a Firewall + +## Firewall Controller + +The firewall-controller allows you to configure Firewalls using CRDs from inside of Kubernetes clusters. Firewall configuration provided by the firewall-controller is hot reloaded. + + +## Lifecycle + +Firewall Machines are managed by metal-stack. The local state of Firewall machines is ephemeral, as the authoritative configuration is stored in metal-api. Manual changes to the configuration are not supported and will be overridden. Use metalctl or Firewall CRDs to apply changes to firewall configuration. + (fire-walling in metal-stack, firewall-controller and headscale integration) From 0f0f2cb96df3f5f9b6facfa5d106d480e1971b78 Mon Sep 17 00:00:00 2001 From: Benjamin Ritter <7373184+l0wl3vel@users.noreply.github.com> Date: Thu, 11 Jun 2026 12:20:53 +0200 Subject: [PATCH 02/12] Update docs/05-Concepts/03-Network/02-firewalls.md Co-authored-by: Valentin Knabel --- docs/05-Concepts/03-Network/02-firewalls.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/docs/05-Concepts/03-Network/02-firewalls.md b/docs/05-Concepts/03-Network/02-firewalls.md index 1bbe5dda..35701420 100644 --- a/docs/05-Concepts/03-Network/02-firewalls.md +++ b/docs/05-Concepts/03-Network/02-firewalls.md @@ -7,7 +7,7 @@ draft: true # Firewalls -Firewalls in metal-stack are a kind of machine that connects networks to each other or the internet and allows traffic filtering. +Firewalls in metal-stack are machines that connect networks to each other or the internet and allows traffic filtering. Only firewalls can have multiple networks attached. From c1cb857f509ee057baea11ac30e99403efdddf5e Mon Sep 17 00:00:00 2001 From: Benjamin Ritter <7373184+l0wl3vel@users.noreply.github.com> Date: Thu, 11 Jun 2026 12:21:23 +0200 Subject: [PATCH 03/12] Update docs/05-Concepts/03-Network/02-firewalls.md Co-authored-by: Valentin Knabel --- docs/05-Concepts/03-Network/02-firewalls.md | 1 - 1 file changed, 1 deletion(-) diff --git a/docs/05-Concepts/03-Network/02-firewalls.md b/docs/05-Concepts/03-Network/02-firewalls.md index 35701420..4f792ff9 100644 --- a/docs/05-Concepts/03-Network/02-firewalls.md +++ b/docs/05-Concepts/03-Network/02-firewalls.md @@ -38,7 +38,6 @@ SONiC can provide basic L3/L4 packet filtering capabilities using its ACLs, whic - Switch ASICs vary widely in supported maximum number of active rules and supported ACL features - We require NAT, which is not supported when offloading to ASICs - Going past stateless filtering to stateful tracking requires specialized DPU hardware, increasing cost and reducing accessibility - - We offer Intrusion Detection as a feature, which is also not possible while utilizing offloading - Using dedicated machines allows us to give you the same great isolation guarantees as with regular machines. If you provision the Firewall, you do not share resources or access with other tenants. From 57469cc8d82bc588bdd93f7f50cae2cbeb2a90a9 Mon Sep 17 00:00:00 2001 From: Benjamin Ritter <7373184+l0wl3vel@users.noreply.github.com> Date: Thu, 11 Jun 2026 12:21:48 +0200 Subject: [PATCH 04/12] Update docs/05-Concepts/03-Network/02-firewalls.md Co-authored-by: Valentin Knabel --- docs/05-Concepts/03-Network/02-firewalls.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/docs/05-Concepts/03-Network/02-firewalls.md b/docs/05-Concepts/03-Network/02-firewalls.md index 4f792ff9..57f10cfa 100644 --- a/docs/05-Concepts/03-Network/02-firewalls.md +++ b/docs/05-Concepts/03-Network/02-firewalls.md @@ -39,7 +39,7 @@ SONiC can provide basic L3/L4 packet filtering capabilities using its ACLs, whic - We require NAT, which is not supported when offloading to ASICs - Going past stateless filtering to stateful tracking requires specialized DPU hardware, increasing cost and reducing accessibility - We offer Intrusion Detection as a feature, which is also not possible while utilizing offloading -- Using dedicated machines allows us to give you the same great isolation guarantees as with regular machines. If you provision the Firewall, you do not share resources or access with other tenants. +- Using dedicated machines allows us to provide the same great isolation guarantees as with regular machines. If you provision the Firewall, you do not share resources or access with other tenants. In short, to offer comparable features to our current solution, we would need to disable ASIC offloading and either punt all traffic to either the weak main switch CPU, causing unpredictable performance or use specialized DPUs. From 56996d25fc391cbcc1aee237eda82ab52194d75a Mon Sep 17 00:00:00 2001 From: Benjamin Ritter Date: Thu, 11 Jun 2026 10:25:16 +0000 Subject: [PATCH 05/12] fix: expand ToR to Top-of-rack Signed-off-by: Benjamin Ritter --- docs/05-Concepts/03-Network/02-firewalls.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/docs/05-Concepts/03-Network/02-firewalls.md b/docs/05-Concepts/03-Network/02-firewalls.md index 57f10cfa..e848b998 100644 --- a/docs/05-Concepts/03-Network/02-firewalls.md +++ b/docs/05-Concepts/03-Network/02-firewalls.md @@ -31,7 +31,7 @@ We see limited benefit in using firewall appliances for the following reasons: - Appliances and bandwidth would be shared between tenants, increasing management complexity and reducing tenant isolation -### Why not use the ToR Leaf switches? +### Why not use Top-of-rack Leaf switches as firewalls? SONiC can provide basic L3/L4 packet filtering capabilities using its ACLs, which can be offloaded to the switching silicon. But unfortunately that is not enough for our requirements: From 630d4d71ecab20a445ced3389281207e4ac62884 Mon Sep 17 00:00:00 2001 From: Benjamin Ritter Date: Thu, 11 Jun 2026 10:29:37 +0000 Subject: [PATCH 06/12] feat: improve external firewall explanation Signed-off-by: Benjamin Ritter --- docs/05-Concepts/03-Network/02-firewalls.md | 3 ++- 1 file changed, 2 insertions(+), 1 deletion(-) diff --git a/docs/05-Concepts/03-Network/02-firewalls.md b/docs/05-Concepts/03-Network/02-firewalls.md index e848b998..3e020545 100644 --- a/docs/05-Concepts/03-Network/02-firewalls.md +++ b/docs/05-Concepts/03-Network/02-firewalls.md @@ -27,7 +27,8 @@ Nevertheless we encourage you to use metal-stack firewalls for north-south traff We see limited benefit in using firewall appliances for the following reasons: -- Firewall appliances are a fragmented product category without common APIs or common behaviour, which would bind development time supporting different vendor specific implementations +- Firewall appliances are a fragmented product category without vendor-agnostic API or common behaviour +- As a result of the fragmented landscape supporting the most common firewalls would bind development time - Appliances and bandwidth would be shared between tenants, increasing management complexity and reducing tenant isolation From 6d78f0b510f069d625ca8fc54ecff34bf49e5401 Mon Sep 17 00:00:00 2001 From: Benjamin Ritter Date: Thu, 11 Jun 2026 10:44:06 +0000 Subject: [PATCH 07/12] fix: add TODOs for firewall configuration Signed-off-by: Benjamin Ritter --- docs/05-Concepts/03-Network/02-firewalls.md | 6 ++++++ 1 file changed, 6 insertions(+) diff --git a/docs/05-Concepts/03-Network/02-firewalls.md b/docs/05-Concepts/03-Network/02-firewalls.md index 3e020545..8414531c 100644 --- a/docs/05-Concepts/03-Network/02-firewalls.md +++ b/docs/05-Concepts/03-Network/02-firewalls.md @@ -46,6 +46,12 @@ In short, to offer comparable features to our current solution, we would need to ## Creating a Firewall +:::info +TODO: metalctl/metalcli example + +TODO: Configuring firewalls via firewall-controller CRDs +::: + ## Firewall Controller The firewall-controller allows you to configure Firewalls using CRDs from inside of Kubernetes clusters. Firewall configuration provided by the firewall-controller is hot reloaded. From f89c57b4b5c3d175e7c0e5bc6711bbf71d4c2be2 Mon Sep 17 00:00:00 2001 From: Benjamin Ritter Date: Thu, 11 Jun 2026 14:23:16 +0000 Subject: [PATCH 08/12] fix: introduction to why we use machines as firewalls Signed-off-by: Benjamin Ritter --- docs/05-Concepts/03-Network/02-firewalls.md | 2 ++ 1 file changed, 2 insertions(+) diff --git a/docs/05-Concepts/03-Network/02-firewalls.md b/docs/05-Concepts/03-Network/02-firewalls.md index 8414531c..a710add2 100644 --- a/docs/05-Concepts/03-Network/02-firewalls.md +++ b/docs/05-Concepts/03-Network/02-firewalls.md @@ -23,6 +23,8 @@ Nevertheless we encourage you to use metal-stack firewalls for north-south traff ## Why do we use Machines as Firewalls? +Using machines as firewalls gives us a lot of advantages. Controlling the full software stack running on the firewall gives us a lot of flexibility and enables quicker feature development. Our default [firewall image](https://github.com/metal-stack/metal-images/tree/master/firewall) uses nftables for L4 filtering and [Suricata](https://suricata.io/) for network analysis and threat detection. + ### Why not integrate existing firewall appliances? We see limited benefit in using firewall appliances for the following reasons: From 3e07da379af1deef207430fa5f1a0955eab046f5 Mon Sep 17 00:00:00 2001 From: Benjamin Ritter Date: Mon, 15 Jun 2026 08:41:24 +0000 Subject: [PATCH 09/12] fix: use Machine Size instead of SKU Signed-off-by: Benjamin Ritter --- docs/05-Concepts/03-Network/02-firewalls.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/docs/05-Concepts/03-Network/02-firewalls.md b/docs/05-Concepts/03-Network/02-firewalls.md index a710add2..891e4feb 100644 --- a/docs/05-Concepts/03-Network/02-firewalls.md +++ b/docs/05-Concepts/03-Network/02-firewalls.md @@ -11,7 +11,7 @@ Firewalls in metal-stack are machines that connect networks to each other or the Only firewalls can have multiple networks attached. -Firewalls do not require specialized hardware. For most scenarios any of your Machine SKUs can be used as a Firewall. +Firewalls do not require specialized hardware. For most scenarios any of your Machine Sizes can be used as a Firewall. :::important metal-stack does not support using existing firewall appliances for east-west traffic within metal-stack. From 7166135af19c17310c01f13340fd06a1bd4fb5d1 Mon Sep 17 00:00:00 2001 From: Benjamin Ritter Date: Mon, 15 Jun 2026 08:48:51 +0000 Subject: [PATCH 10/12] fix: add links for dangling TODOs Signed-off-by: Benjamin Ritter --- docs/04-For Operators/01-hardware.md | 2 +- docs/05-Concepts/03-Network/02-firewalls.md | 6 +++++- 2 files changed, 6 insertions(+), 2 deletions(-) diff --git a/docs/04-For Operators/01-hardware.md b/docs/04-For Operators/01-hardware.md index 13eaeac0..89b7f529 100644 --- a/docs/04-For Operators/01-hardware.md +++ b/docs/04-For Operators/01-hardware.md @@ -73,7 +73,7 @@ Other switch series and models might work but were not reported to us. On our switches we run [SONiC](https://sonicfoundation.dev). The metal-core writes network configuration specifically implemented for this operating system. Please also consider running SONiC on your switches if you do not want to run into any issues with networking. -Our previous support for [Cumulus Linux](https://www.nvidia.com/en-us/networking/ethernet-switching/cumulus-linux/) will come to an end. +[Cumulus Linux](https://www.nvidia.com/en-us/networking/ethernet-switching/cumulus-linux/) support in metal-stack is deprecated since Cumulus/Nvidia ended support for all of our currently supported switches. Of course, contributions for supporting other switch vendors and operating systems are highly appreciated. ::: diff --git a/docs/05-Concepts/03-Network/02-firewalls.md b/docs/05-Concepts/03-Network/02-firewalls.md index 891e4feb..c7bb1021 100644 --- a/docs/05-Concepts/03-Network/02-firewalls.md +++ b/docs/05-Concepts/03-Network/02-firewalls.md @@ -52,6 +52,8 @@ In short, to offer comparable features to our current solution, we would need to TODO: metalctl/metalcli example TODO: Configuring firewalls via firewall-controller CRDs + +https://github.com/metal-stack/website/issues/289 ::: ## Firewall Controller @@ -63,4 +65,6 @@ The firewall-controller allows you to configure Firewalls using CRDs from inside Firewall Machines are managed by metal-stack. The local state of Firewall machines is ephemeral, as the authoritative configuration is stored in metal-api. Manual changes to the configuration are not supported and will be overridden. Use metalctl or Firewall CRDs to apply changes to firewall configuration. -(fire-walling in metal-stack, firewall-controller and headscale integration) +## Headscale integration + +TODO: https://github.com/metal-stack/website/issues/290 From b59e0a4dc199da5473461fcc4da709b6fdaf5bd3 Mon Sep 17 00:00:00 2001 From: Benjamin Ritter Date: Mon, 15 Jun 2026 08:50:02 +0000 Subject: [PATCH 11/12] feat: undraft docs/05-Concepts/03-Network/02-firewalls.md Signed-off-by: Benjamin Ritter --- docs/05-Concepts/03-Network/02-firewalls.md | 1 - 1 file changed, 1 deletion(-) diff --git a/docs/05-Concepts/03-Network/02-firewalls.md b/docs/05-Concepts/03-Network/02-firewalls.md index c7bb1021..f581a359 100644 --- a/docs/05-Concepts/03-Network/02-firewalls.md +++ b/docs/05-Concepts/03-Network/02-firewalls.md @@ -2,7 +2,6 @@ slug: /firewalls title: Firewalls sidebar_position: 1 -draft: true --- # Firewalls From a7d4c03e3006b1288a06507076048110884f6fed Mon Sep 17 00:00:00 2001 From: Benjamin Ritter Date: Mon, 15 Jun 2026 13:15:33 +0000 Subject: [PATCH 12/12] fix: wording Signed-off-by: Benjamin Ritter --- docs/05-Concepts/03-Network/02-firewalls.md | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/docs/05-Concepts/03-Network/02-firewalls.md b/docs/05-Concepts/03-Network/02-firewalls.md index f581a359..778ada78 100644 --- a/docs/05-Concepts/03-Network/02-firewalls.md +++ b/docs/05-Concepts/03-Network/02-firewalls.md @@ -33,7 +33,7 @@ We see limited benefit in using firewall appliances for the following reasons: - Appliances and bandwidth would be shared between tenants, increasing management complexity and reducing tenant isolation -### Why not use Top-of-rack Leaf switches as firewalls? +### Why not use Top-of-Rack Leaf switches as firewalls? SONiC can provide basic L3/L4 packet filtering capabilities using its ACLs, which can be offloaded to the switching silicon. But unfortunately that is not enough for our requirements: @@ -64,6 +64,6 @@ The firewall-controller allows you to configure Firewalls using CRDs from inside Firewall Machines are managed by metal-stack. The local state of Firewall machines is ephemeral, as the authoritative configuration is stored in metal-api. Manual changes to the configuration are not supported and will be overridden. Use metalctl or Firewall CRDs to apply changes to firewall configuration. -## Headscale integration +## VPN integration TODO: https://github.com/metal-stack/website/issues/290