In the world of Industrial IoT and factory automation, we often encounter challenges due to physical and logical network separations. Machine builders deliver standardized systems that all use the same subnet internally. The problem becomes apparent when such a machine needs to be connected to a central Kubernetes cluster.
Traditionally, this problem is solved by installing a physical NAT router next to each machine. This is expensive, prone to errors, and difficult to scale. Furthermore, it is complicated to properly manage such IP address overlap.
In this blog post, I'll show you how we use Cilium and eBPF to make a Kubernetes node function as a software-defined OT gateway.
The Case Study: Bakery Automation Solutions
Imagine a supplier of automated industrial bakeries. Each production line has a Siemens PLC, sensors, and displays with static IP addresses in the 192.168.2.0/24 subnet. The customer wants to control all production lines across different locations from a central Kubernetes cluster (running Node-RED).
Normally, there is a reliance on expensive physical routers. Here, there is a Thin Edge node next to each production line. This is an industrial PC equipped with Talos Linux, Kubernetes, Cilium, and two network interface cards (Dual-NIC).
The Surroundings
Our test setup consists of a Kubernetes cluster with 4 Talos nodes:
- Nodes 1–3: Control plane nodes.
- Amsterdam-1 (Thin Edge node):
- ens3: Connected to the primary IT network (the cluster network).
- ens4: Directly connected to the PLC in the isolated OT network (192.168.2.x).
- Cilium (eBPF and Egress Gateway are enabled):
View here
➜ kubectl get nodes -L locatie
NAME STATUS ROLES AGE VERSION LOCATIE
node-1 Ready control-plane 3h59m v1.35.0
node-2 Ready control-plane 3h59m v1.35.0
node-3 Ready control-plane 3h59m v1.35.0
amsterdam-1 Ready gateway 3h59m v1.35.0 amsterdam-1
Copy code
➜ talosctl -n amsterdam-1 get address ens4/192.168.2.3/24
NODE NAMESPACE TYPE ID LINK
amsterdam-1 network AddressStatus ens4/192.168.2.3/24 ens4
Copy code
The solution: a software-defined gateway
With Cilium, we use a feature to route traffic without the PLC realizing that it is communicating with an external cluster.
Cilium Egress Gateway
With a CiliumEgressGatewayPolicy , we force traffic from our central application (the Node-RED pod) to leave the cluster via the specific interface of the Thin Edge node (amsterdam-1).
Cilium uses eBPF to dynamically apply Source NAT (SNAT). The PLC sees an incoming request from 192.168.2.3. The PLC processes the client’s request as if the traffic originated from the local network.
apiVersion: cilium.io/v2
kind: CiliumEgressGatewayPolicy
metadata:
name: traffic-to-192-168-2-0
spec:
selectors:
- podSelector:
matchLabels:
io.kubernetes.pod.namespace: amsterdam-1
destinationCIDRs:
- "192.168.2.0/0"
egressGateway:
nodeSelector:
matchLabels:
node-role.kubernetes.io/gateway: "gateway"
locatie: "amsterdam-1"
interface: ens4
Copy code
Proof of concept: Does it really work?
To validate the setup, we'll run a test pod on node-1. This node is not physically connected to the factory floor.
Step 1: Check the routing
We use the Cilium CLI to check whether the policy is active on the Thin Edge node:
➜ kubectl -n kube-system exec cilium-q6vrs -- cilium bpf egress list
Source IP Destination CIDR Egress IP Gateway IP
10.244.1.167 0.0.0.0/0 192.168.2.3 192.168.4.20
Copy code
Traffic from our pod (10.244.1.167) is being routed properly through the Edge node's IP address.
Step 2: The Connection Test
We are running a curl command from the pod at node-1 to the PLC's IP address:
➜ kubectl -n edge-namespace get pods -o wide
NAME READY STATUS RESTARTS AGE IP NODE
curl 1/1 Running 0 71m 10.244.1.167 node-1
➜ kubectl -n amsterdam-1 exec curl -- curl -s http://192.168.2.2
200 OK
Copy code
The answer 200 OK indicates that the connection was established successfully. Although the pod runs on a central server in the data center, it reaches the PLC on the factory floor via the Cilium Egress Gateway.
Why This Is Future-Proof
Moving routing from physical hardware to a software-defined layer provided by Cilium offers the following advantages:
- Costs: No need for expensive industrial routers per machine or production line.
- Management: All network configurations are now defined in code and deployed using GitOps. Setting up an environment for a new machine is as simple as adding an extra YAML file.
- Security: Traffic can be filtered and monitored using Cilium network policies, even for traffic leaving the cluster.
This eliminates the need for manual NAT configurations and managed switches on-site. With Cilium, we make the factory floor a true part of the software-defined network.
To learn how SUE sets up and manages Kubernetes clusters, visit the Kubernetes page. The Kubernetes guide covers everything from setting up your first cluster to production.
