OpenShift Networking Deep Dive: OVN-Kubernetes Architecture and Traffic Flow

When a customer clicks Buy in an online store, their request passes through a load balancer, an OpenShift router, a Service and an application Pod. The application then contacts an external payment API.

That single request crosses several network layers. What follows is a practical way to understand what OVN-Kubernetes does at each of them, and where to look when something goes wrong.

This article describes a typical OpenShift setup with OVN-Kubernetes. Gateway placement, NAT and load balancer behaviour differ by OpenShift version, cloud provider and cluster configuration, so check the documentation for your release before you change anything.

Two jobs: plan the network and move packets

OVN-Kubernetes is easier to understand if you treat its control plane and data plane as separate parts. The control plane defines the network. The data plane applies that design and forwards traffic.

Kubernetes tracks objects such as Pods, Services, EndpointSlices and NetworkPolicies. OVN-Kubernetes watches these objects and builds a logical network from them.

The OVN Northbound Database holds the intended design: logical switches, routers, ports and policies. The ovn-northd process translates that design into logical flows. The Southbound Database distributes those flows and related bindings to each node. On every node, ovn-controller converts them into local OpenFlow rules for Open vSwitch (OVS).

OVN plans the network path. OVS forwards the packets.

More precisely: OVN sets the logical behaviour, ovn-controller programs the local OVS state, and OVS forwards packets.

The networks under OpenShift

The request may touch several networks:

• Machine network. Connects the cluster nodes.
• Pod network. Gives Pods their IP addresses.
• Service network. Provides virtual IP addresses for Services.
• Host network. Carries traffic to or from a node itself. A Pod using host networking shares the node's network namespace.

Decide these address ranges before you install the cluster. They must not overlap with each other, or with any network the cluster needs to reach: corporate, cloud, VPN or partner networks.

The physical network that connects nodes is the underlay. The virtual network for Pod traffic is the overlay. In most overlay setups, packets between nodes travel through the underlay inside a Geneve tunnel.

How OVN turns Kubernetes objects into network rules

OVN uses familiar networking ideas:

• A logical switch connects logical ports.
• A logical router connects networks and provides routing.
• A logical port represents a connection point, such as a Pod or a node.
• A logical flow is a rule describing how traffic should be handled.
• A chassis is a node that participates in the OVN network.

The translation works like this:

1Kubernetes object
2      ↓
3OVN logical object
4      ↓
5Logical flow
6      ↓
7Local OVS rule
8      ↓
9Packet action

When Kubernetes creates a Pod, OVN-Kubernetes adds it to the logical network. When you create a NetworkPolicy, OVN-Kubernetes adds rules that allow or block matching traffic.

This separation is why checking the Kubernetes object is not always enough when troubleshooting. The object can be correct while a node still fails to apply or use its network rules.

How a Pod gets connected

When OpenShift starts a Pod, it creates a network namespace. That gives the Pod its own network interface, IP address and routes.

The CNI path then connects the Pod to the OVN-Kubernetes network:

1. OVN-Kubernetes assigns the Pod an IP address.
2. A virtual Ethernet pair is created. It acts like a cable between the Pod and the node.
3. One end is placed inside the Pod. The other connects to OVS.
4. OVS connects the Pod to the logical network.
5. The Pod receives its address and default route.

The Pod sends ordinary IP packets. It does not need to know whether the destination is on the same node or somewhere else in the cluster.

East-west traffic: Pod to Pod

Same node

If two Pods are on the same node, OVS forwards the traffic locally:

1Pod A → veth → OVS → veth → Pod B

The source Pod sends a packet to the destination Pod's IP. OVS checks its rules and delivers the packet directly. The packet never touches the node network.

Different nodes

If the destination Pod is on another node, the source node sends the packet through a Geneve tunnel:

1Pod A → OVS on worker-a
2      → Geneve tunnel over the node network
3      → OVS on worker-b → Pod B

Geneve wraps the original Pod packet inside a new packet so it can cross the node network.

1Inner packet: the original Pod traffic
2Source:       Pod A IP
3Destination:  Pod B IP
4
5Outer packet: carries it between nodes
6Source:       worker-a IP
7Destination:  worker-b IP
8UDP:          6081

The destination node removes the outer packet and delivers the original to Pod B. The inner packet still carries the Pod IPs. The outer packet uses node IPs. OpenShift's firewall documentation lists UDP port 6081 for Geneve, so make sure the node firewalls allow it.

Why Geneve makes MTU important

Geneve adds outer IP, UDP and tunnel headers around the original packet. Those headers take space, which leaves less room for the Pod's packet than the physical MTU suggests.

If the effective MTU is too large for the path, packets fragment or drop. The symptoms are confusing: small requests work but larger transfers stall, or same-node traffic works while cross-node traffic fails. When you see that pattern, check MTU along the whole path, including node interfaces and the physical network.

Service traffic: a stable address for changing Pods

Applications should call a Service rather than depend on a specific Pod IP. A Service provides a stable virtual IP and port. Its selector identifies the Pods that can serve requests, and Kubernetes records the ready backends in EndpointSlices.

1Frontend Pod → Service IP and port → selected Pod IP and port

The Service IP is the address the client uses. The selected Pod IP is the backend that receives the traffic. The Service IP is virtual and is not assigned to a Pod. OVN-Kubernetes sets up the forwarding that sends Service traffic to a ready backend.

If the selected Pod is on the same node, traffic is forwarded locally. If it is on another node, the packet takes the cross-node path. When a backend becomes unhealthy or disappears, Kubernetes updates the EndpointSlices so new traffic goes to the remaining ready endpoints.

If a Pod IP works but the Service does not, check the Service's selector and port, then confirm that the EndpointSlices list ready backends.

For Services exposed outside the cluster, externalTrafficPolicy affects which nodes receive traffic and whether the original client IP is preserved. internalTrafficPolicy affects which endpoints are used for internal traffic. Their effect depends on the Service type and the cluster configuration.

NetworkPolicy: is the packet allowed?

A packet can have a valid route and still be blocked. NetworkPolicy controls which connections selected Pods can receive or make. OVN-Kubernetes turns policy rules into rules enforced in the datapath.

A common pattern is to deny traffic by default, then allow only the connections an application needs. The catalog service, for example, accepts requests from the frontend and denies other Pod traffic.

Check the policy's Pod and namespace selectors, ports and protocols. Check DNS too. If a policy blocks DNS, a Pod can reach an IP address but fail to connect by service name.

North-south traffic: user to application

The customer's request travels through the cluster like this:

1Client
2  → Cloud load balancer
3  → Ingress endpoint
4  → OpenShift router
5  → Service
6  → Application Pod

The load balancer forwards the connection to an ingress endpoint. The OpenShift router reads the Route or Ingress settings and sends the request to the target Service. The Service selects a ready application Pod.

OpenShift Routes use one of three TLS modes:

• Edge. TLS ends at the router. Traffic from the router to the Pod may be unencrypted.
• Passthrough. The router forwards encrypted traffic and TLS ends at the application.
• Re-encrypt. TLS ends at the router, then a new TLS connection is made to the Pod.

If the router returns a 503, check that the Route points to the right Service and that the Service has ready endpoints.

Egress: Pod to an external API

After processing the order, the application calls the payment API:

1Application Pod → OVS → Cluster gateway → External network → Payment API

The Pod sends traffic to the external destination. OVN-Kubernetes and the cluster gateway route it out. Depending on the setup, the Pod's source address is changed to the node's address or to another egress path.

With default egress, an external service sees a node address after source NAT. With a configured Egress IP, selected Pod traffic uses a predictable source address:

1Selected Pod → Egress IP → External firewall → Partner API

That matters when a partner firewall only accepts requests from approved addresses. An egress firewall can also limit which destinations particular workloads may reach. Check what your OpenShift version and environment support before you write external firewall rules around it.

Find where the packet stops

Start at the source and trace the connection toward the destination. Check one layer at a time instead of inspecting components at random.

1oc get pods -o wide -n shop
2oc get svc,endpointslices -n shop
3oc get routes,ingress -n shop
4oc get networkpolicy -n shop
5oc get co network

These show where Pods run, whether a Service has ready endpoints, whether a Route exists, and whether the network operator reports a problem.

Test a Service from a Pod:

1oc exec -n shop deploy/frontend -- \
2  curl -v http://catalog:8080/health

Check a Pod's address and routes:

1oc exec -n shop <pod-name> -- ip addr
2oc exec -n shop <pod-name> -- ip route

Then ask, in order:

1. Can the Pod resolve the destination name?
2. Can it reach the destination IP?
3. Can it reach the Service IP?
4. Does the Service have ready endpoints?
5. Does NetworkPolicy allow the connection?
6. Does the failure happen only across nodes?
7. Can the nodes exchange Geneve traffic?
8. Could MTU be failing larger packets?
9. Is the router receiving the request?
10. Is an external firewall blocking traffic?

For deeper checks, use the supported OpenShift procedures to inspect OVN, OVS and packet captures. tcpdump on a node shows whether traffic reaches a particular interface. Node-level commands and component layouts change between releases, so do not modify OVN or OVS state by hand.

Common failure patterns

• Pod IP works, Service fails. Check the Service port, selector and EndpointSlices.
• Same node works, cross-node fails. Check node connectivity, Geneve, firewall rules and MTU.
• Service works, Route returns 503. Check the router, the Route target and ready endpoints.
• DNS works on nodes but not in Pods. Check Pod DNS and NetworkPolicy rules.
• Only one namespace fails. Check that namespace's policies and label selectors.
• External service sees the wrong source IP. Check the gateway, NAT and Egress IP setup.

Production checklist

Before deploying applications:

• Choose IP ranges that do not overlap with connected networks.
• Confirm that nodes can exchange the required traffic, including UDP 6081.
• Check MTU across the full path, including tunnels.
• Decide how outside traffic enters and how Pod traffic leaves.
• Plan partner firewall rules around the source addresses they will see.
• Use NetworkPolicy to allow only the connections each workload needs.
• Monitor network operator health, DNS errors, packet drops, latency and router errors.
• Add extra networks only when you clearly need them. Each one adds setup and troubleshooting work.

Conclusion: trace the packet

Do not fix OpenShift networking by guessing what is broken. Trace the packet.

Follow the path in order:

1DNS → Pod IP → Service → Endpoint → NetworkPolicy
2    → OVS → Geneve → Router → Gateway → External firewall

Kubernetes objects describe what should happen. OVN turns them into a logical network. ovn-controller programs each node's OVS state, and OVS forwards the packets. NetworkPolicy allows or blocks traffic along the way.

When a connection fails, work out where it is going and check each step: names, addresses, endpoints, policies, nodes, tunnels and external firewalls. That turns a vague "the network is broken" into a specific problem you can investigate.

Sources

• OVN-Kubernetes network plugin, OpenShift Container Platform 4.16
• Configuring the firewall, OpenShift Container Platform 4.16
• Network security, OpenShift Container Platform 4.16
• Multiple networks, OpenShift Container Platform 4.16