Maximum transmission unit and maximum segment size
Magic WAN has operation requirements that customers should know about to make sure their network works as intended. Customers should pay particular attention to the maximum transmission unit (MTU) and maximum segment size (MSS) values. The incorrect configuration of these values might lead to loss of performance or inability to deliver data packets.
The maximum transmission unit (or MTU) ↗ is a measurement representing the largest data packet that a network-connected device will accept. The MTU almost always applies to Layer 3 of the OSI model in networking, and includes the entire packet, including all headers (TCP, IP, etc.) and the data (payload) itself. For example, packets must be no bigger than 1,500 bytes to be routable through the Internet.
The maximum segment size (or MSS) ↗ refers to the amount of data that can be sent in a single TCP datagram packet. This value is determined by subtracting the size of the IP and TCP headers from the MTU, which will instruct the router how large the payload can be. It applies to Layer 4 of the OSI model in networking.
One common misconception about MSS/MTU is that setting these values negatively impacts performance. While there is a slight performance penalty, it is worse not to configure these values to account for the specificities of your network.
Since Magic WAN uses encapsulation to deliver its services, it is also important to understand why MTU and MSS matter in this case.
Encapsulation adds bytes to the packet, since we add a new IP header and (often) some sort of encapsulating header to every packet. For example, in the case of GRE for IPv4, we add 24 bytes - 20 bytes for the IPv4 header, and 4 bytes for the GRE tunnel header.
A network interface which performs GRE encapsulation needs to account for the added overhead by reducing its MTU. Since the MTU maximum size is 1,500 bytes, for IPv4 this means that the MTU can be 1,476 bytes (the original 1,500 bytes minus the 24 bytes from the GRE encapsulation). This reduced MTU defines the maximum size of the IP packet that can be encapsulated by GRE.
If the data packet is larger than what the network interface can accept, it either needs to be dropped or fragmented into smaller packets. When fragmentation occurs, Cloudflare only accepts data packets that can be completely reassembled. If some fragments are missing, all received fragments are discarded. Cloudflare does not forward incomplete packets to the customer.
Setting the do not fragment (DF) bit in the TCP header to 1 denotes that the packet must be dropped rather than fragmented if it is larger than the MTU that intermediary network devices can accept. Most TCP implementations set the do not fragment (DF) bit to 1 to avoid the potential issues caused by fragmentation.
If you are experiencing issues with fragmentation and are unable to set an MSS clamp, Cloudflare can clear the do not fragment (DF) bit for you. When this option is enabled, Cloudflare fragments packets greater than 1,500 bytes, and the packets are reassembled on your infrastructure after decapsulation. This should be a last resort option. Contact your account team for more information.
Fragmentation in Magic WAN
Consider a UDP datagram of size 3,000 bytes (8 bytes for the UDP header + 2,992 bytes for the UDP data). To fit within a standard 1,500 bytes MTU, this UDP datagram would be fragmented across three IP packets as follows:
Suppose that the UDP datagram has source port 389 and is destined for a Magic WAN customer IP address. Let us also suppose that the Magic WAN customer has a firewall rule in place that drops UDP traffic with source port 389, a common Connectionless Lightweight Directory Access Protocol (CLDAP) ↗ reflection attack vector. The three packet fragments above will arrive at Cloudflare, but only the first fragment contains a UDP header with source port information - the second and third fragments, while they contain UDP data, do not have UDP header information. So the question is: which of these fragments does Cloudflare drop and which are delivered to the customer? If we only drop the first parts of fragmented packets, the remaining parts could still generate a large amount of traffic during a DoS attack.
The diagram below shows how the three UDP fragments in the example above flow through Cloudflare and Magic WAN. However, the main takeaways are:
- Cloudflare will never send incomplete packets to a customer: If we do not see all parts of a packet required to fully reassemble that packet, we will not send the partial data fragments on to the customer.
- Magic Firewall operates on fully reassembled packets, not individual fragments: This means that filters that match on UDP/TCP header information, for example, will apply to the fully reassembled packet, not just the initial fragment. So, we will not leak non-initial fragments to customers.
- Customers can still see fragmented packets: By default (without
clear_dont_fragment_bitset), we fragment packets to fit within the configured MTU of the tunnel before sending the data to the customer. If a packet is larger than 1,476 bytes, we will fragment it and send those fragments to the customer for reassembly.
In all cases, Cloudflare sends all fragments to the customer.
Maximum segment size (MSS) is a TCP setting that limits the size of TCP segments. This option is set in the SYN packets during the three-way handshake.
By default, a TCP endpoint will set its MSS value based on its local network interface MTU. For example, for IPv4, if the MTU is 1,500 bytes then MSS will be 1,460 bytes (1,500 bytes minus 20 bytes from the IPv4 header minus 20 bytes from the TCP header).
MSS is a tool that can be used to configure TCP packet size behavior. If a TCP endpoint is known to be behind a network with reduced MTU, changing the MSS value to match the actual path MTU value will force remote endpoints to send packets that fit within the specified MTU. So, if an IPv4 TCP endpoint is known to be behind a GRE tunnel with an MTU of 1,476 bytes, the MSS value in its TCP SYN packets should be 1,436 bytes - 1,476 bytes minus the 20 bytes from the IPv4 header, minus the 20 bytes from the TCP header.
One way to modify the MSS setting is by changing the MTU of the network interface in the router's WAN interface to match the path MTU. Another way to modify MSS is by applying a MSS clamp, where an intermediary network device - such as a router - is configured to modify the MSS TCP option on-the-fly when packets pass through it. Note that changing the MTU on the interface of an intermediary network device is not the same as applying an MSS clamp, and it will not cause the TCP MSS value to be changed.
Refer to MSS clamping recommendations for information on what you should set your MSS clamping to, depending on the type of tunnel.
The MSS value depends on how your network is set up.
- On your Edge router: Apply the clamp to the GRE tunnel internal interface (meaning where the egress traffic will traverse). The MSS clamp should be 1,436 bytes. This may be done automatically once the tunnel is configured, but it depends on your devices.
For IPsec tunnels, the value you need to specify depends on how your network is set up. The MSS clamping value will be lower than for GRE tunnels, however, since the physical interface will see IPsec-encrypted packets, not TCP packets, and MSS clamping will not apply to those.
- On your Edge router: Apply this on your Magic WAN IPsec tunnel internal interface (meaning where the Magic WAN egress traffic will traverse). This may be done automatically once the tunnel is configured but it depends on your devices. TCP MSS clamp should be 1,360 bytes maximum.