UDP vs. TCP

Introduction

Hi, I’m Glenn Fiedler and welcome to the first article in my online book Networking for Game Programmers

In this article we start with the most basic aspect of network programming, sending and receiving data over the network. This is just the beginning – the simplest and most basic part of what network programmers do, but still it is quite intricate and non-obvious as to what the best course of action is. Take care because if you get this part wrong it will have terrible effects on your multiplayer game!

You have most likely heard of sockets, and are probably aware that there are two main types: TCP and UDP. When writing a network game, we first need to choose what type of socket to use. Do we use TCP sockets, UDP sockets or a mixture of both?

The choice you make depends entirely on what sort of game you want to network. So from this point on, and for the rest of this article series, I’m going to assume you want to network an action game. You know games like Halo, Battlefield 1942, Quake, Unreal, CounterStrke, Team Fortress and so on.

In light of the fact that we want to network an action game, we’ll take a very close look at the properties of each type of socket, and dig a bit into how the internet actually works. Once we have all this information, the correct choice becomes very clear!

TCP/IP

TCP stands for “transmission control protocol” and IP stands for “internet protocol”. Together they form the backbone for almost everything you do online, from web browsing to IRC to email, its all built on top of TCP/IP.

If you have ever used a TCP socket, then you’ll know that it is a reliable connection based protocol. This simply means that you make a connection between two machines, then you send data between the two computers much like you are writing to a file on one side, and reading from a file on the other.

This connection is reliable and ordered, meaning that all data you send is guaranteed to arrive at the other side in the same order that you wrote it. Its also a stream of data, meaning that TCP takes care of splitting up your data into packets and sending those across the network for you.

Again, remember its just like writing to a file. So simple!

IP

The simplicity is in stark contrast to what actually goes on at the lower level “IP” protocol underneath TCP.

Here there is no concept of connection, instead packets are passed from one computer to the next. You can visualize this process like a hand-written note being passed from one person to the next across a crowded room, eventually reaching the person it is addressed to, but only after passing through many hands.

There is no guarantee that this note will actually reach the person it is addressed to. The sender just passes the note along and hopes for the best, never knowing whether or not the note was received, unless the other person decides to write back!

Of course, it is in reality a little bit more complicated than this, since of course no one computer knows the exact sequence of computers to pass the packet along to so that it reaches its destination quickly. Sometimes “IP” passes along multiple copies of the same packet, these packets make their way to the destination via different paths, so will most likely arrive at different times.

This is because the internet is designed to be self-organizing and self-repairing, able to route around connectivity problems. It’s actually quite cool if you think about what it is really going on at the low level. You can read all about this in the classic book TCP/IP Illustrated.

UDP

Instead of treating communications between computers like writing to files, what if we want to send and receive packets directly?

We can do this using UDP. UDP stands for “user datagram protocol” and it is another protocol built on top of IP, just like TCP, but this time instead of adding lots of features and complexity it is just a very thin layer over IP.

With UDP we can send a packet to a destination IP address (eg. 112.140.20.10) and port (say 52423), and it will get passed from computer to computer until it arrives at the destination computer or is lost along the way.

On the receiver side, we just sit there listening on a specific port (eg. 52423) and when a packet arrives from any computer (remember there are no connections!), we get notified of the address and port of the computer that sent the packet, the size of the packet, and can read the packet data.

UDP is an unreliable protocol. In practice, most packets that are sent will get through, but you’ll usually have around 1-5% packet loss, and occasionally you’ll get periods where no packets get through at all (remember that there are lots of computers between you and your destination where things can go wrong…)

There is also no guarantee of ordering of packets. You could send 5 packets in order 1,2,3,4,5 and they could arrive completely out of order like 3,1,2,5,4. In practice, they will actually arrive in order almost all of the time, but again, you cannot rely on this!

Finally, although UDP doesn’t do much on top of IP, it does make one guarantee for you. If you send a packet, it will either arrive in whole at the destination, or not arrive at all. So if you send a 256 byte packet to another computer, that computer cannot receive just the first 100 bytes of the packet, it must get the full 256 bytes of data. This is pretty much the only guarantee you get with UDP, everything else is up to you!

TCP vs. UDP

We have a decision to make here, do we use TCP sockets or UDP sockets?

Lets look at the properties of each:

TCP:

• Connection based
• Guaranteed reliable and ordered
• Automatically breaks up your data into packets for you
• Makes sure it doesn’t send data too fast for the internet connection to handle (flow control)
• Easy to use, you just read and write data like its a file

UDP:

• No concept of connection, you have to code this yourself
• No guarantee of reliability or ordering of packets, they may arrive out of order, be duplicated, or not arrive at all!
• You have to manually break your data up into packets and send them
• You have to make sure you don’t send data too fast for your internet connection to handle
• If a packet is lost, you need to devise some way to detect this, and resend that data if necessary

The decision seems pretty clear then, TCP does everything we want and its super easy to use, while UDP is a huge pain in the ass and we have to code everything ourselves from scratch. So obviously we just use TCP right?

Wrong.

Using TCP is the worst possible mistake you can make when developing a networked game! To understand why, you need to see what TCP is actually doing above IP to make everything look so simple!

How TCP really works

TCP and UDP are both built on top of IP, but they are radically different. UDP behaves very much like the IP protocol underneath it, while TCP abstracts everything so it looks like you are reading and writing to a file, hiding all complexities of packets and unreliability from you.

So how does it do this?

Firstly, TCP is a stream protocol, so you just write bytes to a stream, and TCP makes sure that they get across to the other side. Since IP is built on packets, and TCP is built on top of IP, TCP must therefore break your stream of data up into packets. So, some internal TCP code queues up the data you send, then when enough data is pending the queue, it sends a packet to the other machine.

This can be a problem for multiplayer games if you are sending very small packets. What can happen here is that TCP may decide that its not going to send data until you have buffered up enough data to make a reasonably sized packet (say more than 100 bytes or so). This is a problem because you want your client player input to get to the server as quickly as possible, if it is delayed or “clumped up” like TCP can do with small packets, the client’s user experience of the multiplayer game will be very poor. Game network updates will arrive late and infrequently, instead of on-time and frequently like we want.

TCP has an option you can set that fixes this behavior called “TCP_NODELAY”. This option instructs TCP not to wait around until enough data is queued up, but to send whatever data you write to it immediately.

Unfortunately, even if you set this option TCP still has serious problems for multiplayer games.

It all stems from how TCP handles lost and out of order packets, to present you with the “illusion” of a reliable, ordered stream of data.

How TCP implements reliability

Fundamentally TCP breaks down a stream of data into packets, sends these packets over unreliable IP, then takes the packets received on the other side and reconstructs the stream.

But what happens when a packet is lost? What happens when packets arrive out of order or are duplicated?

Without going too much into the details of how TCP works because its super-complicated (please refer to TCP/IP Illustrated) in essence TCP sends out a packet, detects when that packet was lost, then resends that lost packet to the other machine. Duplicate packets are discarded on the receiver side, and out of order packets are resequenced so everything is reliable and in order.

The problem is that if we were to attempt to synchronize this using TCP, whenever a packet is dropped it has to stop and wait for that data to be resent. Yes, even if more recent data arrives, that new data gets put in a queue, and you cannot access it until you receive the lost packet. How long does it take to resend the packet? Well, it is going to take at least round trip latency for TCP to work out that data needs to be resent, and another one way trip from the sender to the receiver for the resent packet to get there. So if you have a 125ms ping, you will be waiting roughly 1/5th of a second for the packet data to be resent at best, and in worst case conditions you could be waiting up to half a second or more (consider what happens if the attempt to resend the packet fails to get through?) Fun times!

Why you should never use TCP to network a multiplayer game

The problem with using TCP for games is that unlike web browsers, or email or most other applications, multiplayer games have a real time requirement on packet delivery. For many parts of your game, for example player input and character positions, it really doesn’t matter what happened a second ago, you only care about the most recent data.

Consider a very simple example of a multiplayer game, some sort of action game like a shooter. You want to network this in a very simple way. Every frame you send the input from the client to the server (eg. keypresses, mouse input controller input), and each frame the server processes the input from each player, updates the simulation, then sends the current position of game objects back to the client for rendering.

So in our simple multiplayer game, whenever a packet is lost, everything has to stop and wait for that packet to be resent. On the client game objects stop receiving updates so they appear to be standing still, and on the server input stops getting through from the client, so the players cannot move or shoot. When the resent packet finally arrives, you receive this stale, out of date information that you don’t even care about! Plus, there are packets backed up in queue waiting for the resend which arrive at same time, so you have to process all of these packets in one frame. Everything is clumped up!

Unfortunately, there is nothing you can do to fix this behavior with TCP, nor would you want to, it is just the fundamental nature of it! This is just what it takes to make the unreliable, packet-based internet look like a reliable-ordered stream.

We don’t want a reliable ordered stream.

We want our data to get as quickly as possible from client to server without ever having to wait for lost data to be resent.

This is why you never use TCP for networking a multiplayer game.

Wait? Why can’t I use both UDP and TCP?

For realtime game data like player input and state, only the most recent data is relevant, but for other types of data, say perhaps a sequence of commands sent from one machine to another, reliability and ordering can be very important.

The temptation then is to use UDP for player input and state, and TCP for the reliable ordered data. If you’re sharp you’ve probably even worked out that you may have multiple “streams” of reliable ordered commands, maybe one about level loading, and another about AI. Perhaps you think to yourself, “Well, I’d really not want AI commands to stall out if a packet is lost containing a level loading command – they are completely unrelated!”. You are right, so you may be tempted to create one TCP socket for each stream of commands.

On the surface, this seems like a great idea. The problem is that since TCP and UDP are both built on top of IP, the underlying packets sent by each protocol will affect each other. Exactly how they affect each other is quite complicated and relates to how TCP performs reliability and flow control, but fundamentally you should remember that TCP tends to induce packet loss in UDP packets. For more information, read this paper on the subject.

Conclusion

My recommendation then is not only that you use UDP, but that you only use UDP. Don’t mix TCP and UDP, instead learn how to implement the specific pieces of TCP that you wish to use inside your own custom UDP based protocol.

The rest of the articles in this series show you how to do this, from creating your own virtual connection based protocol on top of UDP, to creating your own reliability and flow control.

But next, something much more concrete: how to send and receive packets using UDP

Enjoy!