Networking for Game Programmers
Hi, I’m Glenn Fiedler, I’m a professional game developer working in Los Angeles.
In this series of articles, I share some of the basic game networking techniques with you. Until now, a lot of this information has been hard to find, now it’s all in one place and written specifically covering techniques for multiplayer action games.
Maybe you are a hobbyist programmer looking to break-in to game programming, or just somebody interested in learning how multiplayer games work. You may even be a professional game programmer searching for hard to find information about how to network your game.
I hope you find what you are looking for here.
If you find these articles useful and you would like to read more, please donate! Donations encourage me to write more (each article usually takes all my spare time for at least 2 weeks…), and I’m very busy these days working for Sony Santa Monica, so I don’t get a lot of free time!
Again, let me reiterate that, if you would like me to finish writing this series please donate. If I get enough donations, I’ll publish a new article, it’s as simple as that!
Thanks!
Glenn Fiedler
gaffer@gaffer.org
What is the best way to send data between machines? Do you use TCP sockets, UDP sockets or a mixture of both? In this article we explore this fundamental choice, then decide on the best option for networking games.
This article shows how to send and receive UDP packets using BSD sockets. We then wrap this code into a platform independent socket class you can use across MacOS X, Windows and Unix.
In this article I show you how to build your own concept of virtual connection between two machines on top of UDP. We cover the basics of what it means to be connected, how to detect connection, how to filter out packets from other machines, and how to detect disconnection with timeouts.
We want to use UDP to network our game, but UDP does not provide any reliability or flow control. This article covers the basic concepts you need to implement your own reliability system: sequence numbers and acks. We then implement a simple flow control system on top of our reliability system to avoid flooding the connection with too much data.
Multiplayer games are much harder to debug than singleplayer ones. This article describes they key techniques professional game developers use when debugging multiplayer games.
What Every Programmer Needs To Know About Game Networking
An overview of common multiplayer programming techniques and the history of their development. Discover how RTS games were able to synchronize thousands of units over a 28k modem, and how first person shooters hide lag with client side prediction and latency compensation.
Building a Node Mesh
In this article we discuss client/server topology vs. peer-to-peer, then design a simple node mesh structure that may be used to represent either topology. The concept of delta encoding is also introduced, then applied as a bandwidth optimization for mesh updates.
Basic LAN Matchmaking
Even wondered how classic games like Doom or Quake were able to automatically “see” servers running on your LAN? The answer is UDP broadcast packets, a way to send packets to all computers on the LAN-subnet. In this article I show you how to use broadcast packets to create your own basic LAN matchmaking system.
Real World Networking
Welcome to the real world, where routers and real world difficulties like NAT punch-through get in the way of you and the computers you want to talk to. Instead of writing our own NAT punch-through, we solve this problem by creating an abstract packet transport interface, then create implementations of this transport layer using OpenTNL, RakNet and eNet, as well as our own LAN matchmaking and BSD sockets based comms.
Reading and Writing Packets
In this article I show you how to efficiently read and write packets with a bit-packer. I also show you how to implement checkpoints and journaling to detect when you desync packet read and write, as well as detect invalid packets. Finally, I show you a cool trick that unifies read and write into a single serialize operation.
Synchronizing State
How do you synchronize state between two machines? I show you the basics in this article covering object prioritization, object views, and ack aware state replication. Using this system you can easily synchronize the positions of players, and state of game objects in your world. I also introduce the concept of “soft delta-encoding” and show you how to packet the most important objects in your packet, while maintaining a constant packet size.
Network Events
Not everything can be represented as state, even though it would be nice. Sometimes you have to send events. In this article I show you how to implement a reliable event system based on acks from our packet level reliability system. I’ll show you how to implement multiple streams of events, with different reliability guarantees: unreliable, reliable, reliable-ordered, most-recent and important. The last one the is the key for absolute minimum latency events!
Great series already !
Much Information I never found anywhere else.
(ie: tcp influence on udp when using both)
I’m sure next series will answer much other question I have (netgame loop, server + client on same host, network queue management with priorities, threading, compression benefits or not, …)
Here’s VS2005 working solution/proj and all (had to fix some compiler quirks, macro missing parenthesis, variable array allocation on stack, int to bool didn’t work in initialize connect, all working now.)
http://tuan.kuranes.free.fr/sendrecpackets.zip
http://tuan.kuranes.free.fr/VirtualConnectionUdp.zip
You could setup a code.google.com repository (svn +° wiki) with all you code so that patches and all becomes easier… meaning old articles code would also benefits from patches/updates.
Again, thanks for all the work !
hey tuan, thanks for your help – there is a google code repository up here: http://code.google.com/p/netgame/
if you would like direct access then we can work together on it, just send an email to gaffer@gaffer.org and i’ll hook you up
thanks
Good job very interesting (Book marked) keep up the good work Glenn!
I really like your articles, i cant wait to read the new ones!.
A very interesting set of articles: the things you handle are difficult, but you managed to grab these things and make them readable. The new article is really good and I recommend everyone, who wants to know something about networking, reading that one. It’s easy to grab, but a powerfull concept.
Glenn, astonishing!
HEY GLENN GET TO WORK GOD DARNIT! I’ve been waiting for the next articles for a long time.
sorry, i just started work at sony santa monica, so i’ve been a bit busy the last week
expect more articles around christmas, of course if sony approves them
cheers
Hey thanks for letting us know. I have a question, what would you recommend to someone wanting to get up to speed in networking for games ?. I mean there’s enough info about the basics of networking but some of the info that you show here (at least have listed) is not easy to find in the web and you must have gathered all this from somewhere.
Btw do you know how to build networking systems for massive multiplayer games ? I’m not talking about super complex tech, but good enough to have, say 500 people in a single server. If you do then I’m sure millions of ppl would like to know how everything works and I would hate to have to stalk you day and night until you write such articles.
Seriously if you ever finish all the articles you listed here, a few extra on network architecture for mmos wouldn’t hurt.
Or write a book.
ok well most of what i found i had to work out myself, but there are some good resources out there, check out the networking forum FAQ on gamedev.net thats a pretty good collection of articles
cheers
Good articles! Though I’m really familiar with basic networking concepts, your articles are very well written and informative.
I am very much looking forward to your more indepth articles, especially on a node mesh and network synch.
~ Scott
Thank you for these articles. I have to fiddle with networking basics right now and these articles where the most helpful i’ve found. I hope to see more in near future.
Do you have any information on the hardware side of things? Such as good server hosting plans, installing your server app with that hosting, etc…
Heya, I have recently taken a tumble down the nasty physics networking rabbit-hole. Your articles have been a great help with making sense of it all
I was wondering what your thoughts were on
. networking ragdoll physics (assuming [at a minumum] ragdoll root positions are important for gameplay)
. networking physics joints (in general)
Thanks for all the great articles
HJ
thanks, i just wish i had more time to finish the series! maybe after GDC
cheers
Glenn,
Thank you for your work on tutorials… I have looked at the code and first thing that got into my eye was that you are linking with wsock32.lib. If you look at MS documentation and chking the following link:
http://blogs.msdn.com/wndp/archive/2006/12/18/winsock-header-weirdness.aspx
you should notice that you should be using ws2_32.lib if you are including header winsock2.h to make sure nothing bad happens.
thanks i’ll update the code to match, cheers
Great series of articles… hope to see more soon
posted a link on my blog
Is “Real World Networking” gonna happen anytime soon?
The stuff that I have read is really great! But, I really need to know how to actually connect clients through firewalls and NATs in the real world.
I’d make a donation for that info….
Let me save you from waiting!
My recommendation in that article will be either A) use the XBox-Live, or PSN APIs for consoles (which handle NAT punch-through for you) or B) Use a middleware solution for the NAT punch-through on PCs.
You just don’t want to be writing this code yourself. It is too error-prone. Not even the pros roll their own here. Too many combinations of routers and weird behavior. Get somebody else to do it for you!
ps. No donation required for this info
Many thanks!
Are there any middleware services (for PC app development) that you would recommend?
take a look at RakNet and OpenTNL, cheers
Very interesting and valuable read. Do you have plans to finish the tutorials listed after ‘Reliability and Flow Control’ ? I would very much enjoy reading them.
Thank you for the work you’ve put in so far, it’s brilliant.
thanks!
i do plan to finish the articles, but I have no ETA right now i’m pretty busy at work!
Any update on the articles?
I’ve been wanting to try setting up an engine using event sourcing to get the state of an entity. But I’m not sure if there is too much overhead on calculating the state based on past events.
pretty busy working on my montreal talk coming up in a month. i hope to write the article on packet serialization shortly, also the slides from the talk should give you quite a bit of information – they will cover what you are interested in, basics of state based synchronization
cheers
Brilliant, your articles are truly great. I just finished a unit about network programming at Uni. and it wasn’t nearly informative as your articles. Though more broad but beside the point. I wish I had read this article before the socket assignment.
Thanks! Glad you like them!
I am looking forward to your updates。
where can I find related resources such as related source code about your article
code.google.com/p/netgame
Thanks.
I really hope this series is not dead just yet. Its a really interesting reading which is quite easy to understand.
Not dead, just sleeping – I’m really quite busy making games at Sony right now
Thats good enough for me ;]
Hey guys, some more articles have been added.
For the immediate future I plan to expand the “what every programmer needs to know about game networking” series then create a new series about networked physics to support my GDC2009 physics tutorial talk
After GDC I hope to return to the lower level networking articles, eg. reading/writing packets, sending events, synchronizing state etc…
cheers all
Also, thanks to the folks who donated over christmas — keep ‘em coming! cheers all
hi a m an aspirant of game programming
i m presently pursuing a graduation in computer science
and preparing for SAT and GATE
i need further consultancy in this field
i want to know that in which subject or sub field of game programming or development
i will do more study that is
will i study –
NETWORKING
COMPUTER GRAPHICS
ARTIFICIAL INTELLIGENCE
PHYSICS ENGINE
GAME NETWORKING ENGINE
i want to know which subject will i study hardest and most deeply
so that i will get more advantage in terms of experience , money , marketing and easiness
help me
Study them all!