This year I’ll be presenting a new version of Networking for Physics Programmers as part of the physics tutorial day on Tuesday.

As for the material to be presented this year, it will differ significantly from prior years. I’m going for more advanced material this year so I’m not going to spend 5 minutes explaining UDP vs. TCP. We’ll be skipping that and going right to the interesting stuff which is:

1. How to develop your own custom network protocol in UDP. Reliability and acks, node / transport abstraction, how to structure your game protocol and serialize packets.

2. Overview of game networking models. Pure client/server, FPS networking model with client side prediction, deterministic lockstep, authority scheme. Examples of games using each technique. How each networking model works. Advantages and disadvantages. Understand the trade-offs so you can pick the best networking model for your situation.

3. Worked example: authority scheme for networked physics in a large streaming world. Same cube demo. Different networking strategy. This year I’ll be doing it dedicated client/server style instead of P2P and I’ll focus on all the practical details and tricks required to synchronize physics state in under 64kbit/sec such as: compression tricks for position, orientation and linear/angular velocity, how to prioritize object updates when you have too many objects to fit in one packet, using a reliability system and acks to avoid sending at rest objects over and over, lossy compression, quantization and other neat tricks.

Hope to see you Tuesday, March 1, in Room 3007, West Hall Moscone Center. I’ll be talking from 3:30PM- 4PM

Schedule details here.

UPDATE: Unfortunately the original material as presented above consistently took 2.5 hours during practice sessions. As a result, I’ve had to cut section 1 and 3. There is a lot of great info in section 2 and the feedback from practice sessions was consistently that it was the best section of the talk. If you’d like some information about #1 and #3 you can ask questions during Q&A, plus I’ll be available during GDC for impromptu meetings etc. where I can go over this info with a whiteboard or a notepad. DM me on twitter if you’d like to meet up: @gafferongames

Big thanks to everybody who attended the physics tutorial today. You guys had some excellent and very thoughtful questions at the end of the talk. I really enjoyed answering them – thanks for being a great audience!

UPDATE: I just uploaded the final versions of the slides with lots of extra notes + instructions for reproducing the demo sections. If you downloaded the previous versions then you should take another look!

Here are the final keynote slides for “Networking for Physics Programmers” with presenter notes: http://bit.ly/ckf1Xu

Here are the final PDF slides with notes sans superfluous visual effects: http://bit.ly/9CFzWb

Here is the demo shown in the talk: MacOSX 64bit Binary + Source, Win32 Binary

Have fun!

For some bizarre reason I felt inspired to spend my one day off from work during crunch writing an article chronicling the development of modern game protocols. From RTS peer-to-peer lock step, to client/server, right up to FPS games and client-side prediction.

And I’m strangely enamored with the result, it flowed very well when writing it – and I believe it to be the best article I’ve written so far. It’s also the first article where I’ve felt compelled to include references to other people’s work instead of presenting just my own research.

What every programmer needs to know about game networking

You’re a programmer. Have you ever wondered how multiplayer games work?

From the outside it seems magical: two or more players sharing a consistent experience across the network like they actually exist together in the same virtual world. But as programmers we know the truth of what is actually going on underneath is quite different from what you see. It turns out that it’s all an illusion. A massive sleight-of-hand. What you perceive as a shared reality is only an approximation unique to your own point of view and place in time.

Very sad news earlier this week: Pandemic Studios was shut down by EA.

No point moping about although. What’s done is done. I choose to remember the good times not the bad: the friends I made and the colleagues I worked with at Pandemic and the shared experiences we went through. No matter how difficult and stressful they were, these experiences bind us together.

Here is my own personal Pandemic story: having worked for Irrational Games, and then Team Bondi – Pandemic flew me out from Sydney, Australia to work on Mercenaries 2 in Los Angeles. Obviously, this was a very significant event in my life, resulting in me settling in LA, meeting my wife, and eventually being hired by Sony. Along the way there were a lot of ups and downs: Getting acquired by EA, one week having an incredible demo and winning support from management, the next week struggling to get resources for the designers to test in COOP and fighting over stupid things like syncing debris and pausing in multiplayer. It was one hell of a rough ride!

In the last 6 months on Mercs2 – Andrew Goldman, CEO of Pandemic, donated his office to the multiplayer team so that we could all work closely together and make sure Mercs2 shipped with COOP. It’s a story of 8 friends working very closely together (literally!) under tight deadlines. Of persistence and determination in the face of tremendous odds, when many of us – myself included – were not even sure it was even *possible* to network an large open world game like Mercs2. We kept on and in the end, succeeded. Mercs2 became the first open world game to ship with drop-in / drop-out COOP throughout the entire singleplayer campaign.

So here, never before told – is the story of the Mercs2 Multiplayer Room and the men who made it what it was.

Spent most of my flight thinking through my slides and material and making adjustments, then watched through Harry Potter the Half Blood Prince, but we arrived before the movie finished! Now I must find out how it ends…

Arrived Friday night around 9pm. Discovered this while in the customs line that my passport was no longer in my pocket. Holy fucking shit! Frantic 5 minutes – back to the plane and we discovered it had slipped out of my pocket and down the side of my seat. Phew!

Through customs I arrived at the Hilton Bonaventure and checked into a very nice room. At this point, I dressed up warmly and headed out to Old Montreal to check out some restaurants and bars. A very nice area but a bit quieter than I expected. I found a really nice bar with good food and had great short ribs, a nice syrah and Creme Brulee for dessert. Oh yes.

This morning I woke up feeling a bit tired… a slight time difference between here and LA. Went around to explore downtown Montreal. I found St Catherine Street and the main area and walked up along all the shops. Walked past the strip Club area. Pushy bouncers. No thanks I don’t want to go in. No really. I mean it. Hit the Apple store to buy another USB ethernet adaptor for my Macbook Air (I left it home =p), then enjoyed some good Coffee at Cafe Myriade.

A few hours on my talk then met up with Aaron Contreras an old Pandemic buddy. Checked out the plateau area around Ubi and the italian neighbourhood. Good times. Aaron has a sweet studio style apartment. Jealous!

Back at the hotel now and some more work on the slides. I want to finalize the first half of the talk tonight, then hopefully I’ll have a bit of time tomorrow to spend walking around Montreal at lunch instead of spending the whole day preparing for the talk!

cheers all

Update: the demo from the talk is now available at google code.

Traditional networking techniques work well for simple linear motion, but start to break down when networking complex rigid-body physics simulations where objects can tumble, stack and interact with each other. This light-hearted and humorous tutorial takes a look at various options, techniques and pitfalls to watch out for when networking this sort of simulation, providing you with a tool-bag of new techniques and ideas you can use to network your physics based game.

I’ll be performing this talk at the Montreal International Game Summit on November 16, 2009 – 10:15AM in the Verdun Room. I’m honored to be invited to perform one of the first sessions at the conference and hope to see some of you guys there! Please come up after the talk if you would like to discuss anything at all I will be available for questions.

Plus, this will be my first time visiting Montreal so I’m very excited!

I have the demo for the talk almost ready and have been working on the slides over the last few weekends. Right now I’m trying to lock down the overall theme of the talk and the content. My goal is to provide lots of interesting information without going to deep into the specifics on any one thing. Ideally folks should leave the talk with a great outline of the whole field and some great ideas and inspiration to take home to apply to their own projects!

Here is a more detailed version of my talk plan:

I intend to start by asking the question: why is it that very few games provide meaningful interaction with networked physics objects? In other words, folks it’s 2009 – why exactly are we still running around mostly static worlds shooting other people in the face?

There are many reasons. It is difficult to light a dynamic world. Character movement and animation is difficult. But also, it seems very difficult to network a dynamic world. Why is this? Traditional networking techniques don’t work. Objects can stack and tumble and interact with each other.

In truth: It’s like a puzzle. It’s actually really simple – you just need to think in a different way.

Quick 5-10 minute recap of a game networking engine. UDP vs. TCP, how to construct packets, serialization, reliability, flow control. All of the stuff in my online book and more at a brisk pace for folks already in the field. My goal is simply to get everybody on the same page and provide some very useful, concrete information for folks in the field. Here is my network engine and how I do everything.

The plumbing is out of the way. Now, lets explore options for getting a physics simulation on one computer to be simply *viewed* on another. Two main options: 1) run the simulation on one side and present an approximation on the client side (pure client/server model), 2) run the simulation on both sides and synchronize via state replication. Compare and contrast these two approaches. Touch on time synchronization and real-time nature of protocol and why it is now this way vs. traditional networking technique.

Latency compensation. Consider the same situation: A simulation synchronized between two computers, but now client controls their own object. Recap traditional client side prediction from FPS games. Attempt to apply it to network this physics simulation. Problems: Rewind and replay cost. Simulation islands. Complexity. Authority scheme as an alternative. Dining philosophers problem. Distributed programming. Dykstra. Consistency vs. throughput trade-off. Case studies.

Live demo. P2P authority scheme. Late joins. Corrections. Reverse corrections. Challenge: Think outside the box. Maybe client/server should not be the automatic choice? How important is anti-cheat for your game? How important is meaningful interaction with a dynamic world? Which one wins? Is it possible to have both? How?

Future looking. What can you do moving forward? What can physics engine developers do to make their engines network better? What do we need before this becomes practical? What will game networking look like in 10 years? Bandwidth considerations. Real-time protocol requirements. IPv6. RSVP, Flow Routing. QoS guarantees. Network neutrality.

Summary. Bring it back to the original point about the puzzle. Conclusion. Best approaches for each situation. Call to action. Triumphant dance. Q&A.

That’s about it for now. Please be sure let me know what you think and if you are planning on attending the conference. Sound off in the comments below – thanks guys!

It seems that every few months I read a post-mortem or I hear about a case where a team had to cut multiplayer at the last minute, because it didn’t fit into the schedule, or they couldn’t get it working, or because they want to “focus on the singleplayer experience”. This is especially true of teams that try to convert their existing singleplayer codebase to to support multiplayer.

Hell, I’ve even been in this situation myself with Mercs2, and I can tell you we only just made it by the skin of our teeth – despite having a good team, the full support of the studio and plenty of resources to help us out.

So why is it that so many teams fail to integrate multiplayer at the last minute? Why does it at first seem to be going so well, but then bog down as teams struggle to get it multiplayer to shipping quality?

In my experience it boils down to this: debugging multiplayer games is hard. Not just your ordinary hard. More like pulling your hair out, grown man reduced to tears, holy shit it’s 6AM I’ve been up all night and the E3 demo is in 4 hours and it still doesn’t work hard.

So in this article, I’m going to share my general process, techniques and tools that I use for debugging multiplayer games.

I hope these techniques will help you out debugging your multiplayer game!

Read the article in Networking for Game Programmers

Good news! I’ve been invited to speak at the Montreal International Game Summit 2009

I haven’t decided on a name for the talk yet, but it will focus on networked physics and combine material from my GDC 2009 tutorial, lecture and new material I’m going to develop over the next few months.

The basic overview of the talk is something like this:

• How the internet works and why we should use UDP
• Bandwidth considerations, packet structure, 64kbit/sec
• Describe basic algorithm for synchronization: push/pull/serialize with visual smoothing
• Synchronize one server controlled cube, client is just a viewer of simulation
• Synchronize two player controlled cubes, client and server cube.
• Effects of latency, client side prediction
• High simulation cost for rewind and replay, client side prediction breaks down in a dynamic world
• Dynamic world synchronization: options – client side prediction, no prediction, styrofoam, authority scheme
• Fiedler’s Cubes demo showing authority scheme to avoid high cost
• Corrections, reverse corrections
• Compression techniques
• Cool demo to go out
• Extended Q&A

The idea is to pretty much combine all the talks I’ve done so far on networked physics and present the information in a clear, simple manner – anybody who comes to this talk will leave with all the knowledge they need to network a physics simulation.

And of course, I get to spend a weekend in Montreal with my sweetheart about 2 months after we get married

So you see, it’s win-win folks

Incredible. Just Incredible! Heads are screwed on right down under. After years of bungling they’ve made the right call. Just perfect!

Kevin Rudd has announced that the NBN tender process has been terminated, and that the government will go it alone on a new $43 billion broadband network.

The new wholesale-only network will connect 90% of homes with fibre to the home and will offer 100Mbit/s, with “next-generation” wireless and “third-generation” satellites to cover the remaining population. The network will be “open access” so retail ISPs can build their own products to sell to businesses and consumers.

If they keep this up, I might just start getting a bit homesick!

source: whirlpool.net.au

Here are the slides for my tutorial tomorrow

The goal of the tutorial is to show people the basic ideas of how to network a physics simulation

The key points are:

• Use UDP, not TCP
• Aim for 64kbit/sec – That’s 256 byte packets, 30 times per-second **
• Handle out of order packets with a sequence number
• Send input and physics state in your packet
• Snap the physics state and extrapolate
• Run the simulation on both sides, and use the input to improve extrapolation
• Use visual smoothing to hide errors

Of course, there is a lot more to it but I hope this serves as a good starting point if you want to get into networked physics

Cheers!

** actually, that’s a little bit over 64kbit/sec, especially when you include IPsec header for consoles and UDP packet header, it’s just such a nice number for folks to remember, I couldn’t resist a little white lie