Sending and Receiving Packets

October 3, 2008

Introduction

Hi, I’m Glenn Fiedler and welcome to the second article in Networking for Game Programmers.

In the previous article we discussed the options for sending data between computers, and decided to use UDP instead of TCP. We chose UDP so that our data arrives on time without getting clumped up waiting for resent packets.

Now I am going to show you exactly how to send and receive packets using UDP.

BSD sockets

For most modern platforms you have some sort of basic socket layer available based on BSD sockets.

BSD sockets are manipulated using simple functions like “socket”, “bind”, “sendto” and “recvfrom”. You can of course work directly with these functions if you wish, but it becomes difficult to keep your code platform independent because each platform is slightly different.

So although I will first show you BSD socket example code to demonstrate basic socket usage, we won’t be using BSD sockets directly for long. Instead, once we’ve covered all basic socket functionality we’ll abstract everything away into a set of classes, making it easy to you to write platform independent socket code.

Platform specifics

First lets setup a define that lets us detect what our current platform is, so we can handle the slight differences in sockets from one platform to another:

    // platform detection

    #define PLATFORM_WINDOWS  1
    #define PLATFORM_MAC      2
    #define PLATFORM_UNIX     3

    #if defined(_WIN32)
    #define PLATFORM PLATFORM_WINDOWS
    #elif defined(__APPLE__)
    #define PLATFORM PLATFORM_MAC
    #else
    #define PLATFORM PLATFORM_UNIX
    #endif

Now lets include the appropriate headers for sockets. Since the header files are platform specific, we’ll use the platform #define to include different sets of files depending on the platform:

    #if PLATFORM == PLATFORM_WINDOWS

        #include <winsock2.h>

    #elif PLATFORM == PLATFORM_MAC || PLATFORM == PLATFORM_UNIX

        #include <sys/socket.h>
        #include <netinet/in.h>
        #include <fcntl.h>

    #endif

Sockets are built in to the standard system libraries on unix-based platforms, so we don’t have to link to anything extra. However, on Windows we need to link to the winsock library to get socket functionality.

Here is a simple trick to do this without having to change your project or makefile:

    #if PLATFORM == PLATFORM_WINDOWS
    #pragma comment( lib, "wsock32.lib" )
    #endif

I like this trick because I’m super lazy, of course you can always link from your project or makefile if you wish.

Initializing the socket layer

Most unix-like platforms (including macosx) don’t require any specific steps to initialize the sockets layer, however Windows requires that you jump through some hoops to get your socket code working. You must call “WSAStartup” to initialize the sockets layer before you call any socket functions, and “WSACleanup” to shutdown when you are done.

Lets add two new functions:

    inline bool InitializeSockets()
    {
        #if PLATFORM == PLATFORM_WINDOWS
        WSADATA WsaData;
        return WSAStartup( MAKEWORD(2,2), &WsaData ) == NO_ERROR;
        #else
        return true;
        #endif
    }

    inline void ShutdownSockets()
    {
        #if PLATFORM == PLATFORM_WINDOWS
        WSACleanup();
        #endif
    }

Now we have a platform independent way to initialize the socket layer. On platforms that don’t require socket initialization, these functions just do nothing.

Creating a socket

Its time to create a UDP socket, here is how to do it:

    int handle = socket( AF_INET, SOCK_DGRAM, IPPROTO_UDP );

    if ( handle <= 0 )
    {
        printf( "failed to create socket\n" );
        return false;
    }

Next we bind the UDP socket to a port number (eg. 30000). Each socket must be bound to a unique port, because when a packet arrives the port number determines which socket to deliver to. Don’t use ports lower than 1024 because they are reserved for the system.

Special case: if you don’t care what port your socket gets bound to just pass in “0″ as your port, and the system will select a free port for you.

    sockaddr_in address;
    address.sin_family = AF_INET;
    address.sin_addr.s_addr = INADDR_ANY;
    address.sin_port = htons( (unsigned short) port );

    if ( bind( handle, (const sockaddr*) &address, sizeof(sockaddr_in) ) < 0 )
    {
        printf( "failed to bind socket\n" );
        return false;
    }

Now the socket is ready to send and receive packets.

But what is this mysterious call to “htons” in the code above? This is just a helper function that converts a 16 bit integer value from host byte order (little or big-endian) to network byte order (big-endian). This is required whenever you directly set integer members in socket structures.

You’ll see “htons” and its 32 bit integer sized cousin “htonl” used several times throughout this article, so keep an eye out, and you’ll know what is going on.

Setting the socket as non-blocking

By default sockets are set in what is called “blocking mode”. This means that if you try to read a packet using “recvfrom”, the function will not return until a packet is available to read. This is not at all suitable for our purposes. Video games are realtime programs that simulate at 30 or 60 frames per second, they can’t just sit there waiting for a packet to arrive!

The solution is to flip your sockets into “non-blocking mode” after you create them. Once this is done, the “recvfrom” function returns immediately when no packets are available to read, with a return value indicating that you should try to read packets again later.

Here is how put a socket in non-blocking mode:

    #if PLATFORM == PLATFORM_MAC || PLATFORM == PLATFORM_UNIX

        int nonBlocking = 1;
        if ( fcntl( handle, F_SETFL, O_NONBLOCK, nonBlocking ) == -1 )
        {
            printf( "failed to set non-blocking socket\n" );
            return false;
        }

    #elif PLATFORM == PLATFORM_WINDOWS

        DWORD nonBlocking = 1;
        if ( ioctlsocket( handle, FIONBIO, &nonBlocking ) != 0 )
        {
            printf( "failed to set non-blocking socket\n" );
            return false;
        }

    #endif

As you can see above, Windows does not provide the “fcntl” function, so we use the “ioctlsocket” function instead.

Sending packets

UDP is a connectionless protocol, so each time you send a packet you must specify the destination address. You can use one UDP socket to send packets to any number of different IP addresses, there is no one computer at the other end of your UDP socket that you are connected to.

Here is how to send a packet to a specific address:

    int sent_bytes = sendto( handle, (const char*)packet_data, packet_size,
                             0, (sockaddr*)&address, sizeof(sockaddr_in) );

    if ( sent_bytes != packet_size )
    {
        printf( "failed to send packet: return value = %d\n", sent_bytes );
        return false;
    }

Important! The return value from “sendto” only indicates if the packet was successfully sent from the local computer. It does not tell you whether or not the packet was received by the destination computer! UDP has no way of knowing whether or not the the packet arrived at its destination.

In the code above we pass a “sockaddr_in” structure as the destination address. How do we setup one of these structures?

Lets say we want to send to the address 207.45.186.98:30000

Starting with our address in this form:

    unsigned int a = 207;
    unsigned int b = 45;
    unsigned int c = 186;
    unsigned int d = 98;
    unsigned short port = 30000;

We have a bit of work to do to get it in the form required by “sendto”:

    unsigned int destination_address = ( a << 24 ) | ( b << 16 ) | ( c << 8 ) | d;
    unsigned short destination_port = port;

    sockaddr_in address;
    address.sin_family = AF_INET;
    address.sin_addr.s_addr = htonl( destination_address );
    address.sin_port = htons( destination_port );

As you can see, we first combine the a,b,c,d values in range [0,255] into a single integer, with each byte of the integer now corresponding to the input values. We then initialize a “sockaddr_in” structure with the integer address and port, making sure to convert our integer address and port values from host byte order to network byte order using “htonl” and “htons”.

Special case: if you want to send a packet to yourself, there is no need to query the IP address of your own machine, just pass in the loopback address 127.0.0.1 and the packet will be sent to your local machine.

Receiving packets

Once you have a UDP socket bound to a port, any UDP packets sent to your sockets IP address and port are placed in a queue. To receive packets just loop and call “recvfrom” until it fails indicating there are no more packets left in the queue.

Since UDP is connectionless, packets may arrive from any number of different computers. Each time you receive a packet “recvfrom” gives you the IP address and port of the sender, so you know where the packet came from.

Here is how to loop and receive all incoming packets:

    while ( true )
    {
        unsigned char packet_data[256];
        unsigned int maximum_packet_size = sizeof( packet_data );

        #if PLATFORM == PLATFORM_WINDOWS
        typedef int socklen_t;
        #endif

        sockaddr_in from;
        socklen_t fromLength = sizeof( from );

        int received_bytes = recvfrom( socket, (char*)packet_data, maximum_packet_size,
                                   0, (sockaddr*)&from, &fromLength );

        if ( received_bytes <= 0 )
            break;

        unsigned int from_address = ntohl( from.sin_addr.s_addr );
        unsigned int from_port = ntohs( from.sin_port );

        // process received packet
    }

Packets in the queue larger than your receive buffer will be silently discarded. So if you have a 256 byte buffer to receive packets like the code above, and somebody sends you a 300 byte packet, the 300 byte packet will be dropped. You will not receive just the first 256 bytes of the 300 byte packet.

Since you are writing your own game network protocol, this is no problem at all in practice, just make sure your receive buffer is big enough to receive the largest packet your code could possibly send.

Destroying a socket

On most unix-like platforms, sockets are file handles so you use the standard file “close” function to clean up sockets once you are finished with them. However, Windows likes to be a little bit different, so we have to use “closesocket” instead:

    #if PLATFORM == PLATFORM_MAC || PLATFORM == PLATFORM_UNIX
    close( socket );
    #elif PLATFORM == PLATFORM_WINDOWS
    closesocket( socket );
    #endif

Go windows!

Socket class

So we’ve covered all the basic operations: creating a socket, binding it to a port, setting it to non-blocking, sending and receiving packets, and destroying the socket.

But you’ll notice most of these operations are slightly platform dependent, and its pretty annoying to have to remember to #ifdef and do platform specifics each time you want to perform socket operations.

We’re going to solve this by wrapping all our socket functionality up into a “Socket” class. While we’re at it, we’ll add an “Address” class to make it easier to specify internet addresses. This avoids having to manually encode or decode a “sockaddr_in” structure each time we send or receive packets.

Here is what our socket class looks like:

    class Socket
    {
    public:

        Socket();
        ~Socket();
        bool Open( unsigned short port );
        void Close();
        bool IsOpen() const;
        bool Send( const Address & destination, const void * data, int size );
        int Receive( Address & sender, void * data, int size );

    private:

        int handle;
    };

And here is what our address class looks like:

    class Address
    {
    public:

        Address();
        Address( unsigned char a, unsigned char b, unsigned char c, unsigned char d, unsigned short port );
        Address( unsigned int address, unsigned short port );
        unsigned int GetAddress() const;
        unsigned char GetA() const;
        unsigned char GetB() const;
        unsigned char GetC() const;
        unsigned char GetD() const;
        unsigned short GetPort() const;
        bool operator == ( const Address & other ) const;
        bool operator != ( const Address & other ) const;

    private:

        unsigned int address;
        unsigned short port;
    };

Here is how you use these classes to send and receive packets:

    // create socket

    const int port = 30000;
    Socket socket;
    if ( !socket.Open( port ) )
    {
        printf( "failed to create socket!\n" );
        return false;
    }

    // send a packet

    const char data[] = "hello world!";
    socket.Send( Address(127,0,0,1,port), data, sizeof( data ) );

    // receive packets

    while ( true )
    {
        Address sender;
        unsigned char buffer[256];
        int bytes_read = socket.Receive( sender, buffer, sizeof( buffer ) );
        if ( !bytes_read )
            break;
        // process packet
    }

As you can see its much simpler than using BSD sockets directly. As a bonus the code is the same on all platforms, because everything platform specific is handled for you inside the socket and address classes.

Conclusion

We now have a platform independent way to send and receive UDP packets.

UDP is connectionless, and I wanted to create an example program that really hammers this point home. So, I’ve setup a simple example program which reads IP addresses from a text file and sends a packet to these addresses once per-second. Each time this program receives a packet, it prints out which machine it came from, and the size of the packet received.

You could pretty easily set it up so that you have a number of nodes sending packets to each other on your local machine, just pass in different port numbers to multiple instances of the application like this:

    > Node 30000
    > Node 30001
    > Node 30002
    etc...

Then each node will attempt to send packets to each other node, its like a mini peer-to-peer setup.

I developed this program on MacOSX but you should be able to compile it on any unix-like system or Windows pretty easily, so let me know if you have any patches for compatibility on different machines.


Next: Virtual Connection Over UDP




If you enjoyed this article please donate.

Donations offset hosting costs and encourage me to write more articles!

{ 109 comments… read them below or add one }

maelfuad October 13, 2008 at 12:48 pm

Thank you very much for your articles.

I have been following this web for quiet a while, back from your java tinyptc. truly amazing.

Any chance/planning to make tutorial/book about 3d programming like the one used for your tinyptc demo?

Best regards.

Reply

cam October 14, 2008 at 6:04 pm

Are there any performance benefits to running UDP code in blocking mode on a separate thread?

Reply

Glenn Fiedler October 14, 2008 at 11:53 pm

Not sure, but it is a good technique regardless because you can accurately measure packet arrival time

If you use non-blocking sockets your packet arrival time is aliased to the beginning of each game update, whereas a separate thread running blocking sockets will be low CPU use and receive packets as soon as they come in

Of course, you’d have to setup your own producer/consumer to feed packets to the primary thread at the beginning of the frame, but thats not too hard

cheers

Reply

brian October 19, 2008 at 9:45 pm

Nice series. IMHO, don’t litter your code with #ifdef WINDOWS, etc. Put that logic in your build system and choose which files to include (e.g. sockets_windows.c sockets_unix.c) and have each file implement your API.

Reply

Glenn Fiedler October 20, 2008 at 10:36 pm

Thanks, well I see your point, but I don’t agree – primarily because there is enough commonality between each platform, I’d hate to have to make the same edits across multiple files each time I change something

cheers

Reply

BlueRaja June 25, 2011 at 8:06 am

“Thanks, well I see your point, but I don’t agree – primarily because there is enough commonality between each platform, I’d hate to have to make the same edits across multiple files each time I change something”

– That is what inheritance is for :) Make a separate class for each OS, and put the common code in the base class. Then the only `#if PLATFORM == PLATFORM_WHATEVER` code you need is when instantiating your object.

Reply

Glenn Fiedler June 27, 2011 at 6:12 am

Sure you could do it that way, but seems like an awful lot of farting about just to send and receive some packets :)

Reply

Brian Jones October 22, 2008 at 9:16 am

In your Node program, at the end, you have

while ( true )
{
Address sender;
unsigned char buffer[256];
int bytes_read = socket.Receive( sender, buffer, sizeof( buffer ) );
if ( !bytes_read )
break;

printf( “received packet from %d.%d.%d.%d:%d (%d bytes)\n”, sender.GetA(), sender.GetB(), sender.GetC(), sender.GetD(), sender.GetPort(), bytes_read );
}

Shouldn’t that be
if(bytes_read <= 0)
break;

Cause -1 is returned if there are no bytes returned or an error which would keep the inner loop going

Reply

Glenn Fiedler October 22, 2008 at 10:19 am

in this case, we’re using Socket::Receive instead of recvfrom, so its correct, but yeah if you are using recvfrom you need the <= 0

if ( socket == 0 )
return false;

#if PLATFORM == PLATFORM_WINDOWS
typedef int socklen_t;
#endif

sockaddr_in from;
socklen_t fromLength = sizeof( from );

int received_bytes = recvfrom( socket, (char*)data, size, 0, (sockaddr*)&from, &fromLength );

if ( received_bytes <= 0 )
return 0;

unsigned int address = ntohl( from.sin_addr.s_addr );
unsigned int port = ntohs( from.sin_port );

sender = Address( address, port );

return received_bytes;
}

Reply

binary October 24, 2008 at 2:48 am

Is there a reason why you are using:

unsigned int a = 207;
unsigned int b = 45;
unsigned int c = 186;
unsigned int d = 98;
unsigned int destination_address = (a<<24) | (b<<16) | (c<<8) | d;
address.sin_addr.s_addr = htonl( destination_address );

instead of:

sServerAddress.sin_addr.s_addr = inet_addr(“207.45.186.98″);

Reply

Glenn Fiedler October 24, 2008 at 9:03 am

yep. because i’d like to be able to dynamically change the address a,b,c,d without needing to compose a string

take a look at the example source, this all gets wrapped up like this:

Address address( 127, 0, 0, 1, 30000 );

of course, perhaps I should add an address constructor via string, and use inet_addr as well

cheers

Reply

Jake See May 7, 2012 at 12:13 pm

If I am not wrong, inet_addr seems to OR the a.b.d.c in the reverse order

i.e. instead of

unsigned int destination_address = ( a << 24 ) | ( b << 16 ) | ( c << 8 ) | d;

if should be

unsigned int destination_address = ( d << 24 ) | ( c << 16 ) | ( b << 8 ) | a;

Reply

Glenn Fiedler May 7, 2012 at 5:08 pm

That depends whether you are little endian or not, and whether you translate the address to network order before passing in to sockets like I do, eg.:

address.sin_addr.s_addr = htonl( destination.GetAddress() );

Reply

CP Wilkinson October 24, 2008 at 11:08 pm

I compiled your example using Xcode under OSX with no problems, however when using visual c++ 2008 on xp i had an issue rising from

inline bool InitializeSockets()
{
#if PLATFORM == PLATFORM_WINDOWS
WSADATA WsaData;
return WSAStartup( MAKEWORD(2,2), &WsaData ) != NO_ERROR;
#else
return true;
#endif
}

in main this call is


if ( !InitializeSockets() )
{
printf( “failed to initialize socketsn” );
return 1;
}

under OSX the InitializeSockets returns true and skips this, however under xp WSAStartup returns 0 or NO_ERROR if successful NO_ERROR != NO_ERROR returns false and it fails and main returns 1

Reply

Glenn Fiedler October 24, 2008 at 11:12 pm

sorry about that, try “return WSAStartup( … ) == NO_ERROR”

Reply

Myles evans October 29, 2008 at 8:24 am

I am a hobbyist just getting into Game programming and in the future when I start to learn Networking your articles will be extremely helpful. So keep it up!!

Reply

Liam October 30, 2008 at 2:16 am

Glenn I would just like to point to a piece of the code which is not(or may not be) cross platform compatible as this is what your code is striving to be.

int handle = socket( AF_INET, SOCK_DGRAM, IPPROTO_UDP );

Firstly the handle on windows is a unsigned int which is big enough to hold a pointer on a 64 bit edition windows uses the LLP64 model this will fail as an int is 32 bits yet a pointer is 64. Whilst in most implementations PF_NET is an alias for AF_NET I am not sure if this is guaranteed to be correct and technically PF_NET is the correct one to use here(protocol family not address).

Reply

Glenn Fiedler October 30, 2008 at 8:22 am

thanks liam

i did not know that about PF_INET and will fix up code in svn

also, is there a data type i can use for the socket handle that is 64bit compliant – should i just use HANDLE on win32? what about unix like platforms what is the standard here? i assume a file handle type

cheers

Reply

Liam October 30, 2008 at 9:47 am

The winsock header defines the SOCKET type using UNIT_PTR which is defined to be the correct type on a platform yet it would be better to typedef a “Socket” type, this is what I did when creating my network library for my dissertation. On *nix a plain old int is sufficient to be typedefed.

I did come across a platform which did not define IPPROTO_UDP correctly yet the distro in which this was escapes me at the moment; instead I used getprotobyname(“udp”)->p_proto .

Reply

Glenn Fiedler October 30, 2008 at 10:01 am

ok cool, thanks liam i’ll update the svn today

cheers

Reply

Glenn Fiedler October 30, 2008 at 10:02 am

also, what about 64bit unix? i guess they use 32bit handles not pointers so its good yeah?

cheers

Reply

Liam October 30, 2008 at 10:09 am

On nix int will be the correct type.

Reply

foobar December 4, 2008 at 2:39 am

I’m inclined to think that setting your sockets to non-blocking may not be the best idea. That’s because you either have to burn cycles polling (which is a form of busy waiting), or you could recvfrom() periodically, risking to lose packets when the OS runs out of buffers, or you have to use something like select()/poll().
poll() doesn’t even exist under Windows, so you’re limited to select() if Windows is a target platform. Now, select() isn’t only a bitch to use, its also quite limited in a few ways. Sure, you could implement separate, platform-specific layers using IOCP, epoll, kqueue and what, but let’s just assume that we want the code to be a *bit* maintainable too, so that is hardly an option.
Further, using select() first and then recvfrom() means two system calls (and user/kernel/user switches), whereas blocking on recvfrom() is only one system call.
So, all in all, blocking in recvfrom() does seem quite attractive, as it does just what it should do… sleep until there is something we might be interested in.

Reply

Glenn Fiedler December 5, 2008 at 12:37 am

foobar: -10 points. read the article before commenting :)

blocking is not attractive at all, unless perhaps you do the blocking on another thread – we have a multiplayer action game running at 30 or 60 frames per-second, we cant just twiddle our thumbs waiting for a packet to arrive

non-blocking sockets also do not burn cpu when used as described in this article, given than you are just going to query recvfrom n times until it returns WSA_WOULDBLOCK – so its the minimum number of kernel calls for the number of recieves as well

losing packets because too many arrive? not likely, remember its a game simulation running at 30 or 60 frames per second, so we’re polling frequently anyway

about the only reason *not* to use non-blocking in production code is

a) performance, since there are platform specific ways that are a bit faster to do it
b) measuring accurate packet arrival time, you wish to get a timestamp the exact time a packet comes in, you’d actually recv packets on another thread, as to avoid temporal aliasing with the game loop, in this case you’d be best off blocking to conserve CPU on the socket read thread

cheers

Reply

Matt Beals December 14, 2008 at 10:38 pm

RE: 64-bit compliant handle

You could explore using size_t:
http://en.wikipedia.org/wiki/Stddef.h#Type_size_t

It’s a little hackish, but it works as long as whoever implemented your stdlib/stddef isn’t doing something totally insane. Which means it works sometimes. :)

Side note, socklen_t should probably be size_t instead of int (Net.h:268).

Reply

Evan Marks December 15, 2008 at 2:28 pm

Great series of tutorials. I am having trouble with binding my socket (on OSX 10.5.5). How can i determine what the error is? My only guess is that it has something to do with which ports are open on my machine?

Thanks!

Reply

Glenn Fiedler December 15, 2008 at 9:29 pm

yes, it will fail to bind if another socket is already bound to that port

you can check the error via “errno” function

more info here (first google hit for “bind function”)

http://www.opengroup.org/onlinepubs/009695399/functions/bind.html

cheers

Reply

linux reader December 21, 2008 at 2:54 pm

In section ‘Creating a socket’:
“if (handle <= 0) …”

I think that it should be “if (handle == -1) …”, see http://en.wikipedia.org/wiki/Socket.h#socket.28.29

When running in linux, i could not create a socket because ‘socket’ returned 0. When accepting a handle value of 0, the program worked well.

Reply

Liam December 23, 2008 at 11:58 am

Neither the code in the article or the code posted by linux reader is cross platform compatible.
In winsock when a error occurs INVALID_SOCKET is returned which as I have stated already is unsigned and therefore is an error to check for less than or equal to zero. INVAILD_SOCKET is actually zero’s complement which its outcome is determined by which platform (32 or 64 bit) the machine is.
I would post code but it would probably become mangled.
Personally for unix I have an enumeration, the value of which is set to minus one and in windows I have a constant SOCKET which is assigned the value INVALID_SOCKET.

Some code looks like :
if( (m_socket = ::socket(PF_INET, SOCK_DGRAM, getprotobyname(“udp”)->p_proto) ) == GNL_INVALID_SOCKET) …

Reply

Glenn Fiedler December 23, 2008 at 8:40 pm

thanks liam i’ll fix it up, cheers

Reply

Liam December 29, 2008 at 8:12 am

Patch submitted for the Net header
http://code.google.com/p/netgame/issues/detail?id=1

Reply

Madsy January 9, 2009 at 1:32 am

I see you treat insufficient bytes written with sendto as an error, i.e when it returns less bytes than you requested written. When does this exactly happen with UDP? When it returns EAGAIN/WSAWOULDBLOCK? And can one get around this problem by sending the last part later, or avoid it all-together by sending small enough packets?

Reply

Glenn Fiedler January 10, 2009 at 10:26 am

it actually cant happen in UDP, it will send the entire packet or fail with an error code, i just check it for completeness — i want it to return the # of bytes that i sent exactly

so you dont have to do anything with UDP to avoid partial packets getting sent/received, it cant happen

cheers

Reply

Glenn Fiedler January 16, 2009 at 3:47 pm

hey liam, i have applied parts of your patch to my internal code – i will release an updated version shortly once i have tested across all machine types

cheers

Reply

Dave Cahill January 18, 2009 at 12:16 pm

Hi Glenn,

Really enjoying this series, thanks!

Two minor edits for the comprehension of the many readers who will surely pass this way:

* unsigned int maximum_packet_size = sizeof( data );
should read:
unsigned int maximum_packet_size = sizeof( packet_data );

* Wordpress seems keen to put a smiley in the code snippet where you set up the destination_address – if you put a space between the 8 and the end bracket, that should fix it.

Thanks again for a very interesting series,
Dave.

Reply

Glenn Fiedler January 18, 2009 at 12:40 pm

updated, thanks!

Reply

aissp March 11, 2009 at 5:22 pm

short notice so far. in spite of udp is connectionless protocol, some times is may be usefull to use a connect function for client (active, not for server) connection. There are two reason to do it:
(1) you increase performance
(2) you can receive icmp asynchronous messages described the error like ECONNREFUSED (if no server exists for example

struct sockaddr_in peer;
//fill structure
SOCKEt s = (AF_INET, SOCK_DGRAM, 0);
if(connect(s, (struct sockaddr*)& peer, sizeof(peer)))
error();
rc = send(s, buf, bufflen, 0);
……

Reply

John March 11, 2009 at 7:01 pm

These are some great articles. I have been twiddling with a asynchronous client/server and this has been a great help. I just have a few questions right now… Before I found this article I was simply using the documentation to build my framework. I had a class which held a socket which was put onto its own thread. I was looking into non-blocking sockets and I think that I would not need to thread if I set up my sockets to be non-blocking. Would it be best to go with the threaded model (1 thread per socket) or set it up so all the sockets are non-blocking? Can I have multiple sockets per port? Also, can you have multiple connections to a socket or do I need a separate sockets for each client?

Thanks,
John

Reply

Glenn Fiedler March 11, 2009 at 8:49 pm

unless you have some strong need to do your socket processing on a separate thread, just use non-blocking and poll for packets at the beginning of each frame

cheers

Reply

larry Hickey July 18, 2009 at 2:12 pm

I am doing the sendto recvfrom UDP thing, and to solve the problem of the network losing the reply read from a blocking recvfrom, I was going to use select, with a timeout. If select returns with a msg waiting, I can sefely issue recvfrom to get the message and it will not block, IF the message is lost the select times out, and i can re-issue the sendto recvfrom pair on the same socket. Thats fine but what if the server return msg is not really lost, just slow. Then I do the try again, with a separate sendto, recfrrom . Will that be ok,to get Two reply messages to read then?
I dont know how long long the timeout should be on the select either. It would depend on the platform an network I suppose. But then I read your article here on putting the socket into non blocking mode instead. I would then have t o do recvfrom until I get WSA_WOULDBLOCK, but the same problem, how long should I wait before I give up on the network delivery of the reply message? and if its just late, and I proceed to re-do the sendto, will I get the pair of reply messages? I suppose it does not matter if both reply messages arethe same because its just a retry, but the main problem is, how long to wait before I give up on the reply. I could time the average it takes lately and maybe give up after mean + 2 std-dev of time go by. Does that sound like a good idea?

Reply

Glenn Fiedler July 22, 2009 at 9:43 am

hey, i don’t usually do non-blocking but i assume you are doing it on another thread yes? if not, then you should definitely stick with non-blocking

regarding the timeout for select, well it really depends on what you want to do – the key thing to remember is that if you don’t receive a packet from the other side in some time t, then you can consider that person disconnected from the game by timeout

it is usual to set this value to around 10 seconds or so, but this does not mean that your timeout for select should be this value, really if you are doing a separate thread with blocking socket, you should just set timeout to infinite then process packets whenever they come in, the idea is that the call will block your recv thread until there is something to actually receive

for now, i think you should try non-blocking, just poll until there are no more messages to recv at the beginning of each game frame – use a sequence number at the top of your packet to determine the most recent data, throw away the rest

cheers

Reply

Glenn Fiedler September 6, 2009 at 5:55 pm

it’s an error message, of sorts – but i don’t treat it like that

the idea is that in your loop you just keep calling recvfrom until it returns <= 0

this way you keep polling until no packets are available… next frame you try again, until <= 0

Reply

thec September 6, 2009 at 3:22 pm

Hi Gaffer (long time no see btw, want you back to #coders more often!).

I’ve tried this in linux and actually it’s working, but somehow when I do recvfrom(…) I get -1 if there is nothing in the readbuffer, and if I check the errno I get #11, which is defined as:

#define EAGAIN 11 /* Try again */

Which to me maybe shouldn’t be an error message, but it is… so in linux it seems like you do need this check as well.. maybe I’m mistaken somewhere, and if so please correct me.

Thanks gaffer, you have brought quality for many years now and in outbreak we’ve been with you since first versions of ptc :-)

Reply

thec September 7, 2009 at 6:06 am

right… I see how you work around the problem, but it’s not very wise to skip all error messages like disconnect etc…

Reply

Glenn Fiedler September 7, 2009 at 10:03 am

you’re totally correct – in my production code i have an error reporter for all socket operations that logs the error code when they happen, and i have special case handling for handling for EAGAIN

cheers mate

Reply

David Moran November 6, 2009 at 1:36 am

Hello Glenn
Thank you very much for your article, it is the best I have found in internet.
I have one question… what if you want to get only the LAST packet?. I think it has a “time cost” to make a while() and it will be best to get only the last packet (the one you want normally for UDP (real/time) communications; it does not matter was happens before¡)
Is it any way to read directly the last packet?
Thank you very much

Reply

Glenn Fiedler November 6, 2009 at 9:36 am

What you would typically do is put a sequence number at the top of your packet and increment it each time you send one

Then loop and receive all packets, which will typically be very small — then you can use the sequence number to discard all but the most recent packet from each source, per-frame if you wish

cheers

Reply

Martin Bell December 16, 2009 at 2:52 pm

Hello.

Many thanks for your article. I’m just starting to learn the basics of network programming so this is very useful, I think I may have spotted a bug however. Should the use of fcntl for setting the non-blocking flag be like this?

int flags = fcntl(handle, F_GETFL, 0);
fcntl(handle, F_SETFL, flags | O_NONBLOCK);

Ie it has only 3 parameters and sets all the flags in one go. The 4th seems to be ignored in your example and I’m guessing all other flags are being reset to zero as a side effect. I’m testing this on a Mac.

Thanks

Reply

Geo February 3, 2010 at 9:44 am

Hello,

when implementing UDP on the primary thread you need non-blocking sockets. That’s obviuos. What about using select with (0,0) timeout and then recvfrom if select returns ok ? Is it the same with your solution or worse?

thank you!

Reply

Glenn Fiedler February 3, 2010 at 9:55 am

Select is designed for use with TCP so you get notified when one of your (multiple) TCP sockets has data ready to read — there is no reason to use select here, because you only need one UDP socket to service any number of virtual connections. Use non-blocking sockets instead.

Reply

Daniel March 30, 2010 at 3:14 pm

Hi,
Great series of tutorials, would it be OK to instead of having a non-blocking socket and waste cpu cycles, could I keep the socket blocking and make it load up data into a buffer which another thread processes?

Also,
unsigned int a = 207;
unsigned int b = 45;
unsigned int c = 186;
unsigned int d = 98;
– Shouldn’t those be unsigned chars?

Thanks

Reply

Acetate May 6, 2010 at 7:32 am

@Daniel: If you use unsigned chars, shouldn’t you also make sure that they aren’t zeroed out by the shifts occurring just before this assignment (I don’t know if there is any implicit cast): unsigned int destination_address = ( a << 24 ) | ( b << 16 ) | ( c << 8 ) | d; ?

@Glenn: I have been reading your articles (both physics and networking series) on my spare time for a few days now and I learnt A LOT. Those are clear, succinct and overall well written explanations about often overlooked areas of game development. I think I've just made the jump from bad to mediocre game programmer thanks to you. :)

Reply

Nele October 15, 2010 at 7:59 pm

Hi,

Thank you for such great articles. One question regarding blocking/non-blocking.

In case threading is used to handle blocking calls, wouldn’t this just offload the work to the kernel side? In case you have like a 1000 clients connected to your server, you need so many threads to deal with them. In terms of efficiency, how would that compare to just having the main thread poll each one of them in every pass?

Reply

Glenn Fiedler October 20, 2010 at 5:11 am

I’m not really qualified to answer. Typically I’ve worked on low player count games on consoles (<8) — in these cases it's usually P2P and it's pretty obviously for each console how to send/receive packets the best way. I'd recommend trying a few different options and profiling to see what is best for your situation. cheers

Reply

Mystran December 14, 2010 at 1:30 pm

If you have 1000 clients connected to a server, you can still use a single socket if you use UDP. You recvfrom the socket, you get sender address/port + data, and you use the address to map the data to the current client internally (ie, use some associative data structure from address -> client structures). Having 1000 sockets (and much less threads) is pointless, since you can just use a single one, and you’ll recvfrom always the next packet in queue, no matter which client sent it.

In the case of TCP, you’d have single listen() socket, then accept() clients to get the connected sockets. Now 1000 concurrent clients would indeed translate to 1000 concurrent sockets (+ the one to accept more). You still wouldn’t want 1000 threads though, because those are relatively expensive (especially in large numbers). You wouldn’t want to poll each socket manually either, because that’s just waste of electricity (converted to heat when the CPU does useless work). Instead you’d want to make ‘em non-blocking and then use select() or poll() or similar to figure out which ones are worth manual polling, then loop over just those that report activity.

Personally I think TCP is totally overrated (and actually more complex to handle well on server-side), and a lot of things (even things like file-transfers, where you really only need every block once, yet the order is totally irrelevant) are more sensibly done over UDP.

Reply

Peter February 19, 2011 at 10:29 pm

Hey!
Thanks for this very great tutorial!
I implemented it, and it worked great so far, but now i dont want to send strings around, but integers. and multiple of them in one packet. I just cant get it to work, i try to keep the code the same like the acks, but i still dont get how i should write and read them.
Can you give an example? That would be great!
thanks!
Peter

Reply

Glenn Fiedler February 21, 2011 at 10:27 pm

I don’t have an article up on this yet, but the way I do it is with a bitpacker and a unified read/write serialize function.

You can see example source code for this here:

http://code.google.com/p/netgame/source/browse/#svn%2Ftrunk%2F07%20-%20Reading%20and%20Writing%20Packets

Reply

Peter February 21, 2011 at 10:45 pm

thanks! I will right on start digging into this code :D
thanks for your effort, and im very looking forward to the
next tutorial part!
thanks!

Reply

Ameya Deshpande March 19, 2011 at 3:40 am

Nice detailed description is helping me in my academic project in my Masters. Thans to you.
I am working with one comapny in France for the same project. I have to use UDP connections to Implement the game.
If you can please share the algorithm of the codes to create sockets. Or else please let me know how it can be done in java. I am fairly new to Java and socket programming in java.
Thank you !!!

Reply

Glenn Fiedler March 23, 2011 at 3:15 pm

Unfortunately I don’t program in Java! It’s probably really easy though, is there a “Socket” class you can use?

Reply

Michael Day March 29, 2011 at 11:46 pm

Ameya: See DatagramSocket and DatagramPacket Java classes. Google these terms for examples.

Glenn: Great series, by the way. Saw you at GDC a couple of years back and been copying your code ever since.

Reply

Glenn Fiedler March 30, 2011 at 1:59 am

Thanks Michael, cheers

Reply

Jonathon Ogden June 10, 2011 at 1:10 pm

Very good article once more Glenn and you covered cross-platform compatibility. I was wondering if you would mind me making a suggestion which is to either add to this article (or where appropriate) on writing IPv4/IPv6 independent applications? Appreciate it may require a separate article, perhaps I could help out a little if you are interested at all?

Reply

Glenn Fiedler June 11, 2011 at 6:20 pm

Sounds like a good idea for a separate article.

(I haven’t had a decent chance to play around with IPv6 yet because my ISP is dumb.)

Reply

Jonathon Ogden June 12, 2011 at 9:59 pm

I’ve been playing around with it for a little while now and it’s relatively simple. Got some sample code if you wish to review it and use it at all? Can also use a dual-stack architecture where you can bind to IPv4 and IPv6 equivalent addresses to a hybrid socket of sorts so it transparently uses either address family.

Reply

Glenn Fiedler June 14, 2011 at 12:00 am

This sounds really useful, why don’t you upload it to github.net and link it from here?

Reply

Jonathon Ogden June 14, 2011 at 8:29 am

Sure thing! Will do it as soon as I possibly can.

Jonathon Ogden June 16, 2011 at 2:00 pm

Well, there is a lot more to do to this but feel free to use it if it is any good: https://github.com/ByteKiller/Network-Programming/blob/master/NetworkSnippet.cpp

Reply

Glenn Fiedler June 17, 2011 at 4:05 am

Thanks!

Reply

Monco August 30, 2011 at 11:49 am

Awesome article! Thanks a lot :D

Reply

Tin September 13, 2011 at 1:51 am

UDP socket, Linux receive data twice

Hi everyone,

I am using UDP socket to transmit data from a windows program to a program on Linux. The application on Linux receives data and the application on windows sends data. The problem is the function recvfrom(…) return the same data 2 times. So, I wonder if there is any way to set so that the socket will automatically remove data read from the socket queue? (TCP/IP socket does not have this problem).
Thanks very much,
Tin,
poste

Reply

minime October 26, 2011 at 10:40 am

Hi, you write:

Packets in the queue larger than your receive buffer will be silently discarded. So if you have a 256 byte buffer to receive packets like the code above, and somebody sends you a 300 byte packet, the 300 byte packet will be dropped. You will not receive just the first 256 bytes of the 300 byte packet.

but my linux manpage(7 udp) says:

All receive operations return only one packet. When the packet is smaller than the passed buffer, only that much data is returned; when it is bigger, the packet is truncated and the MSG_TRUNC flag is set. MSG_WAITALL is not supported.

What is correct now? (and what is portable, since maybe I /want/ to always drop messages bigger)

Reply

Glenn Fiedler October 28, 2011 at 7:59 pm

interesting. i was not aware of this! it seems you would want to check the MSG_TRUNC flag in addition

Reply

Glenn Fiedler October 28, 2011 at 8:05 pm
Glenn Fiedler October 28, 2011 at 8:07 pm
Kusanagi December 25, 2011 at 9:34 pm

Dont know if this is the correct place, but.
If any one else, has had the same problem.. and do read
all the comments, damn its allot.

I figure they have a chance to find it.

The problem was it would not initialize sockets..
(code downloaded from the svn btw.)

InitializeSockets() in Net.h (obviously..)

I removed all the plattform independent code, windows only now.
for easier read.

inline bool InitializeSockets()
{
WSADATA WsaData;
return WSAStartup( MAKEWORD(2,2), &WsaData ) == NO_ERROR;
}

It only worked with the return on a line by it self with wsastartup:

inline bool InitializeSockets()
{
WSADATA WsaData;
WSAStartup( MAKEWORD(2,2), &WsaData ) == !NO_ERROR;
return WSAStartup;
}

My question:
Do this change anything major?
Or nothing at all.

I am trying to study and understand the code
Using VS2010 Express.

Reply

Glenn Fiedler December 27, 2011 at 8:58 am

It changes a lot, the second code is incorrect and is equivalent to always returning true.

Reply

Karsten Malmquist February 10, 2012 at 10:24 pm

I’m having problems with linker errors on windows, all of the functioncalls to members of the winsock2 library are generating linker errors, I tried to link different files:
wsock2.lib, wsock2.dll, winsock2.lib, winsock2.dll, ws2_32.lib, ws2_32.dll

Maybe the Dev-C++ compiler is causing the problem?

Reply

Glenn Fiedler February 12, 2012 at 5:04 pm

Could be. Try to find an example for your compiler that links and reverse engineer from there.

Reply

Jeaye March 7, 2012 at 9:02 am

Years later and you’re still helping out those who ask. My hat goes off to you, sir; thanks for the fine article. ;)

Reply

Glenn Fiedler March 8, 2012 at 12:34 am

Thank you!

Reply

nyrtZi May 4, 2012 at 11:46 am

Anyone ever considered SCTP? Seems like an interesting mix of TCP and UDP features and could maybe provide a shortcut allowing one to get UDP with benefits without having to write the hard parts.

Reply

Glenn Fiedler May 5, 2012 at 4:42 pm
Tina May 28, 2012 at 9:00 am

Hi,thanks for your article,it’s very useful for me !
But i got one question when i running this codes on my computer!I can’t send the packets to the same network segment to the host,it’s successful to send to the different network segment.what’s the reason about this?
I’m looking forward to your reply.Thanks very much!

Reply

dlopez November 22, 2012 at 2:22 pm

Hi,

I’ve a problem with an application in C “client” that received some packets for UDP and second application “server” to send some packets AudioInputStream to do in JAVA.

My problem is that application to do in C, I’ve read a string “Error UDP frame too small :1024″ but I don’t know, how to solve in JAVA.

The length buffer is 1024 bytes in JAVA and I suppose that I could send this same length buffer, because I change length in JAVA thows a exception.

Reply

antony February 13, 2013 at 2:03 am

How thoughtful of you to have taken extra effort, not only to write a helpful article, but also to write it clearly and succinctly, helping me (and I’m guessing many others) approach and better understand the low-level stuff that is often abstracted away.

How could I not want to follow along?

Reply

Glenn Fiedler February 13, 2013 at 4:47 am

Thanks!

Reply

bob February 14, 2013 at 8:26 pm

hello Glenn Fiedler,

Assume a UDP (SOCK_DGRAM) socket is “connected” to an IP-ADDRESS X/PORT # Y and THERE IS a server at that IP-ADDRESS X bound to that PORT # Y ready to receive messages. Then, what happens if we use a SENDTO call to send a message to a different IP ADDRESS A/Port # B

Reply

Glenn Fiedler February 15, 2013 at 6:04 am

UDP sockets have no concept of connection — bind to the port you listen on and use sendto to send packets to any IP address you want

Cheers

Reply

bob February 15, 2013 at 7:57 pm

That’s because my implementation of UDP is a “connected” state by calling to connect() before send().Now can i use sento() to send a message to a different ip address and port .

Reply

Glenn Fiedler February 16, 2013 at 1:41 am

I repeat. UDP sockets have no concept of “connection” :)

Reply

tushar babbar May 5, 2013 at 7:03 pm

Hey Bob!!!

I believe you are trying to figure out something discussed here, http://stackoverflow.com/questions/5823723/when-will-a-connected-udp-socket-be-closed-by-the-os .

Hope this helps.

Cheers!!!

Reply

bob February 16, 2013 at 8:59 pm

Thanks glenn

Reply

JOHN February 21, 2013 at 7:00 am

THE PROGRAM MUST BE IN JAVA
platform windows
I WANT THAT MUST DISPLAY IN FRAME PACKET SERIAL NUMBER,PACKET SIZE,PACKET LENGTH,CRC ETC… THE FILE MUST BE ABLE SEND TO CLIENT AND SERVER ALSO…

THE FILE MUST NOT REPLACE WHILE SENDING IN PROTOCOL THAT MEANS IF AN ABC.XLS FILES ARE SENDING IN PACKET ….. IT IS DELIVER TO THAT DESTINATION POINT AGAIN SENDING THAT same FILE to same address THAT MUST NOT BE REPLACED CAN ANYONE GIVE THE CODE PLEASE…………..

Reply

Glenn Fiedler February 22, 2013 at 7:14 am
Ephasme March 12, 2013 at 8:12 pm

There is a mistake at the beginning:

We chose UDP so that **our our** data arrives on

Reply

Glenn Fiedler March 12, 2013 at 11:01 pm

Fixed and thanks

Reply

Horst April 3, 2013 at 3:40 pm

Just want to say “Thank you” for sharing your knowledge, you saved me a lot of time and helped me getting started on network programming without having to study gibberish literature.

Thumbs up, go on with your great work.

Reply

Jon Whetzel April 12, 2013 at 5:56 pm

Thanks so much for putting this together, Glenn. I used your demo program during my Introduction to Game Development class, and my students throughly enjoyed it. I packaged your demo as a Visual Studio 2010 project (we’ve been using XNA for lab assignments) and have it up on my website.

http://www.johnnyrocketpants.net/archives/190

Thanks again for these tutorials.

Reply

Glenn Fiedler April 13, 2013 at 12:35 am

Thanks Jon I’m glad it’s useful

Reply

reyaan April 15, 2013 at 2:32 pm

how to Wrap a Packet in IP and send it to a specific IP after creating a UDP connection by using socket programming in c ?? plz reply soon…

Reply

Glenn Fiedler April 16, 2013 at 5:09 pm

Did you not read the article?

Reply

shari May 10, 2013 at 7:08 am

Hi Glenn! it is nice to go through your article. I am using MAC environment objective c xcode to cretae udp connections over socket. Will you Please help me in

How to create socket –syntax
how to send and recieve messages through socket–syntax

Reply

Glenn Fiedler May 10, 2013 at 11:27 pm

You can use all C++ code in this article inside objective C if you change the .m file type to “Objective C++”. Alternatively, there are higher level objective C networking APIs you can look into. cheers

Reply

Brian June 29, 2013 at 10:10 pm

Hello Glenn. I am extremely impressed that you still respond to the posts on this article nearly five years later! This article is great, and I actually have something similar of a Socket class built for a Linux Server to handle connections for sending and receiving TCP packets. I actually use blocking mode on separate threads to send and receive TCP packets (short words such as GET and SET).

I know this was written for game development, but I have a question for you about devices. Say that I had a WiFi chip I wanted to send commands to directly over the internet via a server, but the WiFi chip is connected to my home wireless network through my router’s WiFi (and firewall). Is it possible to somehow detect the WiFi chip’s IP address and send a packet directly to it via the server without opening ports through the firewall? Right now I have to connect to my server from the WiFi chip, and create a socket to communicate back and forth, but I don’t want my WiFi chip to have to go contact the server first. Thanks in advance.

Reply

Glenn Fiedler July 6, 2013 at 3:42 am

The wifi external IP/port is going to depend entirely on your router setup. To do what you ask is basically a form of NAT punchthrough. Generally speaking, this is unreliable and a pain in the ass. You can of course google it, and lookup “STUN server” and various other approaches. However, if you want this to be as reliable as possible I’d stick with your current approach of talking to the server.

cheers

Reply

Anon September 11, 2013 at 9:30 am

Hi, I have recently started to read up on network programming and have found that yours is the one with good guidance and clear instructions and explanation. However, I have encountered a problem. I have made two .exe, one to send packets and another to receive.

I had receive.exe loop the receive function until sender == Address(127, 0, 0, 1, 1990) then the program ends

Send.exe to loop the send function to Address(127, 0, 0, 1, 1990) every second until I close the program

Problem is I don’t receive anything at all, I even tried it on two different computers.
Is there anything wrong with what I am doing?

Also I have tried your example program and it worked well.

Thanks

Reply

Leprechaun September 11, 2013 at 10:32 am

Hey, I have a question, how can I put a domain name instead of an ip address? Thank you!

Reply

Claude Adrian, CodeAngry September 30, 2013 at 5:36 pm

@Leprechaun: See getaddrinfo (or use DNS if you really wanna go low-level). Don’t forget to freeaddrinfo when done to release the PADDRINFO buffer.

Reply

Leave a Comment

{ 2 trackbacks }