Socket programming using the select system call

Server and clients

1.0 Client-Server Paradigm

The Client-Server paradigm divides the software architecture of a system in two parts, the server and its clients. The server works in the background and maintains the system-wide database. Using the database, it provides the functions for system operation and responses to queries from the clients. The clients provide the user interface for human operators using the system. The server and client may be running as different processes on the same computer, or, running on different computers located in the same building, or might be running on computers located in geographically apart continents separated by vast oceans.

Consider the following high-level pseudo code for the server.

initialize socket;

while (1) {
    accept (block for) a connection from a client

    when a connection request comes,
       create a thread for further communication with the 
       requesting client on the socket returned by accept
}

The server has a socket listening for new connections from clients. Assuming the socket to be non-blocking, the server blocks for accepting a connection from a prospective client. However, the server has to receive messages from existing clients on different sockets and has to block while receiving messages on those sockets. We don't do non-blocking receive as it eats up the precious CPU time. One easy solution is to have a multi-process or multi-threaded server, where each process or thread blocks while receiving data from the respective client. But, there are significant overheads in creating new processes or threads (tasks). Also, for good performance, we can only have a limited number of concurrent tasks. The recommendation is to have the number of threads equal to total number of cores in the CPU of the system. So, our solution does not scale with increase in number of clients. We need a system call that monitors a set of sockets and tells us which sockets have data to be read. We can, then, go ahead and read data from those ready sockets and do the needful. Fortunately, the select system call does just that.

2.0 select

#include <sys/time.h>

struct timeval {
    long    tv_sec;         /* seconds */ 
    long    tv_usec;        /* microseconds */
};

#include <sys/select.h>

int select (int nfds, fd_set *readfds, fd_set *writefds,
            fd_set *exceptfds, struct timeval *timeout);

void FD_CLR (int fd, fd_set *set); 
int  FD_ISSET (int fd, fd_set *set); 
void FD_SET (int fd, fd_set *set); 
void FD_ZERO (fd_set *set); 

The select system call monitors three sets of independent file descriptors. The file descriptors to be monitored are specified in the three file descriptor sets pointed by the second, third and fourth parameters to the select call. The file descriptors in the set pointed by readfds are monitored if any of these is ready for reading. That is, a read on ready file descriptor would not block. Similarly, the descriptors in the set pointed by writefds is monitored whether there is space for a write operation. However, a big write might still block. The file descriptor set pointed by exceptfds is monitored whether any descriptor is having an exceptional condition. The first argument, nfds is the biggest file descriptor in the three sets plus 1. The last parameter timeout is a time duration in the struct timeval format. If none of the descriptors in the three sets become ready, select would return after the interval pointed by timeout. If timeout is NULL, select would block till at least one descriptor in the three sets is ready. The pointers to any of the three sets could be NULL, and the corresponding set is ignored.

When select returns, out of the specified file descriptors, the ones that are ready are set. Since the file descriptor sets are modified by select, the sets need to be re-initialized for the next select call. On Linux systems, if a timeout was specified, the un-slept time is returned in the struct timeval pointed by the timeout argument to select.

select returns the number of file descriptors returned in the three file descriptor sets, or -1, in case of error.

There are four macros for modifying file descriptor sets. FD_ZERO initializes a file descriptor set, clearing all the descriptors in it. FD_SET sets a descriptor in the set, while FD_CLR clears a file descriptor. FD_ISSET is for checking whether a descriptor is set in the file descriptor set.

select system call has a limitation that it can only monitor number of file descriptors less than FD_SETSIZE.

3.0 Example: Flight time record and query system

The flight time record and query is a client-server system which keeps track of arrival and departure times of flights at an airport. The server keeps a record of flight times in its database. The clients send requests for store and query the time of flights.

3.1 The Server

The server code is as follows.

/* 
 *           flight-time-server.c: record and provide time of a
 *                                 flight from the airport
 *
 */

#include <sys/types.h>
#include <sys/socket.h>
#include <netinet/in.h>
#include <arpa/inet.h>
#include <netdb.h>
#include <stdio.h>
#include <string.h>
#include <stdlib.h>
#include <errno.h>
#include <syslog.h>
#include <unistd.h>
#include <stdbool.h>
#include <sys/select.h>
#include <ctype.h>
#include <stdint.h>
#include <time.h>

#define FLIGHT_NUM_SIZE            15

#define SERVER_PORT                "4358"
#define STORE_FLIGHT               1
#define FLIGHT_TIME_STORED         2
#define FLIGHT_TIME                3
#define FLIGHT_TIME_RESULT         4
#define FLIGHT_NOT_FOUND           5
#define ERROR_IN_INPUT             9

#define BACKLOG                   10

void error (char *msg);

struct message {
    int32_t message_id;
    char flight_no [FLIGHT_NUM_SIZE + 1];
    char departure [1 + 1]; // 'D': departure, 'A': arrival
    char date [10 + 1]; // dd/mm/yyyy
    char time [5 + 1];   // hh:mm
};

struct tnode {
    char *flight_no;
    bool departure; // true: departure, false: arrival
    time_t flight_time;
    struct tnode *left;
    struct tnode *right;
};

struct message recv_message, send_message;

struct tnode *add_to_tree (struct tnode *p, char *flight_no, bool departure, time_t flight_time);
struct tnode *find_flight_rec (struct tnode *p, char *flight_no);
void print_tree (struct tnode *p);
void trim (char *dest, char *src); 
void error (char *msg);

int main (int argc, char **argv)
{
    const char * const ident = "flight-time-server";

    openlog (ident, LOG_CONS | LOG_PID | LOG_PERROR, LOG_USER);
    syslog (LOG_USER | LOG_INFO, "%s", "Hello world!");
    
    struct addrinfo hints;
    memset(&hints, 0, sizeof (struct addrinfo));
    hints.ai_family = AF_UNSPEC;    /* allow IPv4 or IPv6 */
    hints.ai_socktype = SOCK_STREAM; /* Stream socket */
    hints.ai_flags = AI_PASSIVE;    /* for wildcard IP address */

    struct addrinfo *result;
    int s; 
    if ((s = getaddrinfo (NULL, SERVER_PORT, &hints, &result)) != 0) {
        fprintf (stderr, "getaddrinfo: %s\n", gai_strerror (s));
        exit (EXIT_FAILURE);
    }

    /* Scan through the list of address structures returned by 
       getaddrinfo. Stop when the the socket and bind calls are successful. */

    int listener, optval = 1;
    socklen_t length;
    struct addrinfo *rptr;
    for (rptr = result; rptr != NULL; rptr = rptr -> ai_next) {
        listener = socket (rptr -> ai_family, rptr -> ai_socktype,
                       rptr -> ai_protocol);
        if (listener == -1)
            continue;

        if (setsockopt (listener, SOL_SOCKET, SO_REUSEADDR, &optval, sizeof (int)) == -1)
            error("setsockopt");

        if (bind (listener, rptr -> ai_addr, rptr -> ai_addrlen) == 0)  // Success
            break;

        if (close (listener) == -1)
            error ("close");
    }

    if (rptr == NULL) {               // Not successful with any address
        fprintf(stderr, "Not able to bind\n");
        exit (EXIT_FAILURE);
    }

    freeaddrinfo (result);

    // Mark socket for accepting incoming connections using accept
    if (listen (listener, BACKLOG) == -1)
        error ("listen");

    socklen_t addrlen;
    fd_set fds, readfds;
    FD_ZERO (&fds);
    FD_SET (listener, &fds);
    int fdmax = listener;
    struct sockaddr_storage client_saddr;
    char str [INET6_ADDRSTRLEN];
    struct sockaddr_in  *ptr;
    struct sockaddr_in6  *ptr1;
    struct tnode *root = NULL;

    while (1) {
        readfds = fds;
        // monitor readfds for readiness for reading
        if (select (fdmax + 1, &readfds, NULL, NULL, NULL) == -1)
            error ("select");
        
        // Some sockets are ready. Examine readfds
        for (int fd = 0; fd < (fdmax + 1); fd++) {
            if (FD_ISSET (fd, &readfds)) {  // fd is ready for reading 
                if (fd == listener) {  // request for new connection
                    addrlen = sizeof (struct sockaddr_storage);
                    int fd_new;
                    if ((fd_new = accept (listener, (struct sockaddr *) &client_saddr, &addrlen)) == -1)
                        error ("accept");
                    FD_SET (fd_new, &fds); 
                    if (fd_new > fdmax) 
                        fdmax = fd_new;

                    // print IP address of the new client
                    if (client_saddr.ss_family == AF_INET) {
                        ptr = (struct sockaddr_in *) &client_saddr;
                        inet_ntop (AF_INET, &(ptr -> sin_addr), str, sizeof (str));
                    }
                    else if (client_saddr.ss_family == AF_INET6) {
                        ptr1 = (struct sockaddr_in6 *) &client_saddr;
	                inet_ntop (AF_INET6, &(ptr1 -> sin6_addr), str, sizeof (str));
                    }
                    else
                    {
                        ptr = NULL;
                        fprintf (stderr, "Address family is neither AF_INET nor AF_INET6\n");
                    }
                    if (ptr) 
                        syslog (LOG_USER | LOG_INFO, "%s %s", "Connection from client", str);
                
                }
                else  // data from an existing connection, receive it
                {
                    memset (&recv_message, '\0', sizeof (struct message));
                    ssize_t numbytes = recv (fd, &recv_message, sizeof (struct message), 0);
   
                    if (numbytes == -1)
                        error ("recv");
                    else if (numbytes == 0) {
                        // connection cloded by client
                        fprintf (stderr, "Socket %d closed by client\n", fd);
                        if (close (fd) == -1)
                            error ("close");
                        FD_CLR (fd, &fds);
                    }
                    else 
                    {
                        // data from client
                        bool valid;
                        char temp_buf [FLIGHT_NUM_SIZE + 1];
                        
                        switch (ntohl (recv_message.message_id)) {
                            case STORE_FLIGHT:
                                   valid = true;
                                   // validate flight number
                                   if (recv_message.flight_no [FLIGHT_NUM_SIZE])
                                       recv_message.flight_no [FLIGHT_NUM_SIZE] = '\0';
                                   if (strlen (recv_message.flight_no) < 3)
                                       valid = false;
                                   trim (temp_buf, recv_message.flight_no);
                                   strcpy (recv_message.flight_no, temp_buf);
                                   bool departure;
                                   if (toupper (recv_message.departure [0]) == 'D')
                                       departure = true;
                                   else if (toupper (recv_message.departure [0]) == 'A')
                                       departure = false; 
                                   else
                                       valid = false;

                                   char delim [] = "/";
                                   char *mday, *month, *year, *saveptr;
                                   mday = month = year = NULL;
                                   mday = strtok_r (recv_message.date, delim, &saveptr);
                                   if (mday)
                                       month = strtok_r (NULL, delim, &saveptr);
                                   else 
                                       valid = false;
                                   if (month)
                                       year = strtok_r (NULL, delim, &saveptr);
                                   else 
                                       valid = false;
                                   if (!year)
                                       valid = false;
                                   char *hrs, *min;
                                   // get time
                                   if (recv_message.time [5])
                                       recv_message.time [5] = '\0';
                                   delim [0] = ':';
                                   hrs = min = NULL;
                                   hrs = strtok_r (recv_message.time, delim, &saveptr);
                                   if (hrs) 
                                       min = strtok_r (NULL, delim, &saveptr);
                                   if (!hrs || !min)
                                       valid = false;

                                   time_t ts;

                                   if (valid) {
                                       struct tm tm;

                                       tm.tm_sec = 0;
                                       sscanf (min, "%d", &tm.tm_min);
                                       sscanf (hrs, "%d", &tm.tm_hour);
                                       sscanf (mday, "%d", &tm.tm_mday);
                                       sscanf (month, "%d", &tm.tm_mon);
                                       (tm.tm_mon)--;
                                       sscanf (year, "%d", &tm.tm_year);
                                       tm.tm_year -= 1900;
                                       tm.tm_isdst = -1;

                                       if ((ts = mktime (&tm)) == (time_t) -1)
                                           valid = false;
                                  
                                       time_t now;

                                       if ((now = time (NULL)) == (time_t) -1)
                                           error ("time");

                                       if (ts < now)
                                           valid = false;
                                   }

                                   if (!valid) {
                                       // send error message to client
                                       send_message.message_id = htonl (ERROR_IN_INPUT);
                                       size_t msg_len = sizeof (long);
                                       if (send (fd, &send_message, msg_len, 0) == -1)
                                           error ("send");
                                   }
                                   else
                                   {
                                       // add flight data to tree
                                       root = add_to_tree (root, recv_message.flight_no, departure, ts);
                                       // send confirmation to client
                                       send_message.message_id = htonl (FLIGHT_TIME_STORED);
                                       strcpy (send_message.flight_no, recv_message.flight_no);
                                       strcpy (send_message.departure, (departure) ? "D" : "A");
                                       struct tm *tms;  
                                       if ((tms = localtime (&ts)) == NULL)  
                                            perror ("localtime");                    
                                       sprintf (send_message.date, "%02d/%02d/%d", tms -> tm_mday, 
                                            tms -> tm_mon + 1, tms -> tm_year + 1900);
                                       sprintf (send_message.time, "%02d:%02d", tms -> tm_hour,
                                            tms -> tm_min);
                                       size_t msg_len = sizeof (struct message);
                                       if (send (fd, &send_message, msg_len, 0) == -1)
                                           error ("send");
                                   }
                                   break;
                            case FLIGHT_TIME:
                                   valid = true;
                                   // validate flight number
                                   if (recv_message.flight_no [FLIGHT_NUM_SIZE])
                                       recv_message.flight_no [FLIGHT_NUM_SIZE] = '\0';
                                   if (strlen (recv_message.flight_no) < 3)
                                       valid = false;
                                   if (!valid) {
                                       // send error message to client
                                       send_message.message_id = htonl (ERROR_IN_INPUT);
                                       size_t msg_len = sizeof (long);
                                       if (send (fd, &send_message, msg_len, 0) == -1)
                                           error ("send");
                                       break;
                                   }
                                   char temp_buf [FLIGHT_NUM_SIZE + 1];
                                   trim (temp_buf, recv_message.flight_no);
                                   strcpy (recv_message.flight_no, temp_buf);
                                   struct tnode *ptr;
                                   ptr = find_flight_rec (root, recv_message.flight_no);
                                   if (!ptr) {
                                       memset (&send_message, '\0', sizeof (struct message));
                                       send_message.message_id = htonl (FLIGHT_NOT_FOUND);
                                       strcpy (send_message.flight_no, recv_message.flight_no);
                                       size_t msg_len = sizeof (struct message);
                                       if (send (fd, &send_message, msg_len, 0) == -1)
                                           error ("send");
                                       break;
                                   }
                                   send_message.message_id = htonl (FLIGHT_TIME_RESULT);
                                   strcpy (send_message.flight_no, recv_message.flight_no);
                                   strcpy (send_message.departure, (ptr -> departure) ? "D" : "A");
                                   struct tm *tms;  
                                   if ((tms = localtime (&(ptr -> flight_time))) == NULL)  
                                        perror ("localtime");                    
                                   sprintf (send_message.date, "%02d/%02d/%d", tms -> tm_mday, 
                                            tms -> tm_mon + 1, tms -> tm_year + 1900);
                                   sprintf (send_message.time, "%02d:%02d", tms -> tm_hour,
                                            tms -> tm_min);
                                   size_t msg_len = sizeof (struct message);
                                   if (send (fd, &send_message, msg_len, 0) == -1)
                                       error ("send");
                                   break;

                        }

                    }
                }
            } // if (fd == ...
        } // for
    } // while (1)

    exit (EXIT_SUCCESS);
} // main

// record the flight departure / arrival time    
struct tnode *add_to_tree (struct tnode *p, char *flight_no, bool departure, time_t flight_time)
{
    int res;

    if (p == NULL) {  // new entry
        if ((p = (struct tnode *) malloc (sizeof (struct tnode))) == NULL)
            error ("malloc");
        p -> flight_no = strdup (flight_no);
        p -> departure = departure;
        p -> flight_time = flight_time;
        p -> left = p -> right = NULL;
    }
    else if ((res = strcmp (flight_no, p -> flight_no)) == 0) { // entry exists
        p -> departure = departure;
        p -> flight_time = flight_time;
    }
    else if (res < 0) // less than flight_no for this node, put in left subtree
        p -> left = add_to_tree (p -> left, flight_no, departure, flight_time);
    else   // greater than flight_no for this node, put in right subtree
        p -> right = add_to_tree (p -> right, flight_no, departure, flight_time);
    return p;
}

// find node for the flight for which departure or arrival time is queried
struct tnode *find_flight_rec (struct tnode *p, char *flight_no)
{
    int res;

    if (!p) 
        return p;
    res = strcmp (flight_no, p -> flight_no);
    
    if (!res)
        return p;

    if (res < 0)
        return find_flight_rec (p -> left, flight_no);
    else 
        return find_flight_rec (p -> right, flight_no);
}

// print_tree: print the tree (in-order traversal)
void print_tree (struct tnode *p)
{
    if (p != NULL) {
        print_tree (p -> left);
        printf ("%s: %d %s\n\n", p -> flight_no, (int) p -> departure, ctime (&(p -> flight_time)));
        print_tree (p -> right);
    }
}

void error (char *msg)
{
    perror (msg);
    exit (1);
}

// trim: leading and trailing whitespace of string
void trim (char *dest, char *src)
{
    if (!src || !dest)
       return;

    int len = strlen (src);

    if (!len) {
        *dest = '\0';
        return;
    }
    char *ptr = src + len - 1;

    // remove trailing whitespace
    while (ptr > src) {
        if (!isspace (*ptr))
            break;
        ptr--;
    }

    ptr++;

    char *q;
    // remove leading whitespace
    for (q = src; (q < ptr && isspace (*q)); q++)
        ;

    while (q < ptr)
        *dest++ = *q++;

    *dest = '\0';
}

3.2 The client

The client code is as given below.

/* 
 *       flight-time-client.c : get flight time from the server
 *
 */

#include <sys/types.h>
#include <sys/socket.h>
#include <netinet/in.h>
#include <arpa/inet.h>
#include <netdb.h>
#include <stdio.h>
#include <string.h>
#include <stdlib.h>
#include <errno.h>
#include <unistd.h>
#include <ctype.h>
#include <stdint.h>
#include <time.h>

#define FLIGHT_NUM_SIZE            15

#define SERVER_PORT                "4358"
#define STORE_FLIGHT               1
#define FLIGHT_TIME_STORED         2
#define FLIGHT_TIME                3
#define FLIGHT_TIME_RESULT         4
#define FLIGHT_NOT_FOUND           5
#define ERROR_IN_INPUT             9
#define QUIT                       0

void error (char *msg);

struct message {
    int32_t message_id;
    char flight_no [FLIGHT_NUM_SIZE + 1];
    char departure [1 + 1]; // 'D': departure, 'A': arrival
    char date [10 + 1]; // dd/mm/yyyy
    char time [5 + 1];   // hh:mm
};

struct message message;

int get_input (void);
void error (char *msg);

int main (int argc, char **argv)
{
    if (argc != 2) {
        fprintf (stderr, "Usage: client hostname\n");
        exit (EXIT_FAILURE);
    }

    struct addrinfo hints;
    memset(&hints, 0, sizeof (struct addrinfo));
    hints.ai_family = AF_UNSPEC;
    hints.ai_socktype = SOCK_STREAM;

    struct addrinfo *result;
    int s; 
    if ((s = getaddrinfo (argv [1], SERVER_PORT, &hints, &result)) != 0) {
        fprintf (stderr, "getaddrinfo: %s\n", gai_strerror (s));
        exit (EXIT_FAILURE);
    }

    /* Scan through the list of address structures returned by 
       getaddrinfo. Stop when the the socket and connect calls are successful. */

    int sock_fd;
    socklen_t length;
    struct addrinfo *rptr;
    for (rptr = result; rptr != NULL; rptr = rptr -> ai_next) {
        sock_fd = socket (rptr -> ai_family, rptr -> ai_socktype,
                       rptr -> ai_protocol);
        if (sock_fd == -1)
            continue;

        if (connect (sock_fd, rptr -> ai_addr, rptr -> ai_addrlen) == -1) {
            if (close (sock_fd) == -1)
                error ("close");
            continue;
        }
        
        break;
    }

    if (rptr == NULL) {               // Not successful with any address
        fprintf(stderr, "Not able to connect\n");
        exit (EXIT_FAILURE);
    }

    freeaddrinfo (result);

    int option;

    while (1) {
         option = get_input ();
         if (option == QUIT)
             break;

         // send request to server
         if (send (sock_fd, &message, sizeof (struct message), MSG_NOSIGNAL) == -1)
             error ("send");

         // receive response from server
         if (recv (sock_fd, &message, sizeof (struct message), 0) == -1)
             error ("recv");

         // process server response 
         switch (ntohl (message.message_id)) {
             case FLIGHT_TIME_STORED: 
             case FLIGHT_TIME_RESULT: printf ("\nResponse: \n\n");
                    printf ("\t%s: %s %s %s\n\n", message.flight_no, message.departure, 
                                              message.date, message.time);
                     break;
             case FLIGHT_NOT_FOUND: printf ("\nFlight not found\n\n");
                     break;
             case ERROR_IN_INPUT: printf ("\nError in input\n\n");
                     break;
             default: printf ("\nUnrecongnized message from server\n\n");
         }
    }

    exit (EXIT_SUCCESS);
}

char inbuf [512];

int get_input (void)
{
    int option;

    while (1) {
        printf ("Flight Info\n\n");
        printf ("\tFlight time query\t1\n");
        printf ("\tStore flight time\t2\n");
        printf ("\tQuit\t\t0\n\n");
        printf ("Your option: ");
        if (fgets (inbuf, sizeof (inbuf),  stdin) == NULL)
            error ("fgets");
        sscanf (inbuf, "%d", &option);

        int len;

        switch (option) {

            case 1: message.message_id = htonl (FLIGHT_TIME);
                    printf ("Flight no: ");
                    if (fgets (inbuf, sizeof (inbuf),  stdin) == NULL)
                        error ("fgets");
                    len = strlen (inbuf);
                    if (inbuf [len - 1] == '\n')
                        inbuf [len - 1] = '\0';
                    strcpy (message.flight_no, inbuf);
                    break;

            case 2: message.message_id = htonl (STORE_FLIGHT);
                    printf ("Flight no: ");
                    if (fgets (inbuf, sizeof (inbuf),  stdin) == NULL)
                        error ("fgets");
                    len = strlen (inbuf);
                    if (inbuf [len - 1] == '\n')
                        inbuf [len - 1] = '\0';
                    strcpy (message.flight_no, inbuf);

                    while (1) {
                        printf ("A/D: ");
                        if (fgets (inbuf, sizeof (inbuf),  stdin) == NULL)
                            error ("fgets");
                        message.departure [0] = toupper (inbuf [0]);
                        message.departure [1] = '\0';
                        if ((message.departure [0] == 'A') || (message.departure [0] == 'D'))
                            break;
                        printf ("Error in input, valid values are A and D\n");
                    }
                    
                    printf ("date (dd/mm/yyyy): ");
                    if (fgets (inbuf, sizeof (inbuf),  stdin) == NULL)
                        error ("fgets");
                    strncpy (message.date, inbuf, 10);
                    message.date [10] = '\0';
                    printf ("time (hh:mm): ");
                    if (fgets (inbuf, sizeof (inbuf),  stdin) == NULL)
                        error ("fgets");
                    strncpy (message.time, inbuf, 5);
                    message.time [5] = '\0';
                    break;

            case 0:
                    break;

            default: printf ("Illegal option, try again\n\n");
                     continue;

        }

        return option;
    }
}

void error (char *msg)
{
    perror (msg);
    exit (1);
}
[showad4]

3.3 Running the server and clients

We can compile and run the server and client programs. We will have one instance of the server running and multiple clients will run and connect with the server.

$ gcc flight-time-server.c -o flight-time-server
$ ./flight-time-server 192.168.8.100
flight-time-server[9025]: Hello world!
flight-time-server[9025]: Connection from client 192.168.8.100
Socket 5 closed by client

Running a client,

$ gcc flight-time-client.c -o flight-time-client
$ ./flight-time-client 192.168.8.100
Flight Info

	Flight time query	1
	Store flight time	2
	Quit		0

Your option: 1
Flight no: AS201

Response: 

	AS201: D 30/01/2019 06:30

Clients and servers