Interprocess communication using POSIX Shared Memory in Linux

1.0 Shared Memory

Shared memory, message queues and semaphores form the suite of interprocess communication (IPC) mechanisms available under Linux and other Unix-like systems. In case of shared memory, the system provides a shared memory segment which the calling process can map to its address space. After that, it behaves just like any other part of the process's address space.

POSIX Shared Memory

If we look at the other IPC mechanisms like message queue or the older mechanisms like the pipe or the fifo, the work required for passing message involves, first copying the message from the address space of the first process to the kernel space via a send-like system call and, then, copying the message from the kernel space to the address space of the second process during a receive-like call. In the case of shared memory, the first process writes data in the shared memory segment and the data becomes available to the second process immediately. This makes shared memory faster than other mechanisms and is, in fact, the fastest way of passing data between two processes on the same host system.

2.0 System V and POSIX Shared Memory Calls

For each of the IPC mechanisms, there are two sets of calls, the traditional System V calls and the newer POSIX calls. In this post, we will look at the POSIX shared memory calls. Example programs for server and client processes that communicate via POSIX shared memory are given near the end of this post.

3.0 POSIX Shared Memory Calls

The POSIX shared memory calls seem to be based on the UNIX philosophy that if you do Input/Output operations on an object, that object has to be a file. So, since we do read and write to a POSIX shared memory object, the latter is to be treated as a file. A POSIX shared memory object is a memory-mapped file. POSIX shared memory files are provided from a tmpfs filesystem mounted at /dev/shm. The individual shared memory files are created using the shm_open system call under /dev/shm. There are just two specialized POSIX shared memory system calls, shm_open and shm_unlink, which are analogous to open and unlink system calls for files. Other operations on POSIX shared memory are done using the ftruncate, mmap and munmap system calls for files. A program using POSIX shared memory calls needs to be linked with -lrt.

3.1 shm_open

#include <sys/mman.h> #include <sys/stat.h> #include <fcntl.h> int shm_open (const char *name, int oflag, mode_t mode);

shm_open is like the open system call for files. It opens a POSIX shared memory object and makes it available to the calling process via the returned file descriptor. The first parameter, name, is the name of the shared memory object and is of the form /somename, that is, it is a null-terminated string of a maximum of NAME_MAX characters, and, its first character is a slash and none of the other characters can be a slash. oflag is a bit mask constructed by OR-ing either O_RDONLY or O_RDWR with one or more of the following flags. O_CREAT creates the shared memory object if it does not exist. And, if it creates the object, the last nine bits of the third parameter, mode are taken for permissions except that the bits set in the file mode creation mask are cleared for the object. Also, if the shared memory object is created, the owner and group ids of the object are set the corresponding effective ids of the calling process. A newly created shared memory object is of size zero bytes. It can be made of the desired size by using the ftruncate system call. If O_EXCL flag is used together with the O_CREAT flag, and the shared memory object for the name already exists, shm_open fails and errno is set to EEXIST. There is another flag, O_TRUNC, which if specified, truncates the shared memory object, if it already exists, to size zero bytes. On success, shm_open returns the file descriptor for the shared memory object. If shm_open fails, it returns -1 and errno is set to the cause of the error.

3.2 shm_unlink

#include <sys/mman.h> #include <sys/stat.h> #include <fcntl.h> int shm_unlink (const char *name);

shm_unlink removes the previously created POSIX shared memory object. The name is the name of the shared memory object as described under shm_open, above.

4.0 Other System Calls used in POSIX Shared Memory Operations

4.1 ftruncate

#include <unistd.h> #include <sys/types.h> int ftruncate (int fd, off_t length);

The ftruncate system call makes the object referred to by the file descriptor, fd, of size length bytes. When a POSIX shared memory is created, it is of size zero bytes. Using ftruncate, we can make the POSIX shared memory object of size length bytes. ftruncate returns zero on success. In case of error, ftruncate returns -1 and errno is set to the cause of the error.

4.2 mmap

#include <sys/mman.h> void *mmap (void *addr, size_t length, int prot, int flags, int fd, off_t offset);

With the mmap system call, we can map a POSIX shared memory object to the calling process's virtual address space. addr specifies the address at which it should be mapped. In most cases, we do not care at what address mapping is done and a value of NULL for addr should suffice. length is the length of shared memory object that should be mapped. To keep things simple, we will map the whole object and length for us will be the length of the shared memory object. prot can have the values, PROT_EXEC, PROT_READ, PROT_WRITE and PROT_NONE. PROT_EXEC means that the mapped pages may be executed and PROT_NONE means that the mapped pages may not be accessed. These two values do not make sense for a shared memory object. So we will use PROT_READ | PROT_WRITE value for prot. There are many flags but the only one meaningful for shared memory is MAP_SHARED, which means that the updates to the mapped shared memory are visible to all other processes immediately. fd is, of course, the file descriptor for the shared memory received from an earlier shm_open call. offset is the location in the shared memory object at which the mapping starts; we will use the value zero for offset and map the shared memory object starting right from the beginning. On success, mmap returns the pointer to the location where the shared memory object has been mapped. In case of error, MAP_FAILED, which is, (void *) -1, is returned and errno is set to the cause of the error.

4.3 munmap

#include <sys/mman.h> int munmap (void *addr, size_t length);

munmap unmapps the shared memory object at location pointed by addr and having size, length. On success, munmap returns 0. In case of error, munmap returns -1 and errno is set to the cause of the error.

5.0 An example: System Logger

The System Logger process creates a POSIX shared memory object and maps it to its address space. Clients also map the shared memory object to their address spaces. When a client wants to log a message, it creates a string in the format <client pid><timestamp><message> and writes the string in the shared memory object. The logger reads strings from the shared memory object, one by one, and writes them in a log file in the chronological order.

system Logger

The server code is,

/* * * logger.c: Write strings in POSIX shared memory to file * (Server process) */ #include <stdio.h> #include <stdlib.h> #include <sys/types.h> #include <sys/stat.h> #include <fcntl.h> #include <string.h> #include <unistd.h> #include <semaphore.h> #include <sys/mman.h> // Buffer data structures #define MAX_BUFFERS 10 #define LOGFILE "/tmp/example.log" #define SEM_MUTEX_NAME "/sem-mutex" #define SEM_BUFFER_COUNT_NAME "/sem-buffer-count" #define SEM_SPOOL_SIGNAL_NAME "/sem-spool-signal" #define SHARED_MEM_NAME "/posix-shared-mem-example" struct shared_memory { char buf [MAX_BUFFERS] [256]; int buffer_index; int buffer_print_index; }; void error (char *msg); int main (int argc, char **argv) { struct shared_memory *shared_mem_ptr; sem_t *mutex_sem, *buffer_count_sem, *spool_signal_sem; int fd_shm, fd_log; char mybuf [256]; // Open log file if ((fd_log = open (LOGFILE, O_CREAT | O_WRONLY | O_APPEND | O_SYNC, 0666)) == -1) error ("fopen"); // mutual exclusion semaphore, mutex_sem with an initial value 0. if ((mutex_sem = sem_open (SEM_MUTEX_NAME, O_CREAT, 0660, 0)) == SEM_FAILED) error ("sem_open"); // Get shared memory if ((fd_shm = shm_open (SHARED_MEM_NAME, O_RDWR | O_CREAT, 0660)) == -1) error ("shm_open"); if (ftruncate (fd_shm, sizeof (struct shared_memory)) == -1) error ("ftruncate"); if ((shared_mem_ptr = mmap (NULL, sizeof (struct shared_memory), PROT_READ | PROT_WRITE, MAP_SHARED, fd_shm, 0)) == MAP_FAILED) error ("mmap"); // Initialize the shared memory shared_mem_ptr -> buffer_index = shared_mem_ptr -> buffer_print_index = 0; // counting semaphore, indicating the number of available buffers. Initial value = MAX_BUFFERS if ((buffer_count_sem = sem_open (SEM_BUFFER_COUNT_NAME, O_CREAT, 0660, MAX_BUFFERS)) == SEM_FAILED) error ("sem_open"); // counting semaphore, indicating the number of strings to be printed. Initial value = 0 if ((spool_signal_sem = sem_open (SEM_SPOOL_SIGNAL_NAME, O_CREAT, 0660, 0)) == SEM_FAILED) error ("sem_open"); // Initialization complete; now we can set mutex semaphore as 1 to // indicate shared memory segment is available if (sem_post (mutex_sem) == -1) error ("sem_post: mutex_sem"); while (1) { // forever // Is there a string to print? P (spool_signal_sem); if (sem_wait (spool_signal_sem) == -1) error ("sem_wait: spool_signal_sem"); strcpy (mybuf, shared_mem_ptr -> buf [shared_mem_ptr -> buffer_print_index]); /* Since there is only one process (the logger) using the buffer_print_index, mutex semaphore is not necessary */ (shared_mem_ptr -> buffer_print_index)++; if (shared_mem_ptr -> buffer_print_index == MAX_BUFFERS) shared_mem_ptr -> buffer_print_index = 0; /* Contents of one buffer has been printed. One more buffer is available for use by producers. Release buffer: V (buffer_count_sem); */ if (sem_post (buffer_count_sem) == -1) error ("sem_post: buffer_count_sem"); // write the string to file if (write (fd_log, mybuf, strlen (mybuf)) != strlen (mybuf)) error ("write: logfile"); } } // Print system error and exit void error (char *msg) { perror (msg); exit (1); }

And the client code is,

/* * * client.c: Write strings for printing in POSIX shared memory object * */ #include <stdio.h> #include <stdlib.h> #include <sys/types.h> #include <sys/stat.h> #include <fcntl.h> #include <string.h> #include <time.h> #include <unistd.h> #include <semaphore.h> #include <sys/mman.h> // Buffer data structures #define MAX_BUFFERS 10 #define LOGFILE "/tmp/example.log" #define SEM_MUTEX_NAME "/sem-mutex" #define SEM_BUFFER_COUNT_NAME "/sem-buffer-count" #define SEM_SPOOL_SIGNAL_NAME "/sem-spool-signal" #define SHARED_MEM_NAME "/posix-shared-mem-example" struct shared_memory { char buf [MAX_BUFFERS] [256]; int buffer_index; int buffer_print_index; }; void error (char *msg); int main (int argc, char **argv) { struct shared_memory *shared_mem_ptr; sem_t *mutex_sem, *buffer_count_sem, *spool_signal_sem; int fd_shm; char mybuf [256]; // mutual exclusion semaphore, mutex_sem if ((mutex_sem = sem_open (SEM_MUTEX_NAME, 0, 0, 0)) == SEM_FAILED) error ("sem_open"); // Get shared memory if ((fd_shm = shm_open (SHARED_MEM_NAME, O_RDWR, 0)) == -1) error ("shm_open"); if ((shared_mem_ptr = mmap (NULL, sizeof (struct shared_memory), PROT_READ | PROT_WRITE, MAP_SHARED, fd_shm, 0)) == MAP_FAILED) error ("mmap"); // counting semaphore, indicating the number of available buffers. if ((buffer_count_sem = sem_open (SEM_BUFFER_COUNT_NAME, 0, 0, 0)) == SEM_FAILED) error ("sem_open"); // counting semaphore, indicating the number of strings to be printed. Initial value = 0 if ((spool_signal_sem = sem_open (SEM_SPOOL_SIGNAL_NAME, 0, 0, 0)) == SEM_FAILED) error ("sem_open"); char buf [200], *cp; printf ("Please type a message: "); while (fgets (buf, 198, stdin)) { // remove newline from string int length = strlen (buf); if (buf [length - 1] == '\n') buf [length - 1] = '\0'; // get a buffer: P (buffer_count_sem); if (sem_wait (buffer_count_sem) == -1) error ("sem_wait: buffer_count_sem"); /* There might be multiple producers. We must ensure that only one producer uses buffer_index at a time. */ // P (mutex_sem); if (sem_wait (mutex_sem) == -1) error ("sem_wait: mutex_sem"); // Critical section time_t now = time (NULL); cp = ctime (&now); int len = strlen (cp); if (*(cp + len -1) == '\n') *(cp + len -1) = '\0'; sprintf (shared_mem_ptr -> buf [shared_mem_ptr -> buffer_index], "%d: %s %s\n", getpid (), cp, buf); (shared_mem_ptr -> buffer_index)++; if (shared_mem_ptr -> buffer_index == MAX_BUFFERS) shared_mem_ptr -> buffer_index = 0; // Release mutex sem: V (mutex_sem) if (sem_post (mutex_sem) == -1) error ("sem_post: mutex_sem"); // Tell spooler that there is a string to print: V (spool_signal_sem); if (sem_post (spool_signal_sem) == -1) error ("sem_post: (spool_signal_sem"); printf ("Please type a message: "); } if (munmap (shared_mem_ptr, sizeof (struct shared_memory)) == -1) error ("munmap"); exit (0); } // Print system error and exit void error (char *msg) { perror (msg); exit (1); }

We can compile and run the server and a client as below.

$ gcc logger.c -o logger -lrt -lpthread $ gcc client.c -o client -lrt -lpthread $ ./logger & [1] 3561 $ ps PID TTY TIME CMD 2924 pts/1 00:00:00 bash 3561 pts/1 00:00:00 logger 3562 pts/1 00:00:00 ps $ ./client Please type a message: Hello Logger Please type a message: She sells seashells by the seashore. ...

After running the Logger and four clients for sometime, the contents of the log file were,

$ cat example.log 3563: Sun Apr 3 12:20:11 2016 Hello Logger 3598: Sun Apr 3 12:22:19 2016 Hello Logger 3598: Sun Apr 3 12:23:19 2016 It is a beautiful day! 3632: Sun Apr 3 12:23:48 2016 Hello Logger 3664: Sun Apr 3 12:24:29 2016 Hello Logger 3664: Sun Apr 3 12:27:12 2016 So foul and fair a day I have not seen. (William Shakespeare: Macbeth) 3664: Sun Apr 3 12:28:32 2016 He that lives upon hope will die fasting. (Benjamin Franklin) 3632: Sun Apr 3 12:29:34 2016 It was as helpful as throwing a drowning man both ends of a rope. (Arthur Baer) 3563: Sun Apr 3 12:31:02 2016 She sells seashells by the seashore. 3598: Sun Apr 3 12:32:28 2016 Not so happy, yet much happier. (William Shakespeare: Macbeth)

6.0 See Also

  1. POSIX Semaphores
  2. Interprocess communication using System V Shared Memory in Linux
Software: