SysV Shared Memory Under Unix
A tutorial
Shared memory is the fastest method of inter-process communication, IPC, available to a Unix process. It allows multiple processes to use the same data structures and regulate access to them based on rules the processes define themselves. System V shared memory is faster than shared memory mapping on some Unix operating systems and is still more prevalent than POSIX shared memory. In order to demonstrate how to use shared memory we develop a simple client server application that takes advantage of SysV shared memory and semaphores.
The Problem
We have a program that carries out a long running calculation on a shared data structure at regular intervals. We have a number of clients that use the results of this calculation. We need to protect the calculation so that no client sees the data in an intermediate state.
For our example program we will simulate the long running calculation by using a simple square root of an integer value and a call to sleep to delay execution. The sleep will occur between the update of the integer value and the calculation of its square root.
When using shared memory the use of semaphores to protect user processes from accessing data in an intermediate state is almost always necessary. It adds some small amount of complexity to the example, but is necessary to understand how the whole process works.
The Solution
The square root server
Several include files are necessary for this demonstration. In addition to the standard include files you will find in many C programs we will also require, sys/ipc.h, sys/shm.h, and sys/sem.h to declare the system V IPC functions and structures we will be using.
#include <unistd.h>
#include <math.h>
#include <errno.h>
#include <stdlib.h>
#include <sys/shm.h>
#include <sys/sem.h>
#include <sys/ipc.h>
We need to declare a data type that we will be using in our shared memory. For our purposes we need an integer variable and a float to hold its square root.
struct dummy
{
int i;
float f;
};
The main function performs the following tasks
- Determine the key values for our semaphore and shared memory segment using ftok.
- Get a descriptor for the shared memory segment.
- Attach the shared memory to our address space using shmat.
- Get a descriptor for the semaphore we use to protect the shared memory.
- Set the semaphore to one so it will act as a mutex.
- Begin the main loop.
int
main(int argc, char *argv[])
{
int mem_d;
int sem_d;
struct dummy *dum;
struct sembuf sops;
key_t sem_key, shm_key;
struct sigaction act;
The function ftok is used to convert a pathname into a number that may be used to identify Sys V IPC objects. I'm passing in my home directory and a random number to get two keys. These keys will be used for the shared memory segment and the semaphore in the API calls.
/* ftok returns a key based on a path and id
number for use in
sysv ipc functions. */
shm_key = ftok("/home/kpturvey", 23);
sem_key = ftok("/home/kpturvey", 24);
The next step involves getting the shared memory descriptor. The function, shmget, serves this purpose. Note that we are allowing anyone access to this shared memory, read and write access, you should probably be less permissive in your program.
/* shmget returns an integer memory identifier
associate with the
shared memory segment. */
mem_d = shmget(shm_key, sizeof(struct dummy),
IPC_CREAT | 0666);
if (mem_d < 0 )
{
perror("shmget"); exit(EXIT_FAILURE);
}
The function shmget returns a descriptor for a shared memory segment, but in order to use this segment it must be attached to the process address space used by the running process. The function, shmat, serves this purpose. You should call it specifying zero for the address to attach to. We want to be able to read and write the shared memory but you can attach it read only, if this serves your purposes.
/* We now attach this memory segment to a system
selected address. */
dum = shmat(mem_d, 0, 0);
if (-1 == (int) dum)
{
perror("shmat");
exit (EXIT_FAILURE);
}
We need to get a descriptor for the semaphore we will use to protect the memory. The function, semget will give us this descriptor. We specify that we are only need one semaphore in the set and that it should be created if it doesn't exist. The 0666 indicates that this semaphore will be world readable and writable. This probably isn't what you want for a real application. Set the permissions to something specific to your application.
/* Lets get a semaphore to protect this region. */
sem_d = semget(sem_key, 1, IPC_CREAT | 0666);
The next few lines will initialize our shared memory's integer value to zero. In addition we initialize the semaphore to have a value of one to act as a mutex.
/* initialize our shared variable. I don't think
this is really necessary. */
dum->i = 0;
/* Set the semaphore to one to allow a single
process to access dum at a time. */
sops.sem_num = 0;
sops.sem_op = 1;
sops.sem_flg = 0;
if (semop(sem_d, &sops, 1) < 0)
{
perror("semop");
exit(EXIT_FAILURE);
}
Now we have the main loop for our server program. The server simply gets the semaphore protecting our data, increments the integer, calculates and stores its square root, and releases the semaphore. I have added a sleep to this loop to simulate a long running calculation. The loop is repeated again after waiting one more second. This loop shows how shared memory would normally be used by a real application.
/* Loop through the integers calculating their
square root. We use the semaphore to make
sure that the increment and square root
operations seem atomic. The client should
never see an integer and root that do not
match. */
while (1)
{
printf("Loop\n");
/* Get the semaphore */
sops.sem_num = 0;
sops.sem_op = -1;
sops.sem_flg = 0;
if (semop(sem_d, &sops, 1) < 0)
{
perror("semop");
exit(EXIT_FAILURE);
}
/* Do the work. We sleep in the middle
to show that this could be a longer
running calculation. */
dum->i += 1;
sleep(1);
dum->f = sqrt(dum->i);
/* Post to the semaphore. */
sops.sem_num = 0;
sops.sem_op = 1;
sops.sem_flg = 0;
if (semop(sem_d, &sops, 1) < 0)
{
perror("semop");
exit(EXIT_FAILURE);
}
sleep(1);
}
The Square Root Client
The client application is very similar to the server application. The only difference is that it does not need to initialize the semaphore or the shared memory. Our client simply looks at the shared memory (after obtaining the semaphore) and displays what it contains. If we didn't protect the shared memory using semaphores there would be no way to guarantee that the float value displayed was the root of the integer value displayed. The program could catch the shared memory in an intermediate step of the server calculation.
Because the code is so similar to the server, I will now show the complete source code for the client.
#include <unistd.h>
#include <math.h>
#include <errno.h>
#include <stdlib.h>
#include <sys/shm.h>
#include <sys/sem.h>
#include <sys/ipc.h>
struct dummy {
int i;
float f;
};
int main(int argc, char *argv[])
{
int mem_d;
int sem_d;
struct dummy *dum;
struct sembuf sops;
key_t sem_key, shm_key;
/* ftok returns a key based on a path and
id number for use in
sysv ipc functions. */
shm_key = ftok("/home/kpturvey", 23);
sem_key = ftok("/home/kpturvey", 24);
/* shmget returns an integer memory
identifier associate
with the shared memory segment. */
mem_d = shmget(shm_key, sizeof(struct dummy),
IPC_CREAT | 0666);
if (mem_d < 0 )
{
perror("shmget");
exit(EXIT_FAILURE);
}
/* We now attach this memory segment to a
system selected address. */
dum = shmat(mem_d, 0, 0);
if (SHM_FAILED == dum)
{
perror("shmat");
exit (EXIT_FAILURE);
}
/* Lets get a semaphore to protect this region. */
sem_d = semget(sem_key, 1, IPC_CREAT | 0666);
/* Every second grab the semaphore and display
the values in shared memory. The integer,
root pair should always match.
The semaphore is protecting the update. */
while (1)
{
/* Get the semaphore */
sops.sem_num = 0;
sops.sem_op = -1;
sops.sem_flg = 0;
if (semop(sem_d, &sops, 1) < 0)
{
perror("semop");
exit(EXIT_FAILURE);
}
/* Print out the values in shared memory.
They should always match. */
printf("%d --> %f\n", dum->i, dum->f);
/* Post to the semaphore. */
sops.sem_num = 0;
sops.sem_op = 1;
sops.sem_flg = 0;
if (semop(sem_d, &sops, 1) < 0)
{
perror("semop");
exit(EXIT_FAILURE);
}
sleep(1);
}
}
Kernel Persistence
System V shared memory and semaphores have what is called "Kernel Persistence". The term means that these objects exist for as long as the kernel is running. Our programs did nothing to clean up after they were finished with the shared memory segment and semaphore so they will still exist after you have killed the programs. A user can see all of the IPC objects by using the program ipcs.
When I list out the ipc objects that are used on my system after running this program this is what I see:
T ID KEY MODE OWNER GROUP
Message Queues:
q 0 0x3c1c0466 -Rrw--w--w- root root
q 1 0x3e1c0466 --rw-r--r-- root root
Shared Memory:
m 10000 0x1e7466ac --rw-r----- oracle dba
m 1 0x4e0c0002 --rw-rw-rw- root root
m 2 0x412002d6 --rw-rw-rw- root root
m 3 0x0c6629c9 --rw-r----- root root
m 4 0x06347849 --rw-rw-rw- root root
m 13205 0x9bfb04fc --rw-r----- oracle dba
m 9635 0x1710736c --rw-rw-rw- kpturvey users
Semaphores:
s 0 0x411c061a --ra-ra-ra- root root
s 1 0x4e0c0002 --ra-ra-ra- root root
s 2 0x412002d6 --ra-ra-ra- root root
s 3 0x00446f6e --ra-r--r-- root root
s 4 0x00446f6d --ra-r--r-- root root
s 2053 0x01090522 --ra-r--r-- root root
s45948974 0x1810736c --ra-ra-ra- kpturvey users
Your output should be similar. It is easy to see which of these
objects are mine, I, kpturvey, am listed as the owner. To get rid
of the shared memory segment, now that I'm done with it, I use the
ipcrm utility:
ipcrm -m 9635
We also need to delete the semaphore:
ipcrm -s 45948974
Closing
Shared memory is the fastest form of IPC available to the Unix applications programmer. On many systems the System V API provides a faster implementation of shared memory than the POSIX API. It is my hope that this tutorial will help you to use this form of IPC with few problems.