Using Shared Memory with OSRM
If you have multiple applications accessing OSRM data then it does not make sense for each of those to have a separate copy of the data resident in memory. This is especially true if you’re using a relatively large map, in which case memory consumed by multiple processes might be enormous.
An alternative is to store the map data in shared memory, allowing multiple processes to access a single copy of the data. The official OSRM documentation for using shared memory can be found here. This post gives further details.
You need to configure your system first. There are both system-wide and user-specific changes.
Check on the default shared memory limits.
$ ipcs -lm ------ Shared Memory Limits -------- max number of segments = 4096 max seg size (kbytes) = 18014398509465599 max total shared memory (kbytes) = 18014398509481980 min seg size (bytes) = 1
Those numers are pretty enormous and will probably be sufficient. But if you are using a really big map then you might need to expand them. You can do this by setting the
shmmax kernel parameters. We’ll go the other way and actually reduce the allocations. Add something like the following to the end of
kernel.shmall = 1000000000 kernel.shmmax = 10240000000
shmmax parameter specifies the maximum amount of shared memory (in kB) that can be requested by a single process. The
shmall parameter sets the total amount of shared memory pages that can be used across the entire system. A page is simply a block of memory. You can check on the page size (in bytes) as follows:
$ getconf PAGE_SIZE 4096
Now reload the system settings.
sudo sysctl -p
ipcs again to check that the changes have been applied.
$ ipcs -lm ------ Shared Memory Limits -------- max number of segments = 4096 max seg size (kbytes) = 10000000 max total shared memory (kbytes) = 4000000000 min seg size (bytes) = 1
There are separate limits which apply to individual users.
$ ulimit -a | grep max max locked memory (kbytes, -l) 16384 max memory size (kbytes, -m) unlimited max user processes (-u) 7729
To share a map of any reasonable size we’d need to boost those. Add the following to
/etc/security/limits.conf (the settings will be applied to the
ubuntu hard memlock unlimited # Memory is in units of kB. ubuntu soft memlock 8388608
Now logout and login again so that the changes take effect. Check.
$ ulimit -a | grep max max locked memory (kbytes, -l) 8388608 max memory size (kbytes, -m) unlimited max user processes (-u) 7729
Using Shared Memory
Now we are ready to add OSRM data to shared memory. Suppose that we have data which has already been prepared for use with OSRM.
$ osrm-datastore durban.xml.osrm
$ osrm-datastore --list [info] name shm key timestamp size [info] /static 0 1 16519676 [info] /updatable 1 1 34074239
Now you can launch the OSRM router using data from shared memory.
$ osrm-routed --shared-memory
It’s also possible to have multiple datasets in memory simultaneously. First let’s clean up (this will remove the existing OSRM data in shared memory).
$ osrm-datastore --remove-locks $ osrm-datastore --spring-clean
Now add two maps, giving each a specific name.
$ osrm-datastore --dataset-name durban durban.xml.osrm $ osrm-datastore --dataset-name pietermaritzburg pietermaritzburg.xml.osrm
$ osrm-datastore --list [info] name shm key timestamp size [info] durban/static 0 1 16519676 [info] durban/updatable 1 1 34074239 [info] pietermaritzburg/static 2 1 5691727 [info] pietermaritzburg/updatable 3 1 9880003
You can now launch the OSRM router using either of these shared datasets.
osrm-routed --shared-memory --dataset-name=pietermaritzburg
- According to this thread it’s only possible for OSRM to create shared memory objects for one user. So, for instance, you cannot run
osrm-datastorefor one map as
rootand then again for another map as another user. This will result in a “Permission denied” error.
- If you get a “Permission denied” error when running
osrm-datastoreit might help to reboot the machine.