The following instructions explain how to create a Nominatim database from an OSM planet file. It is assumed that you have already successfully installed the Nominatim software.
NOTE: The OSM Nominatim planet files are imported by breaking up countries into 30 “rank” phases of data layers and respective indexing. Rank 26 (streets) and Rank 30 (other data) take the most amount of time to import. For more information, see: http://wiki.openstreetmap.org/wiki/Nominatim/Development_overview
NOTE: If you have a snap shot of your system, you could practice the following procedure with a small planet file, such as North Dakota, to get comfortable with the process. I did this several times until I got familiar with the “way of the OSM”. Then, reset your system for the full North America import.
- Lots of time and patience. As noted, this process can take several days, depending on the planet file set you choose to import.
- A UPS is highly recommended, as any hiccup in the process will require a restart.
- Ensure you have a good snapshot of current CentOS configuration, so you can recover or run this process multiple times if needed.
- It is assumed that you have already installed and configured your base CentOS OSM Nominatim system:
- See disclaimer
1: Preparing Server for Data File Downloads
Create the following directory structures, if they do not exist:
mkdir -p /opt/nominatim/source_data/planet_files mkdir -p /opt/nominatim/source_data/census_tiger_edge_files chmod +r /opt/nominatim/source_data/ -R chown nominatim:nominatim /opt/nominatim -R
2: Download Data Source “Planet Files”
In this example, we are building an OSM Nominatim server that will only serve up North American addresses. Therefore, we are downloading a subset of the OSM “planet files” that are maintained at: http://download.geofabrik.de/north-america.html
This site maintains a lot of OSM planet file subsets and is updated frequently; they do some good work at geofabrik.
- Download the following to the /opt/nominatim/source_data/planet_files location on your server:
cd /opt/nominatim/source_data/planet_files wget http://download.geofabrik.de/north-america-latest.osm.pbf
NOTE: You can also download different pdf files and merge them into a custom file using OSM Convert. For more information, see https://wiki.openstreetmap.org/wiki/Osmconvert
3. Loading the Planet File
- Temporarily turn off SELINUX, as the load script performs both executable, postgresql and library files calls
setenforce 0 sestatus
- Ensure that PostgreSQL is running:
systemctl status postgresql
- Create temp location to process files and Ensure that nominatim user owns /opt/nominatim/source_data/ and /opt/nominatim/temp
cd /opt/nominatim/ mkdir temp chown nominatim:nominatim /opt/nominatim -R chmod +r /opt/nominatim/source_data/ -R chown nominatim:nominatim /opt/osmosis -R chmod +r /opt/osmosis/ -R su nominatim
- Using a Hypervisor with 4-core 3.2GHz 16GB RAM dedicated to VM, it took about 72 hours to import the North America planet file. The following command will start the import (replace the path to your osm.pdf planet file extract).
NOTE: The command ends with & so it will be disowned by the command terminal, as remoting in for days makes no sense ;-}
cd /opt/nominatim/temp /srv/nominatim/Nominatim/build/utils/setup.php --osm-file /opt/nominatim/source_data/planet_files/north-america-latest.osm.pbf --all --osm2pgsql-cache 1000 2>&1 | tee setup.log & disown
Over the next 72 hours, open a terminal; examine process list and log periodically to ensure the setup.php import process is still running. Examine log to ensure the process finished without error.
4. Make Snapshot of Server
This is a good time to create another VM snapshot of the server and configuration. The next part of the process will import the US Census TIGER data. This import takes a significant time, and having a snapshot can help it you need to restart this process, or create a cloned system.
5. Reconciling PostgreSQL apache DB user to nominatim DB
The OSM Nominatim PHP CLI utilities expect to run as a linux user with the same user name as the user in PostgreSQL.
- To ensure everything goes smooth, I run the setup PHP script under the linux postgres user, which creates the database objects under the postgres DB user.
- The Nominatim web application will also need access privileges to the nominatim database; by default, the web application runs under the apache user context.
- Execute the PostgreSQL command line interface:
psql -U postgres
- Check current DB privileges and ensure apache user is present
- Grant access to apache DB user
postgres=# grant all privileges on database nominatim to apache;
- Grant superuser role to apache DB user
NOTE: Review this and other privileges before putting into production
postgres=# ALTER USER apache WITH SUPERUSER;
- Check DB schema access. A restart of Apache after this may be in order.
- OSM Nominatim Wiki: http://wiki.openstreetmap.org/wiki/Nominatim
- OSM Nominatim Install: http://wiki.openstreetmap.org/wiki/Nominatim/Installation_on_CentOS
- OSM Nominatim Installation Troubleshooting: http://wiki.openstreetmap.org/wiki/Nominatim/Installation/Troubleshooting
If you have hung in there, we are done with the OSM Nominatim planet file load. The next article shall address loading and US Census TIGER data to get full street address resolution. paulsDevBlog.End();