Part 1 | Part 2 | Part 3

Overview:

The following instructions explain how to create a Nominatim database from an OSM planet file. It is assumed that you have already successfully installed the Nominatim software.

NOTE: The OSM Nominatim planet files are imported by breaking up countries into 30 “rank” phases of data layers and respective indexing. Rank 26 (streets) and Rank 30 (other data) take the most amount of time to import. For more information, see: http://wiki.openstreetmap.org/wiki/Nominatim/Development_overview

NOTE: If you have a snap shot of your system, you could practice the following procedure with a small planet file, such as North Dakota, to get comfortable with the process. I did this several times until I got familiar with the “way of the OSM”. Then, reset your system for the full North America import.

Prerequistes:

  • Lots of time and patience. As noted, this process can take several days, depending on the planet file set you choose to import.
  • A UPS is highly recommended, as any hiccup in the process will require a restart.
  • Ensure you have a good snapshot of current CentOS configuration, so you can recover or run this process multiple times if needed.
  • It is assumed that you have already installed and configured your base CentOS OSM Nominatim system:
  • See disclaimer

1: Preparing Server for Data File Downloads

Create the following directory structures, if they do not exist:

mkdir -p /opt/nominatim/source_data/planet_files
mkdir -p /opt/nominatim/source_data/census_tiger_edge_files
chmod +r /opt/nominatim/source_data/ -R

chown nominatim:nominatim /opt/nominatim -R

2: Download Data Source “Planet Files”

In this example, we are building an OSM Nominatim server that will only serve up North American addresses. Therefore, we are downloading a subset of the OSM “planet files” that are maintained at: http://download.geofabrik.de/north-america.html

This site maintains a lot of OSM planet file subsets and is updated frequently; they do some good work at geofabrik.

  1. Download the following to the /opt/nominatim/source_data/planet_files location on your server:
cd /opt/nominatim/source_data/planet_files
wget http://download.geofabrik.de/north-america-latest.osm.pbf

NOTE: You can also download different pdf files and merge them into a custom file using OSM Convert. For more information, see https://wiki.openstreetmap.org/wiki/Osmconvert

3. Loading the Planet File

  1. Temporarily turn off SELINUX, as the load script performs both executable, postgresql and library files calls
    setenforce 0
    
    sestatus
  2. Ensure that PostgreSQL is running:
    systemctl status postgresql
  3. Create temp location to process files and Ensure that nominatim user owns /opt/nominatim/source_data/ and /opt/nominatim/temp
    cd /opt/nominatim/ 
    mkdir temp
    
    chown nominatim:nominatim /opt/nominatim -R
    
    chmod +r /opt/nominatim/source_data/ -R
    
    chown nominatim:nominatim /opt/osmosis -R
    
    chmod +r /opt/osmosis/ -R
    
    su nominatim
  4. Using a Hypervisor with 4-core 3.2GHz 16GB RAM dedicated to VM, it took about 72 hours to import the North America planet file. The following command will start the import (replace the path to your osm.pdf planet file extract).
    NOTE: The command ends with & so it will be disowned by the command terminal, as remoting in for days makes no sense ;-}
    cd  /opt/nominatim/temp
    
    /srv/nominatim/Nominatim/build/utils/setup.php --osm-file /opt/nominatim/source_data/planet_files/north-america-latest.osm.pbf --all --osm2pgsql-cache 1000 2>&1 | tee setup.log &
    
    disown

    Over the next 72 hours, open a terminal; examine process list and log periodically to ensure the setup.php import process is still running. Examine log to ensure the process finished without error.

4. Make Snapshot of Server

This is a good time to create another VM snapshot of the server and configuration. The next part of the process will import the US Census TIGER data. This import takes a significant time, and having a snapshot can help it you need to restart this process, or create a cloned system.

5. Reconciling PostgreSQL apache DB user to nominatim DB

The OSM Nominatim PHP CLI utilities expect to run as a linux user with the same user name as the user in PostgreSQL.

  • To ensure everything goes smooth, I run the setup PHP script under the linux postgres user, which creates the database objects under the postgres DB user.
  • The Nominatim web application will also need access privileges to the nominatim database; by default, the web application runs under the apache user context.
  1. Execute the PostgreSQL command line interface:
    psql -U postgres
  2. Check current DB privileges and ensure apache user is present
    postgres=# du
  3. Grant access to apache DB user
    postgres=# grant all privileges on database nominatim to apache;
  4. Grant superuser role to apache DB user
    NOTE: Review this and other privileges before putting into production
    postgres=# ALTER USER apache WITH SUPERUSER;
  5. Check DB schema access. A restart of Apache after this may be in order.
    postgres=# l

Next step … Open Street Maps Nominatim Part 3: Adding US Census TIGER Address Data

References

If you have hung in there, we are done with the OSM Nominatim planet file load. The next article shall address loading and US Census TIGER data to get full street address resolution. paulsDevBlog.End();