OpenStreetMap as geodata source

OpenStreetMap – the biggest geodata source

Every day OpenStreetMap project gains popularity, new users and new mappers all over the world. Driven by community OpenStreetMap nowadays became one of the biggest geodata sources. All of us already used to utilize the OSM based maps, which are easy to integrate, easy to style and relatively inexpensive. Moreover, there are already a lot of services, for example, routing, geocoding and others, that are based on some OSM data. However, a lot of geodata added to OSM is still underrated and used only for the generation of visual map tiles.

What kind of geodata can I find in the OSM database?

OpenStreetMap contains an enormous amount of data – stating from administrative boundaries and main streets to benches and single trees. Often it’s not only visible information but all available data about the place. For example, the number of floors of a building, websites, opening hours and other data.

We recommend you to start with the OpenStreetMap website to find out what kind of data is available for your use-case and region:

  • Zoom the map in to the region
  • Select “Query features” tool to get the geodata present on the map
  • Click on a map and explore found features
Getting geodata from OSM

How good is the quality of the OSM geodata?

Working with the OSM database you need to understand that the data added by thousands of different people with different backgrounds and knowledge. That’s why sometime it may happen that different tags or even sometimes different data types were used to map places. So it may require additional post-processing of the OSM data. On the other hand, the community works hard to unify tags, data formats, and increase the quality of the data. So the OSM geodata quality constantly increases.

How can I get the geodata from the OSM database?

Even if OpenStreetMap is open and free, it could be tricky to get the data without expertise. Here are some ways how to get data from OSM. Depending on requirements as well as the amount of time and money you plan to spend, some of them may fit better your conditions.

Places API

It’s the simplest way, but the most restricted one. There are a number of companies that provide an API to query amenities and points of interest. This could be the perfect solution if you are simply looking for places and some information about them.

Check our Places API. We offer queries for points of interest and amenities of different types for a specified bounding box. Together with that, we transfer all OSM tags added to a place.

1. Easy
1. Cost money
2. Restricted
Overpass API

Another way to get geodata from OSM is Overpass API. It’s a more complicated way, while it requires a knowledge of Overpass QL query language. However, thanks to the community there are a lot of examples and guidelines on the web.

The API acts as a database – gets queries and returns data. You can create simple queries or complicated ones, with conditions and geo-operations. With the help of Overpass Turbo – web-based frontend, you can run Overpass API queries and get results on an interactive map or as a text.

At the moment there are several Overpass API instances provided by the community, so you can start for free and without registration. But all of them have restrictions and limits.

Geoapify offers a commercial instance of Overpass API. Contact us to get access to it.

1. Flexible and powerful
2. Community support
3. Free for small volumes
1. Requires a knowledge of Overpass QL query language
2. Commercial solution required for big volumes
OSM data extracts

Probably the most complicated but the most powerful and free way to use OSM as a geodata source.

As OpenStreetMap is an open-source project, you can always download the most recent data extracts from a website. By downloading the data you need to keep in mind that the whole planet database is about 49 GB (PBF file, Feb. 2020) and it may make sense to use parts of it – separate continents or countries.

There are different services that provide access to OSM data:

The downloaded file can be processed with a variety of tools or can be imported into a SQL database:

  • Osmosis – converting OSM files among different formats and databases
  • Osm2pgsql – importing OSM files into PostGIS database
1. Flexible and powerful
2. Free for small and big volumes
1. Requires expertise in databases and SQL
2. Big data extracts require decent servers (min 32GB RAM, 500-700GB SSD)
3. The data import may take up to several days
Custom data sets

If fast results and data quality are important for you you can order a custom geodata set from specialists.

Geoapify offers a service to create custom geodata sets. Just describe your needs – what kind of data, data region, data format, and we make the work done for you! Contact us for more information.

1. Save time and resources
2. Highquality results
1. Cost some money

Nominatim geocoder

Nominatim geocoder is a free, open-source address search engine that you can run on your own computer, even without an Internet connection! And this is one of few solutions which allows you to do high volume, batch geocoding for any address on our planet on budget.

Nominatim can do both forward and reverse geocoding and supports all countries, languages and address formats in the world. It is developed and maintained by the OpenStreetMap community and used as a default geocoding service for OSM data.

Nominatim pros and cons

Sounds good so far? Great!

But before you invest hours and days into getting Nominatim up and running, let’s have a quick overview of its strong and weak sides, and make sure it can be a good fit for your needs.

PROSCONS
Free! No license fees, and very few limitations (you need to adhere to ODBL license)You need to invest your time and resources to get it up and running
Based on rich OSM and Wikipedia data, collected and updated by millions of volunteers, companies, and organizations around the worldMost address records are not authoritative, coverage can be spotty and some data can be obsolete or even incorrect
Community project – good activity and number of articles and configuration instructions in InternetNo guarantees, no support, no SLA; incomplete and often outdated instructions
Open source, you can extend or fix any part of itSometimes code is your only documentation
Works on your servers, also offlineYou need (powerful) servers and somebody who can maintain them
Very configurable queriesYou need to know which options to use to get results you want

To be fair, all Nominatim weaknesses are common for any big, community-driven, open-source project. And we at Geoapify have extensive experience with Nominatim and will be happy to support you.

Geocoding addresses with Nominatim

Nominatim accepts geocoding requests on the /search endpoint, in one of two exclusive modes – free-form and structured search.

Free-form address search

Free-form address query requires “q” parameter, like

Nominatim free-form search results for "Chicago"
Nominatim free-form search results for /search?q=chicago

In that mode, Nominatim supports special phrases like “in”, “near”

Nominatim search with special phrase "restaurant in Chicago"
Nominatim search with special phrase /search?q=restaurant+in+chicago

Structured address search

Alternatively, you can use a structured search, where address defined by a combination of:

street=<number> <name>
city=<city>
county=<county>
state=<state>
postalcode=<code>
Nominatim structured country search
Nominatim structured country search with /search?country=us

Nominatim structured geocoding is not as useful as it could be, because it doesn’t guarantee that search will be performed with requested granularity and prevents Nominatim from applying some internal geocoding heuristics. It also less flexible and does not support Nominatim’s special phrases.

We can only recommend to use it as a workaround for scenarios where Nominatim cannot correctly parse the address.

Filtering by location

If you know that results should belong to a specific country or area, it is highly recommended to narrow down search results with “countrycodes=<comma-separated list of country codes>” and “viewbox=<lon1>,<lat1>,<lon2>,<lat2>” parameters.

Nominatim search with location filter
Nominatim search with location filter
/search?q=berlin&countrycodes=us&viewbox=-89.8,39.7,-90,39.8

By default “viewbox” parameter works as location bias and includes results both inside and outside of provided bounding box.

Adding “bounded=1” parameter instructs Nominatim to return only results included in viewbox.

Search by type and amenity

Nominatim has only very limited support for places search. In most cases, you will get better and more reliable results with other services, for instance, Geoapify Places API or Overpass. But for simple cases, Nominatim can be good enough.

There are three ways to perform a search by type and amenity in Nominatim:

  • Using special phrases like “in” or “near”,
    for example
    /search?q=pub+in+new+york
  • Using search bounded by the “viewbox” together with “amenity” parameter,
    for example, searching for pubs in Manhattan area
    /search?viewbox=-74.02631,40.73022,-73.96202,40.69769&bounded=1&amenity=pub
  • Using search bounded by the “viewbox” together with place type in square brackets,
    for example /search?q=[pub]&viewbox=-74.02631,40.73022,-73.96202,40.69769&bounded=1
Searching pubs in New York with Nominatim
Searching pubs in New York with Nominatim

You can find an extensive list of supported place types and amenities on OpenStreetMap wiki.

Reverse geocoding with Nominatim

Nominatim can find and return addresses corresponding to a given latitude and longitude coordinate. For that, you need to use /reverse endpoint together with “lat” and “lon” parameters.

Finding Statue of Liberty by its GPS coordinate with Nominatim
Finding Statue of Liberty by its GPS coordinate with Nominatim
/reverse?lon=-74.044548&lat=40.6892532&format=html

Results granularity

Nominatim currently does not support filtering by object type nor specific geocoding granularity like “city”, “administrative area” or “street”. But there are several workarounds.

First, Nominatim /search endpoint accepts the (undocumented) “featuretype” parameter with one of the following values: “country”, “state”, “city”, and “settlement” (anything below country and above the street level).

Then, reverse geocoding supports “zoom” parameter and returns the results of different granularity depending on its value:

ZOOMRESULT GRANULARITY
3country
5state
8county
10city
14suburb
16major streets
17major and minor streets
18building

In the case of forward geocoding, you will need to do the second call to reverse geocoding with previously looked up coordinates and desired zoom level.

Another alternative requires a call to /details endpoint together with addressdetails=1 parameter and “place_id” matching the id of the previously geocoded result. This way you get full hierarchy from given the place up to country level, including administrative and postal code boundaries.

Output formats

By default Nominatim returns a formatted HTML page, that is ready to be displayed in the browser. If you need a machine-readable response format, you will need to add the “format” parameter and set it to one of “xml”, “json”, “geojson” and “geocodejson” output formats.

We recommend using “geojson” as the only standard and widely supported format. The second favorite format is “geocodejson”, which is generally compatible with GeoJSON specification and in the case of Nominatim includes the most complete and standartized address structure.

Example geocoding output for “/search?q=berlin&format=geojson&limit=1”

{
    "type": "FeatureCollection",
    "licence": "Data © OpenStreetMap contributors, ODbL 1.0. https://osm.org/copyright",
    "features": [
        {
            "type": "Feature",
            "properties": {
                "place_id": 595794,
                "osm_type": "node",
                "osm_id": 240109189,
                "display_name": "Berlin, 10117, Deutschland",
                "place_rank": 15,
                "category": "place",
                "type": "city",
                "importance": 0.887539028249136,
                "icon": "https://nominatim.openstreetmap.org/images/mapicons/poi_place_city.p.20.png"
            },
            "bbox": [
                13.2288599,
                52.3570365,
                13.5488599,
                52.6770365
            ],
            "geometry": {
                "type": "Point",
                "coordinates": [
                    13.3888599,
                    52.5170365
                ]
            }
        }
    ]
}

Geometry and additional details

Nominatim can lookup and return some additional information

addressdetails=1
extratags=1
namedetails=1

Geocoding example with additional fields
“/search?q=berlin&format=geojson&limit=1&addressdetails=1&extratags=1&namedetails=1”

{
    "type": "FeatureCollection",
    "licence": "Data © OpenStreetMap contributors, ODbL 1.0. https://osm.org/copyright",
    "features": [
        {
            "type": "Feature",
            "properties": {
                "place_id": 595794,
                "osm_type": "node",
                "osm_id": 240109189,
                "display_name": "Berlin, 10117, Deutschland",
                "place_rank": 15,
                "category": "place",
                "type": "city",
                "importance": 0.887539028249136,
                "icon": "https://nominatim.openstreetmap.org/images/mapicons/poi_place_city.p.20.png",
                "address": {
                    "city": "Berlin",
                    "state": "Berlin",
                    "postcode": "10117",
                    "country": "Deutschland",
                    "country_code": "de"
                },
                "extratags": {
                    "email": "[email protected]",
                    "capital": "yes",
                    "website": "https://www.berlin.de/",
                    "wikidata": "Q64",
                    "wikipedia": "de:Berlin",
                    "population": "3531201",
                    "name:prefix": "Land und Kreisfreie Stadt"
                },
                "namedetails": {
                    "name": "Berlin",
                    "name:ab": "Берлин",
                    "name:af": "Berlyn",
                    "name:am": "በርሊን",
                    "name:an": "Berlín",
                    "name:ar": "برلين",
                    "name:short": "BE",
                    "name:nds-nl": "Berlien",
                    "old_name:vi": "Bá Linh",
                    "name:bat-smg": "Berlīns",
                    "name:cbk-zam": "Berlín",
                    "name:fiu-vro": "Berliin",
                    "name:roa-rup": "Berlin",
                    "name:roa-tara": "Berline",
                    "name:be-tarask": "Бэрлін"
                }
            },
            "bbox": [
                13.2288599,
                52.3570365,
                13.5488599,
                52.6770365
            ],
            "geometry": {
                "type": "Point",
                "coordinates": [
                    13.3888599,
                    52.5170365
                ]
            }
        }
    ]
}

It is also possible to return the original geometry of the results instead of the coordinate and bounding box. For that you need to specify one of the following parameters:

polygon_geojson=1
polygon_kml=1
polygon_svg=1
polygon_text=1

Multilanguage support

By default, Nominatim uses “Accept-Language” HTTP header to choose the preferred language of the result. This can be changed by “accept-language” parameter, which can be set to one or multiple (comma-separated) language codes – in order of preference. If both the HTTP header and parameter are missing, Nominatim will return each geocoding result using the native language of the place.

Installation and hardware requirements

General installation instructions are very well documented in Nominatim Administration Guide, and there is no point in repeating the same information here.

You should plan that initial production-ready setup and configuration will take at least 2 days and up to several weeks of effort, depending on your situation, requirements, and level of expertise. You also need to have some level of confidence with Linux, Apache web server, PHP and PostgreSQL database to follow the instructions.

Hardware requirements depend on desired coverage (full planet, country, region or city) and if you plan to apply daily or hourly updates to the Nominatim database. For planet import, we recommend a server with at least 8 cores, 32 GB RAM and 700 GB of fast SSD/NVMe storage. With that, you can expect initial data import to take approximately 2-3 days. For smaller regional datasets your hardware requirements will be proportionally lower, but as a bare minimum, you need at least 2 CPUs, 4 GB memory and 50 GB of fast storage.

A possible alternative for testing and small non-profit projects you can use the free community-supported Nominatim instance. It is funded by the OpenStreetMap foundation and normally has the most up to date Nominatim version and data. As a downside, there are no up-time or performance guarantees, and your requests may be blocked without any further notice if you generate than 1 request per second, or send repetitive, systematic or batch queries.

Again, it is normally acceptable for testing and very small projects, but and serious business usage is discouraged. Please check the OSM Nominatim Acceptable Use Policy for more details.

As an alternative, we at Geoapify can offer you a full range of Nominatim support – from business-friendly, managed Nominatim instances, to dedicated installations, as well as consultation and assistance if you need help with configuration.

So, should I use Nominatim for geocoding?

You should definitely consider Nominatim if you need to do high-volume or batch geocoding with thousands and millions of addresses. It also can be very useful for scenarios when you need to do geocoding in secure environments or offline.

Another benefit comes from the open data that Nominatim uses. If something is missing or wrong in underlying OSM or Wikipedia data, you or anyone else can easily add or correct that information. It is normally much more challenging to do that with any alternative services.

Another benefit comes from the open data that Nominatim uses. If something is missing or wrong in underlying OSM or Wikipedia data, you or anyone else can easily add or correct that information. It is normally much more challenging to do that with any alternative services.

But do not expect Nominatim to be as smart as e.g. Google Maps API, especially if you deal with manually entered addresses. On average Nominatim does great job and correctly geocodes around 70% of addresses, while best commercial services can geocode around 90%.

Also, full-text-search in Nominatim is somewhat limited. Elasticsearch based alternatives like Photon or Pelias can give better results if you need to lookup places by name instead of addresses.

And don’t forget about time and resources that need to be invested into configuring, running and maintaining Nominatim installations.

If that is too much for you and you just need smart geocoding that works well, we can recommend giving our Geocoding API a try. It has very liberal pricing, generous free tier that is eligible for business usage and combines a number of technologies and know-how to deliver the best results as fast as possible.

Open GIS data by Copernicus

The importance of available open data plays a big role in developing geospatial and location analytics applications. Copernicus is the European Earth Observation and Monitoring Programme. It provides open GIS data and information to help better understand our planet.

Copernicus database is built with the help of thousands of air-, land- and marine-based sensors, which currently produce and collect more than 12 TB of data per day.

Copernicus open data services – https://www.copernicus.eu

What could you find in Copernicus open GIS data services?

Copernicus services deliver GIS data to allow efficiently manage natural resources, control the quality of air and water, plan cities construction, and optimize usage of agricultural lands.

At the moment Copernicus program offers open GIS data in 6 domains:

  • Atmosphere
  • Marine
  • Land
  • Climate Change
  • Security
  • Emergency

Copernicus Atmosphere

Atmosphere service produces and provides continuous data on atmospheric composition (Ozone, CO2, UV index and so on). Moreover, it allows to create the current situation analysis and make forecasts.

For example, atmospheric composition, air quality, ultra-violet & solar radiation, surface fluxes & emissions, climate forcing are the main focus of the service.

Copernicus Atmosphere
Copernicus Atmosphere

Copernicus Marine

Marine service offers open data on the biochemical and physical state, variability and dynamics of the global ocean and the European regional seas. So information about the coastal & marine environment, marine safety, resources data, as well as weather forecasts and climate observations you can find here.

Global ocean observation
Global ocean observation

Copernicus Land

Copernicus Land service provides geographical GIS data on land cover, land use & vegetation, state & changes, water cycles, and earth surface energy information. You will find the GIS data required for cities planning, forests & water management, agricultural & rural development applications. Moreover, the service offers data about the adaption of the ecosystem on climate changes.

Global Land Service - Vegetation
Global Land Service – Vegetation

Copernicus Climate Change

Climate change forecast as well as information of the current situation you can find in the Copernicus Climate Change service. For instance, surface temperature, greenhouse gases, sea ice & sea level, glaciers states, and other headline climate indicators.

Copernicus Security

The main focus of Copernicus security services is crisis prevention, preparedness and response in three key areas: border surveillance, maritime surveillance, support to EU External Action. The service provides earth observation materials, as well as training how to obtain and use them.

Copernicus Emergency

Copernicus Emergency service offers data required for building solutions on the management of natural disasters, man-made emergency situations, and humanitarian crises.

Together with rapid maps for emergency response, risk & recovery, which are used by prevention and planning and of the early warning systems, the service offers monitoring data to allow preventing floods, droughts, and forest fires.

Open GIS data for fire danger forecast
Fire danger forecast

Is the data free to use?

The GIS data is open and absolutely free to use for non-commercial and commercial use. However, the attribution to the Copernicus program must be provided.

Data formats and access

All the data is provided as products, which are available to download or via APIs. In addition, Copernicus delivers open GIS data in well-known GIS data formats, like shapefile, NetCDF, Grip, and others. Registration is required to access the data.

Visualize the GIS data

Download the required data or setup API connection to visualize the data on a map. By using different data visualization technologies you can achieve the needed accuracy in visualization. Moreover, different styles of maps help you to create a focus on one or another component.

Geoapify offers maps, which could become a base for data visualization. Read more about our maps, APIs and services.

OpenStreetMap is more than just a map

Even if OpenStreetMap has a “map” in its name, it’s not just a map. It’s a collaborative project, which allows collecting and structuring geodata into a database. The OpenStreetMap database, in turn, becomes a base for detailed and up to date maps, map components and map solutions on top of it.

The OpenStreetMap community is the biggest GIS community in the world and counts more than 1.000.000 active contributors so far and grows every day. As a result, we have accurate and up to date maps data for the whole planet. Read OpenStreetMap wiki to learn more.

Are OpenStreetMap data and maps free to use?

OpenStreetMap data is absolutely free! Everybody can download and use OpenStreetMap data for personal or commercial purposes (do not forget about attribution). In addition, there are many open-source projects with a permissive license, which use the OpenStreetMap database and allow creating maps, components, and tools based on them.

But what is the cost of setup, running and maintaining map services? It requires a lot of time, resources and expertise to start, run and keep up to date your own OpenStreetMap database instance, map tiles servers and map components.

That’s why there are third-party commercial companies offering products and components based on OpenStreetMap data.

Products based on OpenStreetMap data

Besides the maps by itself, there are many applications based on OpenStreetMap data – everything around maps, geospatial analytics, and location intelligence. The applications and services could be split by type. Some of the types are listed below.

Map tiles servers

Map tiles it’s prepared parts of maps. During map rendering a mapping library requests required map tiles from a server. Map tile servers could provide different map styles and even different formats. For example, raster or vector formats.

Geocoding and reverse geocoding

Geocoding and reverse geocoding are two base map operation you usually perform. The first one gives a place coordinates from the place address, the second one returns an address by coordinates.

You use geocoding applications for search on a map and input autocomplete.

Routing

Routing applications build a route between two or multiple points. Moreover, some of them are able to calculate the optimal routes and provide turn-by-turn directions. Together with driving or walking transportation modes, a lot of routing applications are able to build routes for public transport.

Places and amenities search

Places and amenities search services allow searching locations for given criteria. For example, “restaurants near me which are open now”, “all P+R in the city” and so on.

Some services provide geodata and polygons for some features. Which could be useful when drawing administrative or political boundaries or other regions on a map.

Isodistances and Isochrones (reachability and travel time maps)

How far can I get within 30min from my home? Where to book a hotel not to spend more than 20 min traveling to my points of interests? Isodistances and isochrones show areas reachable within a given distance or within a given time from a location.

As well as routing, iisolines are built for a transportation mode or a combination of transportation modes (for example, public transport + walking, park + ride).

Travel Time Map
Travel time map

Geoapify offers APIs and components built on OpenStreetMap data

We provide everything you need to create a map or extend an existing map with Geospatial analytics features. Starting from Free tariff plan we offer tariff plans for different load and usage. Register and try our APIs and components for Free.