Building Static Vector Tile Trees from GeoJson

On my team at the NYC Department of City Planning, we have been working with vector tiles and mapboxGL to create fast and beautiful web maps. When it comes to getting vector data into the client for display in mapboxGL, you basically have two options: Consume vector tiles from a {z}/{x}/{y}.mvt tile template, or load a .geojson file. If you choose the latter option, mapboxGL ends up actually converting your geojson file into vector tiles anyway before they are added to the map.

So, given these two options, what are the limitations involved? To date, our only vector tile sources have been Mapbox’s service, and Carto’s undocumented mvt endpoint. Both come with strings attached. For Mapbox, if your app ends up consuming more map views than your service plan allows, you’ll need to upgrade your plan and spend more $. For Carto, besides the cost implications, you’re also entering uncharted territory, as the mvt feature is undocumented and unsupported (it currently throws warnings in MapboxGL because it’s not using the updated v2 vector tile spec). It also doesn’t serve valid vector tiles when the geometries are LineStrings. (this is a bug, but there hasn’t been much activity on it for obvious reasons) The one big perk of using Carto is that it’s a real PostGIS database, and you can generate new vector tiles on-demand using SQL queries (just as you do with Carto’s well-established workflow for generating raster tiles on-demand).

Simply using GeoJSON doesn’t require any dependency on third-party services, but it means that the client has to load the *entire* dataset even if they are only looking at one small part of it. If the geojson file is more than a few hundred KB, this can become a large burden, especially if there is a lot of other map data simultaneously being loaded when your page first loads.

A solution: Static Vector Tile Trees

Remember, vector tile URLs, like raster tile URLs, are URLs. This means you don’t need fancy server technology deconstructing the route and building vector tiles on-the-fly, you can just store the vector tiles in a plain old nested directory structure and serve them up from a plain old webserver.

http://mydomain.com/somevectortileset/11/603/770.mvt

will work just fine as long as a file named

770.mvt

exists in

{webroot}/somevectortileset/11/603

.

Web Mapping Nerds, what's the best way to generate static vector tile pyramids from geoJSON data? Is there an easy button for this yet?

— Chris Whong (@chris_whong) March 3, 2017

I took to twitter asking whether an easy button existed yet for this. I was aware of geojson-vt, the javascript package that MapboxGL uses behind the scenes to cut geojson into vector tiles. I found this thread on github where someone was asking the same question, and ____ said it was indeed possible now that there was also a package that converted the raw data for a vector tile into protocol buffers (vt-pbf). With these two packages, all they needed was some control logic to read the geojson, and produce the directory structure and files.

Lucky me, I had already written code for locally storing static raster tile trees! Given a bounding box, min and max zoom, and a tile template, this script figures out what the tile ranges are for each zoom level, creates the directory tree and saves each tile to file in one fell swoop. This tool was a lot of fun to write because it makes use of the most fundamental algorithms of map tiling that determine which tile address a given lat/long coordinate falls in. This is the projection math that makes web maps possible.

With a little refactoring, I combined the code from my raster tile downloader with geojson-vt and vt-pbf to create geojson2mvt. It’s a thing. It’s on npm. You can use it now.

You pass in the path to a geojson file, and an options object including the name of the root-level folder to make, the WGS84 bounds of the tree to be created, the min and max zooms desired, and the name of the layer in the vector tile (vector tiles can contain many distinct layers)

In the example below, we are capturing all of New York City from zoom levels 8 through 18. The source geoJson is NYC’s bus routes. It’s a 4.4MB geoJSON file. When converted into a static vector tile tree, it becomes 512,357 tiles and the entire tree is only 4.9MB! Individual tiles vary in size of course, but most are in the dozens of kilobytes or smaller, giving us much faster load times.

var geojson2mvt = require('./src');

const filePath = './bus_routes.geojson';

var options = {
  rootDir: 'tiles',
  bbox : [40.426042,-74.599228,40.884448,-73.409958], //[south,west,north,east]
  zoom : {
    min : 8,
    max : 18,
  },
  layerName: 'layer0',
};

geojson2mvt(filePath, options);

Next steps

Now that we have static vector tiles, how should we serve them? For us, we’ll be coming up with a workflow that uses a script written on top of geojson2mvt to build the tiles in a place where our custom express.js api can access them. Then we’ll set up a simple route that uses express.static() to grab the appropriate file, or return a 404 if it doesn’t exist.

Here’s our freshly-cut vector tiles in Maputnik, ready to style:

Once the express api is up and running and serving tiles, we can add the tile template to Maputnik to style the layer. Once we have it looking good, we copy and paste the styles into our codebase along with the tile template. A new map layer is born!

Happy Mapping!