A graphics reporter at NPR
I specialize in maps, but do lots of things
I've decided to retitle this talk...
If this idiot can self-host his own custom tiles, you can too (for cheap!)
I used to make a lot of slippy maps
I used to do a lot of slippy maps in the early days of (free) mapbox. I was an intern at Mapbox back in the earliest of early days. Tilemill!
2014
Here's a map I made a decade ago when I was working at the Dept. of Energy.
But over the last 5+ years I've avoided them.
Why?
2024
Here's that same map now...it's broken! Someone stopped paying their bill.
Not all are broken...but...
What is this (still) costing the Urban Institute? (my former employer)
"You wouldn't pay someone a monthly subscription just to custom host your css files would you?" -Brandon Liu of Protomaps, last night (paraphrase)
Can we FREE the tile?????
NPR is a non-profit
We try to save money wherever possible, and ongoing licenses are something we avoid.
This is a problem
Our peers still make incredible interactive maps
Josh Katz who presented yesterday worked on this. Shout out.
and want in on the fun!
In particular...
The USDA hardiness map was updated about a year ago for the first time in 10+ years.
My colleague Julia Simon did a quick story on this, after it's release in Nov. 2023. Aly Hurt put together this dirty before/after.
It's cool but....
I WANT MORE
But how?
Enter Kevin Schaul, Washington Post
Kevin's blog post started the gears turning for us.
His team created a really amazing interactive map, with all open source tools.
So maybe it's possible.
All I need is
to understand...
🌎 OpenMapTiles for building tiles
🌎 PMTiles for tile hosting
🌎 Maputnik for style editing
🌎 Maplibre-gl-js for client-side rendering
😵💫
Remember, I haven't touched this stuff since 2018 or so. Most of this was foreign to me.
So I tried to dig in
Just a simple workflow for baking your own tiles from OSM data...
"the most bonkers Makefile I have ever seen."
-Kevin Schaul
Did I mention it involved Docker!
Everyone knows how to use Docker!
Everyone knows how to use Docker!
(No I don't)
No
Oh maybe OpenMapTiles parent company will help make this easier...
😓
After a bunch of hours of tutorials, dead ends, and semi-dead ends... I had...
A rough idea of the how the stack should work
But several errors, and no map.
Enter Chris Amico, from MuckRock
He also presented a great, standing room only session at NICAR in Baltimore in March.
https://github.com/eyeseast/nicar24-self-hosted-maps
😧
Guess what?
You don't have to make your own tiles from scratch!
🌎 OpenMapTiles for building tiles
🌎 PMTiles for tile hosting
🌎 Maputnik for style editing
🌎 Maplibre-gl-js for client-side rendering
Its a big ass download. But I believe in you.
You are resourceful.
You can download 120GB
You can also extract a smaller portion
Next, we just put the pmtiles into an S3 bucket.
https://docs.protomaps.com/deploy/aws
Then using MapLibre GL JS, add the PMTiles protocol
That's it.
But wait! There's more!
Don't know how to use S3?
Get an API key and simply pay Brandon for his tiles!
Or just sponsor him anyway
(Talk to Brandon if you can, he is really smart.)
I'm not going to get into the ins and outs of why pmtiles is superior.
Okay, $ talk:
And check out how cheap it is, by comparison to other options
Transferring the data for about 100,000 tiles would cost $1.
The main two axes to save money are:
- to reduce the total number of tiles requested, and
- to reduce the size of each tile.
Hide layers at certain zooms.
For instance, we opted to avoid showing the hillshade layer until the map was zoomed in. This reduced the number of large tiles requested.
Lazy-load tile layers.
Initially the show/hide logic was based on the opacity of layers. This requested 3-4 more tiles than needed to be shown at any given time, which was costly and inefficient. Changing this logic dramatically improved the browser’s paint speed as well.
Cap maximum zoom level in explore mode
At a certain point, the user doesn’t need to zoom any further, so limit it to save on the number of tile requests.
Limit or delay exploration if it’s not essential.
We really wanted to have folks be able to find themselves in the data and explore it. Sometimes this is not necessary. Limiting exploration confined the upper end of possible costs.
Styling!
With Maplibre/Mapbox's style spec
🤢
Maputnik is a great place to start for styling your basemap
I found it to be sort of tricky to get rolling. My recommendation is that you should Use one of the OS defaults styles as a starting point, and then tweak colors, sizes, etc. from there.
DM me on slack if you want me to make a tutorial on how to get rolling with it.
But what about styling the USDA data?
Who doesn't want to write this sort of code:
But our friend the 🤖 helps A LOT
Occasionally, it will lie to you
I don't think this exists, and that's why it lied to me.
transparentize is a css function that exists in sass, but as far as I can tell, not in mapbox or maplibre
Correct me if I'm wrong.
But after styling OSM with maputnik
And our data with help from 🤖
You get a pretty, customizable, fast and CHEAP map
(oh psst: quick aside, I didn't go into the data that we are layering in here...We pre processed it in QGIS and then converted to pmtiles with tippacanoe. Longer blog TK!)
anyway...
Pros
- CHEAP!
- beautiful and customizable
- You keep it forever...(or until S3 explodes)
Cons
- time consuming to learn! and daunting!
- less support options when you get lost, compared to paying for a SaaS
- alt projection support? Not yet.
- harder to style than other user interfaces
https://maplibre.org/roadmap/globe-view/
npr.org/gardenmap
Here's some resources that might be helpful:
### Make your fonts work on your map
- https://maplibre.org/font-maker/
### Help with sprite creation
- https://github.com/flother/spreet
### Maputnik
- https://www.youtube.com/watch?v=XoDh0gEnBQo&t=292s&ab_channel=Lukas
- https://github.com/maplibre/maputnik/wiki/Style-GeoJSON-Files
- style spec: https://docs.mapbox.com/style-spec/guides/
- You may need a maptiler (free) account and API key to use this locally...this is poorly documented: https://cloud.maptiler.com/account/keys/
- You can also use the [web editor](https://maplibre.org/maputnik/?layer=2134303678%7E0#0.49/0/0) but then you can't easily integrate with this repo.
### How to get Geojsons into postgis/openmaptiles
- https://www.youtube.com/watch?v=3xpTBJAL8nc&list=PLGHe6Moaz52Mcq4BC9vczIIizNzwIYocv&index=4&ab_channel=MapTiler (ogr2ogr)
