{
"$type": "site.standard.document",
"bskyPostRef": {
"cid": "bafyreibiyiebpakubaz4vlipbbthp6ggkgcqiptumfthaiim2hk2l6dvoy",
"uri": "at://did:plc:4tuge3k3comfj4nfvqnwkemn/app.bsky.feed.post/3mjdjaheoloc2"
},
"coverImage": {
"$type": "blob",
"ref": {
"$link": "bafkreic2ppmkqjwmk3m5rihx24l5mbzmbu5wkeoz5eibzl6kv6jfyxvlda"
},
"mimeType": "image/png",
"size": 474012
},
"path": "/user/SomeoneElse/diary/408501",
"publishedAt": "2026-04-11T20:49:30.000Z",
"site": "https://www.openstreetmap.org",
"tags": [
"this one",
"OpenHistoricalMap",
"documented",
"shared",
"here",
"vector zoom 14",
"tilemaker",
"mod_mbtiles",
"https://map.atownsend.org.uk/vector/index_svwd01.html#7/51.407/0.178",
"schema",
"set_way_area_name_and_fill_minzoom_sea",
"set_sqkm_name_minzoom",
"eventually",
"has been added to vector",
"did not appear on raster",
"first of those",
"simple script"
],
"textContent": "## The problem\n\nI’ve been creating and serving web-based maps such as this one for some time. That’s based on raster tiles, and an osm2pgsql database is used to store the data that the tiles are created from, on demand as a request to view a tile is made.\n\nFor various reasons I wanted to also create a similar map using vector tiles. With vector tiles what is sent to the client (such as a web browser) is not lots of small pictures that the client stitches together, but instead larger chunks of data, still geographically separated. The client then creates the map itself based on the style that it has been told to show the data in, combined with the data itself.\n\nI’d noticed that the vector maps that I was displaying were sometimes slow to load, especially at some lower zoom levels such as vector zoom 8. Note that vector zoom levels are one less than raster zoom levels, so vector 8 is raster 9.\n\nThis diary entry describes what I did to mitigate the problem (mostly over a year ago now - it’s taken me a while to get around to writing this!).\n\nFor info, also see similar work elsewhere, such as in OpenHistoricalMap.\n\n## The schema and the style\n\nOften with OSM raster tile styles, what is in the osm2pgsql database is a selection of raw OSM keys, and the map style then chooses which of those to show. My raster style wasn’t really like that; it made significant use of lua scripts (called both on initial database load and on all subsequent updates) to convert OSM data into a state in the database in which it was easy to display.\n\nThis approach transferred really well to vector tiles. I documented the schema, and much of the code is actually shared between raster and vector. Once an OSM item has been transformed the raster code adds it to a database and the vector code creates vector tiles.\n\n## Vector tiles\n\nThe individual vector tiles can be seen in debug here. As you zoom in you’ll see that the squares get smaller, as far as vector zoom 14. Those are the highest zoom vector tiles created and things displayed at zoom levels > 14 are actually stored in zoom 14 tiles but only displayed later.\n\nI’m creating vector tiles with tilemaker. That creates a big “.mbtiles” file which I copy to a directory under a web server.\n\n\n /var/www/html/vector/sve01: (29 GiB available)\n drwxr-xr-x 2 root root 4096 Mar 14 17:31 .\n drwxr-xr-x 4 root root 4096 Mar 28 01:04 ..\n -rw-r--r-- 1 root root 6203834368 Apr 10 01:58 tilemaker_sve01.mbtiles\n\n\nI’m using Apache as a web server and I’m using a module mod_mbtiles to allow individual vector tiles to be extracted from that and sent to a client. It would make no sense to send (in this case) 6GB of Britain and Ireland data to a client that only wants to show a map of a small part of Lincolnshire.\n\n## Why are things sometimes slow?\n\nA large vector tile has to be extracted from the large .mbtiles file, and sent to the client. The larger this is, the longer it will take, due to network speed issues among others.\n\nThere’s also an impact on the client - it potentially has to chew through a large amount of data to get to the data that it wants to display.\n\nI tested various scenarios - fast vs slow clients, small vs large MapLibre .json styles (based on the same Schema) and omitting data from tiles to make them just smaller. Of all of those, the most important factor was the size of the vector tiles themselves, so the challenge became “how can I minimise vector tile size at certin low zoom levels”.\n\nI initially looked at vector zoom 8, because it was quite slow, and was also used as a landing page zoom\n\n## Looking at Apache logs\n\nAn example entry in Apache’s `access.log` file looks like this:\n\n\n anipaddress - - [11/Apr/2026:00:36:36 +0100] \"GET /sve01/7/63/41.pbf HTTP/1.1\" 200 1016621 \"https://map.atownsend.org.uk/vector/index.html\" \"Mozilla/5.0 (X11; Linux x86_64; rv:140.0) Gecko/20100101 Firefox/140.0\"\n\n\nHere the path to the vector tile can be clearly seen. The “200” is the code for “yes that request was served successfully”, and the “1016621” is the size of the data returned. The browser had requested https://map.atownsend.org.uk/vector/index_svwd01.html#7/51.407/0.178 , and that zoom 7 tile contains most of southern England.\n\nIn terms of tile size, what I saw before any optimization in March 2025 was this:\n\n\n zoom a big example tile\n 0 19490\n 1 19076\n 2 8468\n 3 5442\n 4 69416\n 5 108941\n 6 735912\n 7 810916\n 8 1455666\n 9 2065707\n 10 1060757\n 11 562848\n 12 532770\n 13 343290\n 14 190211\n\n\n## The various parts of the problem\n\nClearly I needed to reduce the amount of data contained in a zoom 8 tile, but how? Why was it so big in the first place?\n\n### The vector style contains X because someone might want to show it.\n\nI had originally thought of the schema as being one that could support multiple styles. There were a couple of places where I was writing things into tiles because I thought that I (or some other consumer) might want to create something to show it later. This was the same approach as I’d used for the raster tiles database - there information is stored in the database to track changes to certain objects but isn’t used for display.\n\nIn order to reduce vector tile size I removed places where I’d done this on vector.\n\n### All X is displayed at zoom Y\n\nThis was the standard approach I’d used for raster tiles (inherited from early OSM Carto versions). The cost on raster wasn’t especially noticeable, because on raster it only affected render time, and if an old tile existed that was always sent to the user first (or, if zooming in, an overzoomed lower-zoom tile). Where previous tiles existed users weren’t faced with blank space, and the CPU effort to generate a new tile was on the server, not on their device.\n\nOn vector, the architecture means that this is no longer true. A large and complicated vector tile has a direct impact on the client, and the user waiting for a map has to wait while _their client_ creates a map from it.\n\nA challenge is that some features may be either very large or very small. If you look here you can see that some natural parks / nature reserves are large enough to be worth showing at the zoom level (but many aren’t), and similarly some lakes are large enough the show (mainly in Ireland) but that the smaller English and Welsh lakes aren’t.\n\nIn the code, the `water` logic can be seen here. That then calls set_way_area_name_and_fill_minzoom_sea to decide which vector tiles to actually write details to. That in turn honours `sqkm` values for point features representing “large woolly areas” by using set_sqkm_name_minzoom.\n\nIf you zoom in here you’ll see initially just Irish lakes, then the largest Welsh and English ones, and eventually the smallest.\n\nThis approach allows some previously unshown large features to appear. Fir example, the large military area off Essex has been added to vector at relatively low cost (there are few of that size) but did not appear on raster.\n\n### Finding out how many of X of size Y there are\n\nI have a rendering database for use for raster tiles for the same map style, and it can be useful for queries like this:\n\n\n gis=> select osm_id,name,way_area from planet_osm_polygon where \"natural\" = 'water' order by way_area desc;\n\n\nThe top values are as expected:\n\n\n osm_id | name | way_area\n ------------+----------------------------------------------------------------+---------------\n -1121118 | Lough Neagh | 1.13296e+09\n -189915 | Lough Corrib - Loch Coirib | 4.6095802e+08\n -12889 | Lough Derg | 3.21772e+08\n\n\nand if we look at it the other way around:\n\n\n gis=> select osm_id,name,way_area from planet_osm_polygon where \"natural\" = 'water' and name!='' order by way_area asc;\n\n osm_id | name | way_area\n ------------+----------------------------------------------------------------+---------------\n 1303491300 | Porthmadog Harbour | 0.010894\n 304781034 | Rainwater Collection Butt | 1.12869\n 304780098 | Rainwater Collection Butt | 1.12869\n 304781037 | Rainwater Collection Butt | 1.12869\n 304781038 | Rainwater Collection Butt | 1.12869\n 304781033 | Rainwater Collection Butt | 1.12869\n 304781035 | Rainwater Collection Butt | 1.12869\n 304781036 | Rainwater Collection Butt | 1.12869\n 969325744 | Well | 1.62326\n 1449480418 | Ditch of Bert | 1.67907\n 728196775 | 130 | 1.98514\n\n\nFor info, the first of those seems to be a bit of extreme “tagging for the renderer”. The “Rainwater Collection Butts” are “reservoirs”(!) on some allotments and “Ditch of Bert” is a school pond.\n\nThe raster database “way_area” is not the same value as Tilemaker’s vector one, but it is still useful.\n\n## Results of this optimisation\n\nBefore, the largest vector zoom 8 tile was 1455666. Afterwards, it was reduced to 729085, around 50% of the previous size. Success!\n\n## A worked example (in 2026)\n\nIn order to come up with more data for this diary entry, I wrote a simple script to analyse Apache logs and report on the largest tile at each zoom level. I then loaded only “Greater London” locally, in order to create a baseline to test against. Tile sizes were:\n\n\n 129951 6\n 281487 7\n 389658 8\n 2115753 9\n 1881853 10\n 3541994 11\n 1686270 12\n 802204 13\n 492326 14\n\n\nNote that some low zoom tiles show much more than just the loaded area and so are artificially smaller; but the same date will be used for subsequent tests meaning that differences are relevant,\n\nOne obvious difference is the jump in size at vector zoom 11. That corresponds to where buildings are first drawn. To see if that is coincidence, let’s move buildings to vector zoom 12 as a test:\n\n\n 129719 6\n 282138 7\n 390274 8\n 2117479 9\n 1884428 10\n 880557 11\n 1686348 12\n 802561 13\n 492375 14\n\n\nSo that’s a big difference, and not a coincidence.\n\nNext, how to look at building sizes? Let’s load Greater London into a raster database and have a look at way_area there. Here are some examples:\n\n\n osm_id | name | way_area\n ------------+--------------------------------------------------------------------------------------+----------\n 18926167 | Dagenham Engine Plant | 463567\n -1895281 | The O2 | 218893\n 200652378 | Brent Cross Shopping Centre | 110867\n 357278392 | Sainsbury's | 49090.2\n 227053396 | Wanis Cash and Carry | 25921.4\n 30327519 | Barking Bus Garage | 12756.5\n\n\nUsing MapLibre debug it’s easy to see the vector way_area corresponding to these places - zoom in until the label is shown, and way_area is an attribute of the label in the style.\n\nI want to include the largest buildings only in zoom 11 vector tiles, more at z12 and z13, and everything at z14. I experimented with values until I was happy with both the reduction in tile size and the visual results. The logic I ended up with was this:\n\n\n if ( passedt.way_area > 5000 ) then\n MinZoom( 11 )\n else\n if ( passedt.way_area > 2500 ) then\n MinZoom( 12 )\n else\n if ( passedt.way_area > 1250 ) then\n MinZoom( 13 )\n else\n MinZoom( 14 )\n end\n end\n end\n\n\nand the results?\n\n\n 129588 6\n 281145 7\n 389723 8\n 2115639 9\n 1882556 10 100%\n 898182 11 25%\n 726818 12 43%\n 552760 13 69%\n 492121 14 100%\n\n\nThat’s a significant reduction in tile size. Arguably zooms 11 and 12 are more usable because they’re not cluttered with lots of very small buildings\n\n## What next?\n\nClearly there’s more work to do, both regards to “tile size” and legibility. Zooms 9 and 10 are pretty crowded in London and the tertiary colour (which first appears there) really doesn’t work, especially over some types of landuse.",
"title": "Speeding up access to vector tiles"
}