Although last month it was said the Volume 10 Dunedin-Mosgiel maps had been brought to a close, this has in fact only just happened. The delay was mostly due to the need to get the Volumes section of the NZRM website accessible again, which required the menu to be reinstated at the top of each page. This proved somewhat difficult to achieve but it has now been completed, and so it is possible to upload the completed maps of this section and connect them to the Volume 10 page. This is yet to be updated to link to the files but should be active in a day or two.

There has been activity into several sections of Volume 6 as well, with maps now being generated for Naenae and Paekakariki as well as investigations into other parts of this volume, which covers the entire Wellington suburban area and the full Wairarapa Line except for Woodville Station (part of Volume 5). Volume 6 was originally slated for completion this year along with several other volumes that are near completion. Because of various delays this is now a bit behind. However there are some aerial surveys of Volume 6 it is hoped to pick up in a bulk collection from Linz in the next few months, a list of which is being prepared and which is likely to take in all of Volumes 6, 7, 8 and 9 previously prioritised for this year. Much of what is happening in V6 is mostly to do with it being expanded to include the NIMT near Wellington and getting extra aerial surveys that are becoming available into the new and existing areas. V6 also needs to be republished due to the improved filenaming system that makes it a lot easier to locate a specific map at a known distance on the route. So Volume 6 will definitely be a key focus for the rest of this year.

As mentioned the plan for this year is to get the webmaps updated as well. This hasn’t happened for a few years and was supposed to happen a year ago but never occurred. One of the key issues for getting an update done is the change in volume definitions that took place last year and another is setting up the build environment. The determination is that while it will be unlikely that all volumes will be ready to be included in the webmaps update, it is possible merely to update one volume at a time, and so at 31 December, it is expected the webmaps site will be updated only with the new content from Volume 6. The data from the other volumes will be updated in steps throughout 2025, with the expectation that each of the other 11 volumes will be added to the webmaps and captured at monthly intervals, so that at the end of 2025, a full update will have been completed over the whole year.

The build of webmaps requires two broad stages: updating each volume project to the new specifications, and updating the build project. For web tile creation, which is the stage where the millions of XYZ tiles required to be served by the web server for each zoom level are generated, a special global build project is used along with a special layer in each volume called “extents” which specifies the exact area to be captured in the tiles. Obviously, rather than capturing the entire country, we only want to capture the specific areas that rail corridors pass through, in order to keep our volume of web data to the minimum level needed. The global project is used along with a set of scripts that generate the tiles automatically by stepping through the extents layer and capturing the tiles for each of the specified extents. The web maps client running in the browser (provided by the Leaflet library written in Javascript) gives the user a choice of base layers (which will be updated to be NZRM topography, Linz Basemaps and Linz Topomap, omitting the aerlal layers choices presently offered) selected by radio buttons, and overlay layers specified by checkboxes. The bases come from either web provided layers from the Linz servers or a topographic base layer generated in the global build project for each webmaps update. The overlays are generated in the NZRM webmaps global build project by combining source layers in seven distinct groups, resulting in there being seven separate groups of XYZ tiles generated. When the user chooses which check boxes to select in the browser, the Leaflet client then accesses these seven groups of tiles to decide which ones to load.

So when the global project is built, it goes through the 12 source projects (one for each of the 12 volumes) getting the extents to be captured in the particular volume, and then it selects the NZRM topographic base layer and captures that, it then selects in turn each of the seven groups of source layers and captures them one at a time, to produce a total of eight groups of tiles each being an XYZ folder tree in a specific source path. There are 20 zoom levels so it also has to step through each of these levels to create tiles for that level. After this, the tiles can’t go straight onto the web server as is because there are so many of them that the overhead of transmitting them to the web server would be very slow and inefficient, and because the tiles at a size of just 256×256 pixels are very small files, they would waste a lot of disk space stored as single files on a disk, and finally because many filesystems are unable to store millions of files on one disk partition. Instead, another script is run to insert all of the tiles for each of the 8 layers into a SQLite database file (one per layer), using the mbtiles format. These mbtiles files are then uploaded to the web server. In addition to the Leaflet javascript client running in the web browser that collects the files and creates the web map that users can explore on their device, the server side functionality consists of a simple PHP script supplied by Mapbox as free open source software. This essentially receives requests for particular tiles from the Leaflet client and extracts them from the relevant mbtiles files stored on the web server.

Work is therefore going on at the moment to get the volume 6 data up to date, create the build environment on the build computer, and update the global build project with renamed source layers and any style changes that were made. The extents layers are affected by the changes in volume definitions, so the Volume 6 extents need to be added to with the ones from other volumes that have been brought into Volume 6, and removing any that have been taken out and put into other layers. Most layers’ source filenames and paths have changed over the last year due to data restructuring so the old ones in the build project file have to be replaced with the new ones. So many manual steps have to be completed. The build computer is usually set up with its own web server to enable the webmaps site to be tested locally before it goes live. On the live web server, two separate file paths are maintained to enable the live data to be checked using a test script before it is switched over to the live one.

Anyway as you can hopefully see by now, getting the web maps production environment all set up to do the capture is a lot of work and having to complete this work by the end of the year for all 12 volumes is quite a process. Because the project is working gradually through all the 12 volumes in turn, some of the projects have not been updated to the new standards yet and may still contain incorrect data boundaries, incorrect data layer filenames, and unadjusted extents layers, and therefore cannot be included in a full global build.