Well here’s another update, and promises, promises, promises…. the last week has slowed things down a bit due to having to deal with other issues, but we are back up to speed now. Plus there have been more than a few issues with Gimp that caused crashes, meaning having to go back and redo stuff and try to find out what wouldn’t crash.

The reason there are issues with Gimp and needing a lot of hardware resources (a PC with 32 GB of RAM and a 200 GB SSD swap partition to use for the tile cache), is our preference to use the highest resolution imagery we can. For base imagery that is a resolution of 0.075 metres per pixel (i.e. 7.5 centimetres) and everything snowballs from there. Take for example Lyttelton. We could just do the main railyard but have chosen to cover all the areas where tracks went to, and that means a canvas of 12 tiles wide and 4 tiles high. That translates into a size in pixels of 57600 by 28800, or 1.6 billion pixels total. The highest resolution imagery from Retrolens (usually around 1:4300 scale but occasionally 1:3400) fits in well with that base resolution. By the time we’ve added in a few eras of historical imagery (from 1940 to 2000 generally) then the result, in this case, is a file size of 21 GiB. It’s only since Gimp 2.10 came out that it’s been possible for it to handle files greater than 4 GiB in size, which is a real blessing because otherwise multiple files would be needed which would be very inefficient and time consuming. 

We started out some of these Gimp projects covering multiple stations but then as more layers were added (we use a rule of thumb that no project will have more than 100 layers) some of these projects grew too big and had to be split into multiple files. That happened in the case of Lyttelton, which used to have Heathcote along for the ride, which is what was happening earlier this week when everything kept crashing because the system or Gimp hit some sort of limit. Heathcote is a big area to cover because it takes in Ferrymead, so the result was a lot of area to cover and eventually the need to split the files, which we can now recognise because the current version of the Heathcote file is 30 GiB and that is the upper limit of file size we’re prepared to work with, from experience. With a file that size it takes ages to load into Gimp, the system will easily use half the total tile cache just to bring it up with no edits, and it can take three hours to save (and Gimp can still crash partway through the save if the file is too big and lose everything). So you can see we are working with some real limits.

The resource issues for doing these highest quality images is the key issue that affects the kind of coverage we do of stations around the country. Christchurch is a particular exception that is going to be the highest quality for all of Greater Christchurch, across multiple generations, and backed up by research as well. That simply can’t be justified across NZ, and only major yards will get done to that level. Most sites will be done with just the NZR survey, an example being Bluff that was recently draw up, it was done to high quality but only one generation of historical images. With corridor surveys we can cover the smaller stations as well, but if we can’t get coverage of them when they were open, there won’t be any. Before we dropped the Midland Line coverage to focus on the Greater Christchurch stuff, we had covered just about all the stations to Kotuku with corridor surveys plus the major stations and some research as well. The same will go through for the MNL and MSL eventually as far as we can get to.

Anyway back onto Greater Christchurch, we only have the MNL from Riccarton through to Ashley to complete. The tiles for the rest will start to get spat out as quickly as we can from here, because map drawing is way behind and really needs to be pushed along as quickly as possible now. So all that MSL stuff will be pushed out as map tiles and then be drawn alongside finishing that stretch of MNL as quickly as possible.