Lua PerlinNoiseMap memory efficiency improvements

hmmmm
Member
 
Posts: 47
Joined: Tue Apr 02, 2013 04:04

Lua PerlinNoiseMap memory efficiency improvements

by hmmmm » Sun May 17, 2015 16:16

Hello Minetest Modding Community,

Recently I've had some complaints about extremely high memory usage in mods that make extensive use of Perlin noise maps (core.get_perlin_map/PerlinNoiseMap). I investigated and found that the most memory-intensive part of noise is, by far, placing the resulting computation into a Lua table. Optimizing memory usage was possible, but taking advantage of the improvements required some additions to the Lua API.

The first (and simplest) of the changes is to pass along a pre-existing table to receive the results of the noise calculations. This not only saves CPU time as all entries have already been allocated, but it minimizes Lua object creation so memory usage stays stable throughout execution. It can be used like so:
Code: Select all
local noise_buffer = {}
local noise = nil
...
function do_noise_thing(pmin, pmax)
   ...
   -- Notice that we reuse the same noise object instead of creating a new one each call
   noise = noise or core.get_perlin_map(noise_params, chunk_size)

   -- Here, the variable 'noisevals' is merely a reference to the table noise_buffer, not a copy.
   local noisevals = noise:get3dMap_flat(pmin, noise_buffer)
   ...
end


The second change requires the use of new PerlinNoiseMap methods calc2dMap, calc3dMap, and getMapSlice.
First, calc2dMap or calc3dMap is called to compute the noise. Then, when a chunk of noise is needed, it can be retrieved from the internal buffer using getMapSlice(). getMapSlice() takes a slice of noise at the specified coordinates, relative to the position inside the buffer (not absolute map coordinates!) and starts at 1, as is standard for Lua. If a coordinate is not specified in the slice offset parameter, then all noise along that axis is written to the output table. Therefore, a horizontal plane would be {y=}, a vertical column would be {x=, z=}, a single row spanning the X axis would be {y=, z=}, the whole buffer would be {}, and a single point would be {x=, y=, z=}. getMapSlice's second parameter, the slice size, is the length along that specified axis to retrieve.

Here is an example of these new methods in use:
Code: Select all
local noise_buffer = {}
local noise = nil
...
function do_noise_thing(pmin, pmax)
   ...
   noise = noise or core.get_perlin_map(noise_params, chunk_size)

   -- Calculate the noise starting at pmin, storing the result internally.
   noise:calc3dMap(pmin)

   -- Generate the terrain
   local slice_count = 0
   local nvals
   local ni
   for z = pmin.z, pmax.z do
      -- Get an 80x80x2 slice of the noise result, if needed
      if slice_count == 0 then
         slice_count = 2
         nvals = noise:getMapSlice({z=z-pmin.z+1}, {z=slice_count}, noise_buffer)
         ni = 1
      end

      for y = pmin.y, pmax.y do
      for x = pmin.x, pmax.x do
         if nvals[ni] > 0.5 then
            ...
         end
         ni = ni + 1
      end
      end

      slice_count = slice_count - 1
   end
   ...
end


With the above code, execution time is 1.8% slower than the previous example, but uses a whole 40x less memory. The table noise_buffer consumes 200KB instead of 8000KB.

I hope you all find this information useful in minimizing the memory footprint of your mods making use of noise maps.

- hmmmmm
 

User avatar
paramat
Developer
 
Posts: 3443
Joined: Sun Oct 28, 2012 00:05
Location: UK
GitHub: paramat
IRC: paramat
 

User avatar
Krock
Developer
 
Posts: 4391
Joined: Thu Oct 03, 2013 07:48
Location: Switzerland
GitHub: SmallJoker

Re: Lua PerlinNoiseMap memory efficiency improvements

by Krock » Mon May 18, 2015 15:58

Thanks for pointing to this. Does it count for Lua and LuaJIT?
Look, I programmed a bug for you. >> Mod Search Engine << - Mods by Krock - DuckDuckGo mod search bang: !mtmod <keyword here>
 

User avatar
paramat
Developer
 
Posts: 3443
Joined: Sun Oct 28, 2012 00:05
Location: UK
GitHub: paramat
IRC: paramat
 


Return to Modding Discussion



Who is online

Users browsing this forum: No registered users and 8 guests