Page 1 of 1

How does the SQLite DB work ?

PostPosted: Fri Jun 22, 2018 09:42
by LMD
A question to you, core developers :

After working with SQLite(for MTPC), I have learned from my mistakes, and am feeling quite safe and have learned much. Now, I'm asking myself whether it would be good to modify the SQLite DB from outside(Java) to make map ops not in LUA, but from outside program; more specific, I want to have - Java program which creates Schematics from 3d models -> writes them directly into map.
So, I know how it basically works : You got indices & blobs. Each indice(int) stores blob(chunk) position in map and is primary key. Blob is mapchunk(which size ? 80 ?). How do indices work(which order ? x+y*mapsize+z*mapsize² ?). And now, the final question : How do the binary large objects work ? Of course, every node has content ID. So, are you just storing the content IDs in giant array, indiced by x+y*chunksize+z*chunksize² ? Where's the metadata ?

Re: How does the SQLite DB work ?

PostPosted: Fri Jun 22, 2018 10:33
by ExeterDad
I did a casual read of this just the other day. I believe what you are looking for is documented here:
https://github.com/minetest/minetest/bl ... format.txt

Re: How does the SQLite DB work ?

PostPosted: Fri Jun 22, 2018 11:14
by LMD
Thanks ! Exactly what I was searching for !
This answers my indexing question :
Code: Select all
The key
--------
"pos" is created from the three coordinates of a MapBlock using this
algorithm, defined here in Python:

  def getBlockAsInteger(p):
      return int64(p[2]*16777216 + p[1]*4096 + p[0])

  def int64(u):
      while u >= 2**63:
          u -= 2**64
      while u <= -2**63:
          u += 2**64
      return u

It can be converted the other way by using this code:

  def getIntegerAsBlock(i):
      x = unsignedToSigned(i % 4096, 2048)
      i = int((i - x) / 4096)
      y = unsignedToSigned(i % 4096, 2048)
      i = int((i - y) / 4096)
      z = unsignedToSigned(i % 4096, 2048)
      return x,y,z

  def unsignedToSigned(i, max_positive):
      if i < max_positive:
          return i
      else:
return i - 2*max_positive


Blob/Mapchunk size : 16x16

Giant array node index :
Code: Select all
- The location of a node in each of those arrays is (z*16*16 + y*16 + x).


...I guess it's gonna be too complicated to access that from outside :( However, some ops still make sense, such as swapping mapchunks etc :|

Re: How does the SQLite DB work ?

PostPosted: Fri Jun 22, 2018 13:54
by ExeterDad
Take a peek at the python minetest mapper version. It might be written in a way that may help with the "understanding"

Re: How does the SQLite DB work ?

PostPosted: Wed Jul 04, 2018 08:15
by BuckarooBanzay
I implemented a map server in java the last few weeks, maybe you find something useful there:

Parsers:

Re: How does the SQLite DB work ?

PostPosted: Wed Jul 04, 2018 16:04
by LMD
Nice !

Re: How does the SQLite DB work ?

PostPosted: Fri Aug 24, 2018 07:48
by BuckarooBanzay
LMD wrote:Nice !


So, did you make any progress...?

Re: How does the SQLite DB work ?

PostPosted: Fri Aug 24, 2018 12:47
by LMD
I sort of understood it, but am working on other projects.