How does the SQLite DB work ?

For people working on the C++ code.
User avatar
LMD
Member
 
Posts: 682
Joined: Sat Apr 08, 2017 08:16
Location: Somewhere over the Rainbow : Bonn, Germany
GitHub: appgurueu
In-game: LMD + PRO_LMD + Limo

How does the SQLite DB work ?

by LMD » Fri Jun 22, 2018 09:42

A question to you, core developers :

After working with SQLite(for MTPC), I have learned from my mistakes, and am feeling quite safe and have learned much. Now, I'm asking myself whether it would be good to modify the SQLite DB from outside(Java) to make map ops not in LUA, but from outside program; more specific, I want to have - Java program which creates Schematics from 3d models -> writes them directly into map.
So, I know how it basically works : You got indices & blobs. Each indice(int) stores blob(chunk) position in map and is primary key. Blob is mapchunk(which size ? 80 ?). How do indices work(which order ? x+y*mapsize+z*mapsize² ?). And now, the final question : How do the binary large objects work ? Of course, every node has content ID. So, are you just storing the content IDs in giant array, indiced by x+y*chunksize+z*chunksize² ? Where's the metadata ?
Magic-CTF(soon)MTPCFractal-SuiteModel Creator Schematic CreatorColorfulXmas BallsWebsite • Never judge me by my age
 

User avatar
ExeterDad
Member
 
Posts: 1717
Joined: Sun Jun 01, 2014 20:00
Location: New Hampshire U.S.A
In-game: ExeterDad

Re: How does the SQLite DB work ?

by ExeterDad » Fri Jun 22, 2018 10:33

I did a casual read of this just the other day. I believe what you are looking for is documented here:
https://github.com/minetest/minetest/bl ... format.txt
 

User avatar
LMD
Member
 
Posts: 682
Joined: Sat Apr 08, 2017 08:16
Location: Somewhere over the Rainbow : Bonn, Germany
GitHub: appgurueu
In-game: LMD + PRO_LMD + Limo

Re: How does the SQLite DB work ?

by LMD » Fri Jun 22, 2018 11:14

Thanks ! Exactly what I was searching for !
This answers my indexing question :
Code: Select all
The key
--------
"pos" is created from the three coordinates of a MapBlock using this
algorithm, defined here in Python:

  def getBlockAsInteger(p):
      return int64(p[2]*16777216 + p[1]*4096 + p[0])

  def int64(u):
      while u >= 2**63:
          u -= 2**64
      while u <= -2**63:
          u += 2**64
      return u

It can be converted the other way by using this code:

  def getIntegerAsBlock(i):
      x = unsignedToSigned(i % 4096, 2048)
      i = int((i - x) / 4096)
      y = unsignedToSigned(i % 4096, 2048)
      i = int((i - y) / 4096)
      z = unsignedToSigned(i % 4096, 2048)
      return x,y,z

  def unsignedToSigned(i, max_positive):
      if i < max_positive:
          return i
      else:
return i - 2*max_positive


Blob/Mapchunk size : 16x16

Giant array node index :
Code: Select all
- The location of a node in each of those arrays is (z*16*16 + y*16 + x).


...I guess it's gonna be too complicated to access that from outside :( However, some ops still make sense, such as swapping mapchunks etc :|
Magic-CTF(soon)MTPCFractal-SuiteModel Creator Schematic CreatorColorfulXmas BallsWebsite • Never judge me by my age
 

User avatar
ExeterDad
Member
 
Posts: 1717
Joined: Sun Jun 01, 2014 20:00
Location: New Hampshire U.S.A
In-game: ExeterDad

Re: How does the SQLite DB work ?

by ExeterDad » Fri Jun 22, 2018 13:54

Take a peek at the python minetest mapper version. It might be written in a way that may help with the "understanding"
 

BuckarooBanzay
Member
 
Posts: 122
Joined: Tue Apr 24, 2018 05:58
Location: Switzerland
GitHub: thomasrudin-mt
In-game: BuckarooBanzai
 

User avatar
LMD
Member
 
Posts: 682
Joined: Sat Apr 08, 2017 08:16
Location: Somewhere over the Rainbow : Bonn, Germany
GitHub: appgurueu
In-game: LMD + PRO_LMD + Limo
 

BuckarooBanzay
Member
 
Posts: 122
Joined: Tue Apr 24, 2018 05:58
Location: Switzerland
GitHub: thomasrudin-mt
In-game: BuckarooBanzai
 

User avatar
LMD
Member
 
Posts: 682
Joined: Sat Apr 08, 2017 08:16
Location: Somewhere over the Rainbow : Bonn, Germany
GitHub: appgurueu
In-game: LMD + PRO_LMD + Limo

Re: How does the SQLite DB work ?

by LMD » Fri Aug 24, 2018 12:47

I sort of understood it, but am working on other projects.
Magic-CTF(soon)MTPCFractal-SuiteModel Creator Schematic CreatorColorfulXmas BallsWebsite • Never judge me by my age
 


Return to Partly official engine development



Who is online

Users browsing this forum: No registered users and 0 guests