[Mod] JustTest Database [0.9][jtdb]

[lag01]

Helper library for Minetest mods.
(Mods i used it for are "jtdb" and "names_per_ip")
Simple persistent key/value data storage.

For medium sized datasets - when simply reading/writing files is not enough, but setting up real database is overkill.

License: LGPL 2.1
Dependencies: none

Browse on:GitHub
Download:zip

[rubenwardy]

Your topic title is badly formed, please rename it

Subject of topic must be in one of these formats:

• [Mod] <Mod Name> [<modname>]
• [Mod] <Mod Name> [<version number>] [<modname>]
• [Modpack] <Name>
• [Modpack] <Name> [<version number>]
  eg.
• [Mod] More Blox [0.1] [moreblox]
• [Modpack] The Awesome Modpack

It should be

Code: Select all

[Mod] JustTest Database [0.9][jtdb]


[lag01]

Your topic title is badly formed, please rename it
...
It should be

Code: Select all

[Mod] JustTest Database [0.9][jtdb]


Thank you!

[sorcerykid]

Very impressive! Thanks for sharing these mods from your server. Any chance of also publishing the braziers?

[rnd]

Finally something worthy commenting on. Short description of how it works:

STRUCTURES:
dbfile : containing values, format: each value separate line
idfile : contains positions of value for given key in dbfile, format as key + \n + position of key in dbfile

self.id = hashtable containing all positions of keys; self.id[key] stores position in dbfile where value is stored
actual value is retrieved as:
self.dbfile:seek("set", self.id[key]);local value = self.dbfile:read("*line")

self.cache = dont read value from disk if we already read it before

OPERATION:
writing: just seek to end of dbfile and move position of self.id[key] there and simply append write new value in dbfile (at end)
-occasionaly (1 in 10000) write out idfile (if self.flush enabled actually write to file system)
since we could have many redundant new values maintain function only writes recent one
( at position specified by self.id[key]). this prunes dbfile of all duplicates.
-1 in 10 actually write out dbfile.

reading: given key look up position in dbfile at self.id[key]. Optionally cache the result as self.cache[key] = value

PROBLEMS: if you get crash before 1 in 10000 happens your recent value updates/creations are lost unless you have self.flush enabled. Positions in idfile are updated (append), causing idfile to grow ( maintain fixes this by writing only key/values from pairs(self.id)) - but this wont help since position now points to unflushed dbfile ( non existent position )

[lag01]

Thanks sorcerykid! I definitively want to publish braziers (they are included in my mese_protector mod). After that CleanerPro incident i wanted to double-chek if protectors works as intended, before publishing...

Hi rnd, thank you, i am really flattered to receive such great feedback!
Thanks to pointing to weak point in my logic - all random operations was created only
for maintenance purpose, saving to disk was just side effect and i probably didn't gave it enough attention.
Actually id appends should be flushed the same time values are...
I will double-check, losing unflushed data on crash is only natural, but inconsistent database is not ok.
Thank you again!