[Mod] JustTest Database [0.9][jtdb]

Post Reply
User avatar
lag01
Member
Posts: 321
Joined: Sun Mar 16, 2014 03:41
GitHub: AndrejIT
IRC: lag01
In-game: lag
Contact:

[Mod] JustTest Database [0.9][jtdb]

by lag01 » Post

Helper library for Minetest mods.
(Mods i used it for are "jtdb" and "names_per_ip")
Simple persistent key/value data storage.

For medium sized datasets - when simply reading/writing files is not enough, but setting up real database is overkill.

License: LGPL 2.1
Dependencies: none

Browse on:GitHub
Download:zip
Last edited by lag01 on Sun Sep 16, 2018 21:19, edited 1 time in total.

User avatar
rubenwardy
Moderator
Posts: 6972
Joined: Tue Jun 12, 2012 18:11
GitHub: rubenwardy
IRC: rubenwardy
In-game: rubenwardy
Location: Bristol, United Kingdom
Contact:

Re: [MOD]jtdb[just test database][0.9]

by rubenwardy » Post

Your topic title is badly formed, please rename it
Subject of topic must be in one of these formats:
  • [Mod] <Mod Name> [<modname>]
  • [Mod] <Mod Name> [<version number>] [<modname>]
  • [Modpack] <Name>
  • [Modpack] <Name> [<version number>]
    eg.
  • [Mod] More Blox [0.1] [moreblox]
  • [Modpack] The Awesome Modpack
It should be

Code: Select all

[Mod] JustTest Database [0.9][jtdb]
Renewed Tab (my browser add-on) | Donate | Mods | Minetest Modding Book

Hello profile reader

User avatar
lag01
Member
Posts: 321
Joined: Sun Mar 16, 2014 03:41
GitHub: AndrejIT
IRC: lag01
In-game: lag
Contact:

Re: [MOD]jtdb[just test database][0.9]

by lag01 » Post

rubenwardy wrote:Your topic title is badly formed, please rename it
...
It should be

Code: Select all

[Mod] JustTest Database [0.9][jtdb]
Thank you!

User avatar
sorcerykid
Member
Posts: 1841
Joined: Fri Aug 26, 2016 15:36
GitHub: sorcerykid
In-game: Nemo
Location: Illinois, USA

Re: [Mod] JustTest Database [0.9][jtdb]

by sorcerykid » Post

Very impressive! Thanks for sharing these mods from your server. Any chance of also publishing the braziers?

User avatar
rnd
Member
Posts: 220
Joined: Sun Dec 28, 2014 12:24
GitHub: ac-minetest
IRC: ac_minetest
In-game: rnd

Re: [Mod] JustTest Database [0.9][jtdb]

by rnd » Post

Finally something worthy commenting on. Short description of how it works:

STRUCTURES:
dbfile : containing values, format: each value separate line
idfile : contains positions of value for given key in dbfile, format as key + \n + position of key in dbfile

self.id = hashtable containing all positions of keys; self.id[key] stores position in dbfile where value is stored
actual value is retrieved as:
self.dbfile:seek("set", self.id[key]);local value = self.dbfile:read("*line")

self.cache = dont read value from disk if we already read it before

OPERATION:
writing: just seek to end of dbfile and move position of self.id[key] there and simply append write new value in dbfile (at end)
-occasionaly (1 in 10000) write out idfile (if self.flush enabled actually write to file system)
since we could have many redundant new values maintain function only writes recent one
( at position specified by self.id[key]). this prunes dbfile of all duplicates.
-1 in 10 actually write out dbfile.

reading: given key look up position in dbfile at self.id[key]. Optionally cache the result as self.cache[key] = value

PROBLEMS: if you get crash before 1 in 10000 happens your recent value updates/creations are lost unless you have self.flush enabled. Positions in idfile are updated (append), causing idfile to grow ( maintain fixes this by writing only key/values from pairs(self.id)) - but this wont help since position now points to unflushed dbfile ( non existent position )
1EvCmxbzl5KDu6XAunE1K853Lq6VVOsT

User avatar
lag01
Member
Posts: 321
Joined: Sun Mar 16, 2014 03:41
GitHub: AndrejIT
IRC: lag01
In-game: lag
Contact:

Re: [Mod] JustTest Database [0.9][jtdb]

by lag01 » Post

Thanks sorcerykid! I definitively want to publish braziers (they are included in my mese_protector mod). After that CleanerPro incident i wanted to double-chek if protectors works as intended, before publishing...

Hi rnd, thank you, i am really flattered to receive such great feedback!
Thanks to pointing to weak point in my logic - all random operations was created only
for maintenance purpose, saving to disk was just side effect and i probably didn't gave it enough attention.
Actually id appends should be flushed the same time values are...
I will double-check, losing unflushed data on crash is only natural, but inconsistent database is not ok.
Thank you again!

Post Reply

Who is online

Users browsing this forum: No registered users and 24 guests