[Mod] JustTest Database [0.9][jtdb]

User avatar
lag01
Member
 
Posts: 284
Joined: Sun Mar 16, 2014 03:41
GitHub: AndrejIT
IRC: lag01
In-game: lag

[Mod] JustTest Database [0.9][jtdb]

by lag01 » Sun Sep 16, 2018 07:18

Helper library for Minetest mods.
(Mods i used it for are "jtdb" and "names_per_ip")
Simple persistent key/value data storage.

For medium sized datasets - when simply reading/writing files is not enough, but setting up real database is overkill.

License: LGPL 2.1
Dependencies: none

Browse on:GitHub
Download:zip
Last edited by lag01 on Sun Sep 16, 2018 21:19, edited 1 time in total.
 

User avatar
rubenwardy
Moderator
 
Posts: 5455
Joined: Tue Jun 12, 2012 18:11
Location: United Kingdom
GitHub: rubenwardy
IRC: rubenwardy
In-game: rubenwardy

Re: [MOD]jtdb[just test database][0.9]

by rubenwardy » Sun Sep 16, 2018 21:09

Your topic title is badly formed, please rename it

Subject of topic must be in one of these formats:
  • [Mod] <Mod Name> [<modname>]
  • [Mod] <Mod Name> [<version number>] [<modname>]
  • [Modpack] <Name>
  • [Modpack] <Name> [<version number>]
    eg.
  • [Mod] More Blox [0.1] [moreblox]
  • [Modpack] The Awesome Modpack


It should be

Code: Select all
[Mod] JustTest Database [0.9][jtdb]
 

User avatar
lag01
Member
 
Posts: 284
Joined: Sun Mar 16, 2014 03:41
GitHub: AndrejIT
IRC: lag01
In-game: lag

Re: [MOD]jtdb[just test database][0.9]

by lag01 » Sun Sep 16, 2018 21:20

rubenwardy wrote:Your topic title is badly formed, please rename it
...
It should be

Code: Select all
[Mod] JustTest Database [0.9][jtdb]


Thank you!
 

User avatar
sorcerykid
Member
 
Posts: 835
Joined: Fri Aug 26, 2016 15:36
Location: Illinois, USA
GitHub: sorcerykid
In-game: Nemo

Re: [Mod] JustTest Database [0.9][jtdb]

by sorcerykid » Mon Sep 17, 2018 04:32

Very impressive! Thanks for sharing these mods from your server. Any chance of also publishing the braziers?
 

User avatar
rnd
Member
 
Posts: 201
Joined: Sun Dec 28, 2014 12:24
IRC: ac_minetest
In-game: rnd

Re: [Mod] JustTest Database [0.9][jtdb]

by rnd » Mon Sep 17, 2018 07:07

Finally something worthy commenting on. Short description of how it works:

STRUCTURES:
dbfile : containing values, format: each value separate line
idfile : contains positions of value for given key in dbfile, format as key + \n + position of key in dbfile

self.id = hashtable containing all positions of keys; self.id[key] stores position in dbfile where value is stored
actual value is retrieved as:
self.dbfile:seek("set", self.id[key]);local value = self.dbfile:read("*line")

self.cache = dont read value from disk if we already read it before

OPERATION:
writing: just seek to end of dbfile and move position of self.id[key] there and simply append write new value in dbfile (at end)
-occasionaly (1 in 10000) write out idfile (if self.flush enabled actually write to file system)
since we could have many redundant new values maintain function only writes recent one
( at position specified by self.id[key]). this prunes dbfile of all duplicates.
-1 in 10 actually write out dbfile.

reading: given key look up position in dbfile at self.id[key]. Optionally cache the result as self.cache[key] = value

PROBLEMS: if you get crash before 1 in 10000 happens your recent value updates/creations are lost unless you have self.flush enabled. Positions in idfile are updated (append), causing idfile to grow ( maintain fixes this by writing only key/values from pairs(self.id)) - but this wont help since position now points to unflushed dbfile ( non existent position )
 

User avatar
lag01
Member
 
Posts: 284
Joined: Sun Mar 16, 2014 03:41
GitHub: AndrejIT
IRC: lag01
In-game: lag

Re: [Mod] JustTest Database [0.9][jtdb]

by lag01 » Mon Sep 17, 2018 11:30

Thanks sorcerykid! I definitively want to publish braziers (they are included in my mese_protector mod). After that CleanerPro incident i wanted to double-chek if protectors works as intended, before publishing...

Hi rnd, thank you, i am really flattered to receive such great feedback!
Thanks to pointing to weak point in my logic - all random operations was created only
for maintenance purpose, saving to disk was just side effect and i probably didn't gave it enough attention.
Actually id appends should be flushed the same time values are...
I will double-check, losing unflushed data on crash is only natural, but inconsistent database is not ok.
Thank you again!
 


Return to WIP Mods



Who is online

Users browsing this forum: Bing [Bot] and 1 guest