by rnd » Mon Sep 17, 2018 07:07
Finally something worthy commenting on. Short description of how it works:
STRUCTURES:
dbfile : containing values, format: each value separate line
idfile : contains positions of value for given key in dbfile, format as key + \n + position of key in dbfile
self.id = hashtable containing all positions of keys; self.id[key] stores position in dbfile where value is stored
actual value is retrieved as:
self.dbfile:seek("set", self.id[key]);local value = self.dbfile:read("*line")
self.cache = dont read value from disk if we already read it before
OPERATION:
writing: just seek to end of dbfile and move position of self.id[key] there and simply append write new value in dbfile (at end)
-occasionaly (1 in 10000) write out idfile (if self.flush enabled actually write to file system)
since we could have many redundant new values maintain function only writes recent one
( at position specified by self.id[key]). this prunes dbfile of all duplicates.
-1 in 10 actually write out dbfile.
reading: given key look up position in dbfile at self.id[key]. Optionally cache the result as self.cache[key] = value
PROBLEMS: if you get crash before 1 in 10000 happens your recent value updates/creations are lost unless you have self.flush enabled. Positions in idfile are updated (append), causing idfile to grow ( maintain fixes this by writing only key/values from pairs(self.id)) - but this wont help since position now points to unflushed dbfile ( non existent position )
If you want to tell people the truth, make them laugh, otherwise they'll kill you. Oscar Wilde