DB Class

For people working on the C++ code.
Post Reply
User avatar
wieszak
Member
Posts: 40
Joined: Mon Feb 20, 2012 17:00

DB Class

by wieszak » Post

Hello.

I decided to make some maybe small but needed for my next step change to code. I moved all database function to class Database, and create class Database_SQLite3 as implementation to currently used sqlite3. Howewer run into small problem - i need in this class link to ServeMap which is awful... Maybe you are able to patch it further to remove this. But i want to keep current API - functions from class Database takes and returns MapBlock - i want to keep serialization in this class, so i could later use another style of serialization for another db. My next step is to provide MySQL implementation and test if it would behave better for multiplayer servers.

Here is the patch against current 0.4.3 http://para.zonk.pl/patch-db.diff

If you like it maybe include this into mainstream?

User avatar
cornernote
Member
Posts: 844
Joined: Wed Jul 11, 2012 15:02

by cornernote » Post

+1 for a mysql option

User avatar
wieszak
Member
Posts: 40
Joined: Mon Feb 20, 2012 17:00

by wieszak » Post

Ok, i managed to run server with mysql. Unfortunately now database credentials are hardcoded, so i need to find out how to move them to config files and some similiar stuff. First impressions (as singleplayer) - i have feeleing, that server runs more smoothly: with sqlite i have some short hungups then and then, with mysql there is none. But maybe i need play longer... Anyway, there is some work to do before i realese full patch, and of course i'm not dropping support for sqlite, i plan leave it as default especially for single player - it is much easier to use for enduser.

User avatar
cornernote
Member
Posts: 844
Joined: Wed Jul 11, 2012 15:02

by cornernote » Post

Nice, I'm keen to have a play with it.

User avatar
wieszak
Member
Posts: 40
Joined: Mon Feb 20, 2012 17:00

by wieszak » Post

Ok. Here it is first 'public' version of patch for mysql support:

http://para.zonk.pl/mysql-0.1.1.patch

Server is sqlite3 by default, to use mysql you need to put in minetest.conf values for mysql_server, mysql_user, mysql_password and mysql_database. Database need to be created, tables are created by server, user need to get select, update, insert, delete and create priviliges. Each worlds data is located in table <world_name>_blocks, so better do not use fancy characters in name - it is not checked yet ;)

In few (or more...) days i will announce 24/7 server running on mysql with this patch for multiplayer testing.

TODO:
- maybe move config data to world config not server so user could choose storage at world creation...
- maybe move worlds metadata and players to mysql also
- change serialization for mysql - it would be helpful for next:
- write some external online tools to admin world ;)
Last edited by wieszak on Mon Sep 24, 2012 19:45, edited 1 time in total.

User avatar
rubenwardy
Moderator
Posts: 6271
Joined: Tue Jun 12, 2012 18:11
GitHub: rubenwardy
IRC: rubenwardy
In-game: rubenwardy
Location: United Kingdom
Contact:

by rubenwardy » Post

+1

Two questions:

How do you install/compile/build/add the changes in the above link?

Do you need to installing anything (ie: MySQL) to run in MySQL version?

User avatar
xyz
Member
Posts: 450
Joined: Thu Nov 10, 2011 14:25

by xyz » Post

wieszak wrote:Ok. Here it is first 'public' version of patch for mysql support:

http://para.zonk.pl/mysql-0.1.patch

Server is sqlite3 by default, to use mysql you need to put in minetest.conf values for mysql_server, mysql_user, mysql_password and mysql_database. Database need to be created, tables are created by server, user need to get select, update, insert, delete and create priviliges. Each worlds data is located in table <world_name>_blocks, so better do not use fancy characters in name - it is not checked yet ;)

In few (or more...) days i will announce 24/7 server running on mysql with this patch for multiplayer testing.

TODO:
- maybe move config data to world config not server so user could choose storage at world creation...
- maybe move worlds metadata and players to mysql also
- change serialization for mysql - it would be helpful for next:
- write some external online tools to admin world ;)
Nice.
BTW, line 260 of your patch seems to be wrong

Code: Select all

+        if (mysql_select_db(m_database,"minetest")) {
And there is also no FindMySQL.cmake

It works, but I can't determine whether it works faster or not.

User avatar
wieszak
Member
Posts: 40
Joined: Mon Feb 20, 2012 17:00

by wieszak » Post

Yes, this line is even unnecessary, removed from patch (link edited)

FindMySQL.cmake i've got from mysql package. I know that it is not standart module for cmake, but it is not related witch patch itself. I will provide link to donwload it as soon as i remember from where i exectly got it.

For me is slighty faster but in singleplayer. Will see if it makes any differences in multiplayer when i (or somebody) setup server for people to test...

rubenwardy:
- download sources and patch, unpack sources
- enter mintetest directory
- apply patch with : patch -p1 </somewhere/mysql-0.1.1.patch
- compile minetest as in minetest's readme

If problems i see you sometimes at redcrabs server, ask me there (i'm smoki) or here .
Patch requires at least mysql client libraries and mysql headers to compile, then any mysql server to connect.

User avatar
fgr
Member
Posts: 86
Joined: Mon Sep 17, 2012 08:22

by fgr » Post

Hi,

is this still in development? I am searching a simple function to store events into a mysql db but i rather wont change the whole minetest backend db from a sqlite to mysql if there is no benefit on performance...

Dragonop
Member
Posts: 1179
Joined: Tue Oct 23, 2012 12:59
GitHub: Dragonop
IRC: Dragonop
In-game: Dragonop
Location: Argentina

by Dragonop » Post

what is this mod adds
If you have any pending matters with me, I encourage you to contact me via e-mail or PM. I still lurk from the shadows and very much care about the Minetest project. Sadly I have not been involved for the past years but I still look back fondly on the time I spent in your company.
If you are checking my profile, it's probably because of something.

It's "Drag" for the friends.

download/file.php?id=4472

User avatar
Alfino
Member
Posts: 32
Joined: Sun Sep 16, 2012 12:11

by Alfino » Post

Dragonop wrote:what is this mod adds
this isn't mods

User avatar
fgr
Member
Posts: 86
Joined: Mon Sep 17, 2012 08:22

by fgr » Post

any hint how to change the backend db in config? just still patched it and the database is still sqlite... no readme nor anything else, could you provide some more infos to it?

also i get the followin error:

Code: Select all

patching file src/CMakeLists.txt
patching file src/client.cpp
patching file src/clientserver.h
patching file src/database-mysql.cpp
patch: **** malformed patch at line 571: diff -ruN celeron55-minetest-9cadaf8/src/database.cpp celeron55-minetest-9cadaf8-mysql/src/database.cpp
any clue?

thanks
Last edited by fgr on Wed Oct 31, 2012 20:25, edited 1 time in total.

User avatar
xyz
Member
Posts: 450
Joined: Thu Nov 10, 2011 14:25

by xyz » Post

fgr wrote:any hint how to change the backend db in config? just still patched it and the database is still sqlite... no readme nor anything else, could you provide some more infos to it?

also i get the followin error:

Code: Select all

patch: **** malformed patch at line 571: diff -ruN celeron55-minetest-9cadaf8/src/database.cpp celeron55-minetest-9cadaf8-mysql/src/database.cpp
any clue?

thanks
I've already included that patch with two more backends (dummy and leveldb) into my db_backends branch

User avatar
fgr
Member
Posts: 86
Joined: Mon Sep 17, 2012 08:22

by fgr » Post

ho thats nice, so i need only compile it and create the mysql db, or does it by its own (the db)? because if i run the minetestserver (even in the config i postet the right mysql settings) there will no mysql db be used....???? The server creates the sqlite db as usual....

Code: Select all

 
Server creating detached inventory "creative"
creative inventory size: 111
21:46:19: INFO[ServerThread]: ServerMap: SQLite3 database structure was createdServerMap: SQLite3 database opened
Last edited by fgr on Wed Oct 31, 2012 20:56, edited 1 time in total.

User avatar
wieszak
Member
Posts: 40
Joined: Mon Feb 20, 2012 17:00

by wieszak » Post

Sory, been off some time.

fgr: default is still sqlite3, mysql is used when in minetest.conf you provide database credential - it is described in post with patch. However i dont know how xyz it did, need to look into it.

User avatar
Pitriss
Member
Posts: 254
Joined: Mon Aug 05, 2013 17:09
GitHub: Pitriss
IRC: pitriss
In-game: pitriss
Location: Czech republic, Bohumin

by Pitriss » Post

i want to ask, is there any progress in this patch? can be applied on latest client?
I reject your reality and substitute my own. (A. Savage, Mythbusters)
I'm not modding and/or playing minetest anymore. All my mods were released under WTFPL. You can fix/modify them yourself. Don't ask me for support. Thanks.

User avatar
fgr
Member
Posts: 86
Joined: Mon Sep 17, 2012 08:22

by fgr » Post

why isnt this considered anymore? is there any possibility to get short attention to it? thanks a lot!!!!!!

User avatar
xyz
Member
Posts: 450
Joined: Thu Nov 10, 2011 14:25

by xyz » Post

fgr wrote:why isnt this considered anymore? is there any possibility to get short attention to it? thanks a lot!!!!!!
It's merged already. MySQL backend isn't present though.

User avatar
fgr
Member
Posts: 86
Joined: Mon Sep 17, 2012 08:22

by fgr » Post

xyz wrote:
fgr wrote:why isnt this considered anymore? is there any possibility to get short attention to it? thanks a lot!!!!!!
It's merged already. MySQL backend isn't present though.
So it's not fully integrated because of different opinions about mysql. The game will probably have lack of speed when lot of mods loaded. This wont get better with leveldb.

User avatar
xyz
Member
Posts: 450
Joined: Thu Nov 10, 2011 14:25

by xyz » Post

fgr wrote:
xyz wrote:
fgr wrote:why isnt this considered anymore? is there any possibility to get short attention to it? thanks a lot!!!!!!
It's merged already. MySQL backend isn't present though.
So it's not fully integrated because of different opinions about mysql. The game will probably have lack of speed when lot of mods loaded. This wont get better with leveldb.
I don't see how this is related to not having MySQL backend.

User avatar
aldobr
Member
Posts: 316
Joined: Sun Nov 25, 2012 05:46

by aldobr » Post

Have you ever tought about NoSQL ?

User avatar
lightonflux
Member
Posts: 384
Joined: Mon Nov 11, 2013 07:22
In-game: lof
Location: Germany

by lightonflux » Post

What is the current state of your MySQL support? Is it dead or just stagnating?

User avatar
sfan5
Moderator
Posts: 3933
Joined: Wed Aug 24, 2011 09:44
GitHub: sfan5
IRC: sfan5
Location: Germany

by sfan5 » Post

You could use lua to communicate between Minetest and MySQL (or any other way of saving things).
1) Compile this branch
2) Use the lua backend in world.mt
3) Make the database_lua_scriptfile setting point at a Lua file that provides the following functions:

Code: Select all

function db_init()
    --Initialize database
end

function db_beginSave()
    --Is called before multiple transactions take place
end

function db_endSave()
    --Is called after multiple transactions have taken place
end

function db_saveBlock(x, y, z, blobdata)
    --Save MapBlock
end

function db_loadBlock(x, y, z)
    --Load MapBlock (return string)
    -- "" means the MapBlock could not be loaded (e.g. Not existant, Error occurred)
end

function db_listAllLoadableBlocks()
    --Return list of positions of all existing MapBlocks
end

function db_destroy()
    --Close database
end
Last edited by sfan5 on Tue Mar 11, 2014 17:41, edited 1 time in total.
Mods: Mesecons | WorldEdit | Nuke & Minetest builds for Windows (32-bit & 64-bit)

User avatar
twoelk
Member
Posts: 1388
Joined: Fri Apr 19, 2013 16:19
GitHub: twoelk
IRC: twoelk
In-game: twoelk
Location: northern Germany

by twoelk » Post

At some time in the future an open db api that allows easy coding of plugins to use any database out there now and maybe to come in the future? That is if Lua isn't allready sufficient for the job.

User avatar
Johnny Joy
Member
Posts: 55
Joined: Fri Sep 05, 2014 20:26
GitHub: johnnyjoy
In-game: jjb

Re: DB Class

by Johnny Joy » Post

This is a great patch. What ever happened to it? Why did it not progress to the next level?

Post Reply

Who is online

Users browsing this forum: No registered users and 1 guest