[Solved] Simple non-cryptographic numerical hash in Lua

Post Reply
User avatar
texmex
Member
Posts: 1752
Joined: Mon Jul 11, 2016 21:08
GitHub: tacotexmex
In-game: tacotexmex

[Solved] Simple non-cryptographic numerical hash in Lua

by texmex » Post

I need a way to convert a string to a numerical hash. I don't care how simple. I know there's xxhash for Lua but since Lua libraries can't be used in Minetest I need something that can be embedded. Ideas?
Last edited by texmex on Sun Oct 06, 2019 22:23, edited 1 time in total.
Mods | Support Mesehub: bc1qluuests9rxmlnvpjrhsnyjg9ucwy6z3r0y3srw

User avatar
rubenwardy
Moderator
Posts: 6424
Joined: Tue Jun 12, 2012 18:11
GitHub: rubenwardy
IRC: rubenwardy
In-game: rubenwardy
Location: United Kingdom
Contact:

Re: Simple non-cryptographic numerical hash in Lua

by rubenwardy » Post

minetest.sha1 exists

User avatar
texmex
Member
Posts: 1752
Joined: Mon Jul 11, 2016 21:08
GitHub: tacotexmex
In-game: tacotexmex

Re: Simple non-cryptographic numerical hash in Lua

by texmex » Post

Yay, didn’t know that. Is it demanding? I won’t be using it in runtime so probably it’s not an issue.

It’s base 16 encoded, yes? From what I understand that can be converted to numbers with tonumber(sha1hash,16)?
Mods | Support Mesehub: bc1qluuests9rxmlnvpjrhsnyjg9ucwy6z3r0y3srw

User avatar
Krock
Developer
Posts: 4587
Joined: Thu Oct 03, 2013 07:48
GitHub: SmallJoker
Location: Switzerland
Contact:

Re: [Solved] Simple non-cryptographic numerical hash in Lua

by Krock » Post

Usually hash functions return a human-friendly format as hexadecimals printed to a string. Yes, it's base 16 but it won't fit into the numerical type of Lua (double), so only the first few bytes will be of actual use. With the Minetest sha1 function you can also get the raw bytes, if that's more like what you need.
I use that regularly to generate colors which are quite unique, but persistent over server restarts. Quite helpful.
Look, I programmed a bug for you. >> Mod Search Engine << - Mods by Krock - DuckDuckGo mod search bang: !mtmod <keyword here>

User avatar
texmex
Member
Posts: 1752
Joined: Mon Jul 11, 2016 21:08
GitHub: tacotexmex
In-game: tacotexmex

Re: [Solved] Simple non-cryptographic numerical hash in Lua

by texmex » Post

Even better! It’s not important for me to get the whole sequence, I’ll just use it for predictable uniqueness. Ca you tell me more about your use case, Krock? I’m curious.
Mods | Support Mesehub: bc1qluuests9rxmlnvpjrhsnyjg9ucwy6z3r0y3srw

User avatar
Krock
Developer
Posts: 4587
Joined: Thu Oct 03, 2013 07:48
GitHub: SmallJoker
Location: Switzerland
Contact:

Re: [Solved] Simple non-cryptographic numerical hash in Lua

by Krock » Post

texmex wrote:I’m curious.
Base image + colourized overlay texture for unique looking items: (unmodified pixels are transparent in the overlay)
Image

Or giving usernames a more or less unique color:
Image
Attachments
usecase.png
(16.18 KiB) Not downloaded yet
Look, I programmed a bug for you. >> Mod Search Engine << - Mods by Krock - DuckDuckGo mod search bang: !mtmod <keyword here>

User avatar
texmex
Member
Posts: 1752
Joined: Mon Jul 11, 2016 21:08
GitHub: tacotexmex
In-game: tacotexmex

Re: [Solved] Simple non-cryptographic numerical hash in Lua

by texmex » Post

That looks sweet. I'm looking to do this too eventually. Any chance to see this colorization code?
Mods | Support Mesehub: bc1qluuests9rxmlnvpjrhsnyjg9ucwy6z3r0y3srw

User avatar
Tcll
Member
Posts: 79
Joined: Thu Jul 25, 2019 21:43
GitHub: Tcll
IRC: Tcll Tcll5850 DarkPikachu
In-game: Tcll
Location: The Gates of Darkness.

Re: [Solved] Simple non-cryptographic numerical hash in Lua

by Tcll » Post

is murmurhash available??

I noticed the crafting system hashes recipes using murmur_hash_64_ua() hash function
I can't write an lua implementation because only lua 5.3 supports 64bit integers...

it would be nice to have something like minetest.mmh64( "basestring", seed=0xdeadbeef ) that returned the appropriate hash :)

EDIT: also I just want to note SHA1 should be deprecated for at least SHA256 after word finally started spreading of it's poor security a few years ago.
it's been officially insecure for quite a few years now, just nobody's paid any attention until recent.
(how so? different values (sometimes many) return the same hash)

... I know it's just a game, and security isn't really that important... but still
I've been into muh-security for a few years now, and would at least like to see an implentation...

so yeah, hope to see minetest.sha256() soon :)

User avatar
rubenwardy
Moderator
Posts: 6424
Joined: Tue Jun 12, 2012 18:11
GitHub: rubenwardy
IRC: rubenwardy
In-game: rubenwardy
Location: United Kingdom
Contact:

Re: [Solved] Simple non-cryptographic numerical hash in Lua

by rubenwardy » Post

Security depends on the use case. Sha1 runs the risk of malicious collisions, but that may not matter if you're just generating colored names or inventory hashes

User avatar
Tcll
Member
Posts: 79
Joined: Thu Jul 25, 2019 21:43
GitHub: Tcll
IRC: Tcll Tcll5850 DarkPikachu
In-game: Tcll
Location: The Gates of Darkness.

Re: [Solved] Simple non-cryptographic numerical hash in Lua

by Tcll » Post

right, exactly :)
the problem is that's all we're limited to currently

murmur shouldn't be hard to wrap for something better, but sha256 needs to actually be written
but even still, hash functions are decently simple to wright :)

User avatar
sorcerykid
Member
Posts: 1537
Joined: Fri Aug 26, 2016 15:36
GitHub: sorcerykid
In-game: Nemo
Location: Illinois, USA

Re: [Solved] Simple non-cryptographic numerical hash in Lua

by sorcerykid » Post

My Simple Cipher mod offers a variety of industry standard hashing functions:

https://github.com/sorcerykid/cipher

User avatar
Tcll
Member
Posts: 79
Joined: Thu Jul 25, 2019 21:43
GitHub: Tcll
IRC: Tcll Tcll5850 DarkPikachu
In-game: Tcll
Location: The Gates of Darkness.

Re: [Solved] Simple non-cryptographic numerical hash in Lua

by Tcll » Post

nice job using the slow workaround of formatting numbers to bytes (strings) to get around the 32bit limitation of LUA 5.1 ;D

I'm trying to get the minetest devs to integrate a few of these into faster native C here :)

though on that note, why are we still using LUA 5.1 in 2021 when LUA 5.3 solves a ton of these issues??
minor rant:
+ Spoiler
EDIT: btw, I've found I actually don't need these hashes for my issues
LUA can hash anything in tables... which leads me to ask why we haven't deprecated SHA1??

User avatar
sorcerykid
Member
Posts: 1537
Joined: Fri Aug 26, 2016 15:36
GitHub: sorcerykid
In-game: Nemo
Location: Illinois, USA

Re: [Solved] Simple non-cryptographic numerical hash in Lua

by sorcerykid » Post

Tcll wrote:
Tue Feb 09, 2021 12:02
nice job using the slow workaround of formatting numbers to bytes (strings) to get around the 32bit limitation of LUA 5.1 ;D
Evidently you are new to Lua, because 5.1 uses 64-bit doubles for all numbers.

Moreover, both the Adler32 algorithm and the Fletcher32 algorithm literally operate on character strings (i.e. bytes) when written in C. So I'm not "getting around" any limitation. You really need to do a bit more research before coming into these Minetest forums acting so arrogant and smug.

User avatar
Tcll
Member
Posts: 79
Joined: Thu Jul 25, 2019 21:43
GitHub: Tcll
IRC: Tcll Tcll5850 DarkPikachu
In-game: Tcll
Location: The Gates of Darkness.

Re: [Solved] Simple non-cryptographic numerical hash in Lua

by Tcll » Post

I wasn't trying to be arrogant and smug (a little self-righteous maybe, but nothing offensive), I'm sorry that comment sounded as such to you... :/
I'd gotten my info from these (primary) sources: (there were others I'd derrived info from as well)
https://stackoverflow.com/questions/310 ... t-integers
https://github.com/tst2005/lua-bit64/bl ... /bit64.lua
https://gist.github.com/tkaemming/25dfcd4998c470d630e1
as well as numerous individuals (which btw, someone mentioned LUA runs off 32bit floats, though I know for certain 64bit was introduced in 5.3)

yes I'm still a bit new to LUA
but I don't speak out unless I have at least some knowledge of what I'm talking about

again, I'm sorry if I sounded offensive, it wasn't my intent.

EDIT: btw, jsyk, if I had the option, I would've given your post an upvote before making that comment.
I'm glad to see more work being done to bring in more functionality, despite the shortcomings of LUA

Post Reply

Who is online

Users browsing this forum: No registered users and 1 guest