[Solved] Simple non-cryptographic numerical hash in Lua

[texmex]

I need a way to convert a string to a numerical hash. I don't care how simple. I know there's xxhash for Lua but since Lua libraries can't be used in Minetest I need something that can be embedded. Ideas?

[rubenwardy]

minetest.sha1 exists

[texmex]

Yay, didn’t know that. Is it demanding? I won’t be using it in runtime so probably it’s not an issue.

It’s base 16 encoded, yes? From what I understand that can be converted to numbers with tonumber(sha1hash,16)?

[Krock]

Usually hash functions return a human-friendly format as hexadecimals printed to a string. Yes, it's base 16 but it won't fit into the numerical type of Lua (double), so only the first few bytes will be of actual use. With the Minetest sha1 function you can also get the raw bytes, if that's more like what you need.
I use that regularly to generate colors which are quite unique, but persistent over server restarts. Quite helpful.

[texmex]

Even better! It’s not important for me to get the whole sequence, I’ll just use it for predictable uniqueness. Ca you tell me more about your use case, Krock? I’m curious.

[Krock]

I’m curious.

Base image + colourized overlay texture for unique looking items: (unmodified pixels are transparent in the overlay)


Or giving usernames a more or less unique color:

[texmex]

That looks sweet. I'm looking to do this too eventually. Any chance to see this colorization code?

[Tcll]

is murmurhash available??

I noticed the crafting system hashes recipes using murmur_hash_64_ua() hash function
I can't write an lua implementation because only lua 5.3 supports 64bit integers...

it would be nice to have something like minetest.mmh64( "basestring", seed=0xdeadbeef ) that returned the appropriate hash :)

EDIT: also I just want to note SHA1 should be deprecated for at least SHA256 after word finally started spreading of it's poor security a few years ago.
it's been officially insecure for quite a few years now, just nobody's paid any attention until recent.
(how so? different values (sometimes many) return the same hash)

... I know it's just a game, and security isn't really that important... but still
I've been into muh-security for a few years now, and would at least like to see an implentation...

so yeah, hope to see minetest.sha256() soon :)

[rubenwardy]

Security depends on the use case. Sha1 runs the risk of malicious collisions, but that may not matter if you're just generating colored names or inventory hashes

[Tcll]

right, exactly :)
the problem is that's all we're limited to currently

murmur shouldn't be hard to wrap for something better, but sha256 needs to actually be written
but even still, hash functions are decently simple to wright :)

[sorcerykid]

My Simple Cipher mod offers a variety of industry standard hashing functions:

https://github.com/sorcerykid/cipher

[Tcll]

nice job using the slow workaround of formatting numbers to bytes (strings) to get around the 32bit limitation of LUA 5.1 ;D

I'm trying to get the minetest devs to integrate a few of these into faster native C here :)

though on that note, why are we still using LUA 5.1 in 2021 when LUA 5.3 solves a ton of these issues??
minor rant:

Spoiler

that'd be like me using Python 2.7 in 2021 when Python 3.4 is a much better choice

oh and uhh, btw, python has array.array(), which is muuuuuuuuuch faster for doing this same stuff ;)
though I can understand security reasons for not using python over LUA, even with restricted namespaces, despite having much more functionality and speed...
(I'm dealing with my own numerous issues here, though at least I hide your IP)

EDIT: btw, I've found I actually don't need these hashes for my issues
LUA can hash anything in tables... which leads me to ask why we haven't deprecated SHA1??

[sorcerykid]

nice job using the slow workaround of formatting numbers to bytes (strings) to get around the 32bit limitation of LUA 5.1 ;D

Evidently you are new to Lua, because 5.1 uses 64-bit doubles for all numbers.

Moreover, both the Adler32 algorithm and the Fletcher32 algorithm literally operate on character strings (i.e. bytes) when written in C. So I'm not "getting around" any limitation. You really need to do a bit more research before coming into these Minetest forums acting so arrogant and smug.

[Tcll]

I wasn't trying to be arrogant and smug (a little self-righteous maybe, but nothing offensive), I'm sorry that comment sounded as such to you... :/
I'd gotten my info from these (primary) sources: (there were others I'd derrived info from as well)
https://stackoverflow.com/questions/310 ... t-integers
https://github.com/tst2005/lua-bit64/bl ... /bit64.lua
https://gist.github.com/tkaemming/25dfcd4998c470d630e1
as well as numerous individuals (which btw, someone mentioned LUA runs off 32bit floats, though I know for certain 64bit was introduced in 5.3)

yes I'm still a bit new to LUA
but I don't speak out unless I have at least some knowledge of what I'm talking about

again, I'm sorry if I sounded offensive, it wasn't my intent.

EDIT: btw, jsyk, if I had the option, I would've given your post an upvote before making that comment.
I'm glad to see more work being done to bring in more functionality, despite the shortcomings of LUA