[Solved] Simple non-cryptographic numerical hash in Lua
- texmex
- Member
- Posts: 1753
- Joined: Mon Jul 11, 2016 21:08
- GitHub: tacotexmex
- In-game: tacotexmex
[Solved] Simple non-cryptographic numerical hash in Lua
I need a way to convert a string to a numerical hash. I don't care how simple. I know there's xxhash for Lua but since Lua libraries can't be used in Minetest I need something that can be embedded. Ideas?
Last edited by texmex on Sun Oct 06, 2019 22:23, edited 1 time in total.
- rubenwardy
- Moderator
- Posts: 6978
- Joined: Tue Jun 12, 2012 18:11
- GitHub: rubenwardy
- IRC: rubenwardy
- In-game: rubenwardy
- Location: Bristol, United Kingdom
- Contact:
Re: Simple non-cryptographic numerical hash in Lua
minetest.sha1 exists
- texmex
- Member
- Posts: 1753
- Joined: Mon Jul 11, 2016 21:08
- GitHub: tacotexmex
- In-game: tacotexmex
Re: Simple non-cryptographic numerical hash in Lua
Yay, didn’t know that. Is it demanding? I won’t be using it in runtime so probably it’s not an issue.
It’s base 16 encoded, yes? From what I understand that can be converted to numbers with tonumber(sha1hash,16)?
It’s base 16 encoded, yes? From what I understand that can be converted to numbers with tonumber(sha1hash,16)?
- Krock
- Developer
- Posts: 4650
- Joined: Thu Oct 03, 2013 07:48
- GitHub: SmallJoker
- Location: Switzerland
- Contact:
Re: [Solved] Simple non-cryptographic numerical hash in Lua
Usually hash functions return a human-friendly format as hexadecimals printed to a string. Yes, it's base 16 but it won't fit into the numerical type of Lua (double), so only the first few bytes will be of actual use. With the Minetest sha1 function you can also get the raw bytes, if that's more like what you need.
I use that regularly to generate colors which are quite unique, but persistent over server restarts. Quite helpful.
I use that regularly to generate colors which are quite unique, but persistent over server restarts. Quite helpful.
Look, I programmed a bug for you. >> Mod Search Engine << - Mods by Krock - DuckDuckGo mod search bang: !mtmod <keyword here>
- texmex
- Member
- Posts: 1753
- Joined: Mon Jul 11, 2016 21:08
- GitHub: tacotexmex
- In-game: tacotexmex
Re: [Solved] Simple non-cryptographic numerical hash in Lua
Even better! It’s not important for me to get the whole sequence, I’ll just use it for predictable uniqueness. Ca you tell me more about your use case, Krock? I’m curious.
- Krock
- Developer
- Posts: 4650
- Joined: Thu Oct 03, 2013 07:48
- GitHub: SmallJoker
- Location: Switzerland
- Contact:
Re: [Solved] Simple non-cryptographic numerical hash in Lua
Base image + colourized overlay texture for unique looking items: (unmodified pixels are transparent in the overlay)texmex wrote:I’m curious.
Or giving usernames a more or less unique color:
- Attachments
-
- usecase.png (16.18 KiB) Viewed 2195 times
Look, I programmed a bug for you. >> Mod Search Engine << - Mods by Krock - DuckDuckGo mod search bang: !mtmod <keyword here>
- texmex
- Member
- Posts: 1753
- Joined: Mon Jul 11, 2016 21:08
- GitHub: tacotexmex
- In-game: tacotexmex
Re: [Solved] Simple non-cryptographic numerical hash in Lua
That looks sweet. I'm looking to do this too eventually. Any chance to see this colorization code?
- Tcll
- Member
- Posts: 87
- Joined: Thu Jul 25, 2019 21:43
- GitHub: Tcll
- IRC: Tcll Tcll5850 DarkPikachu
- In-game: Tcll
- Location: The Gates of Darkness.
Re: [Solved] Simple non-cryptographic numerical hash in Lua
is murmurhash available??
I noticed the crafting system hashes recipes using murmur_hash_64_ua() hash function
I can't write an lua implementation because only lua 5.3 supports 64bit integers...
it would be nice to have something like minetest.mmh64( "basestring", seed=0xdeadbeef ) that returned the appropriate hash :)
EDIT: also I just want to note SHA1 should be deprecated for at least SHA256 after word finally started spreading of it's poor security a few years ago.
it's been officially insecure for quite a few years now, just nobody's paid any attention until recent.
(how so? different values (sometimes many) return the same hash)
... I know it's just a game, and security isn't really that important... but still
I've been into muh-security for a few years now, and would at least like to see an implentation...
so yeah, hope to see minetest.sha256() soon :)
I noticed the crafting system hashes recipes using murmur_hash_64_ua() hash function
I can't write an lua implementation because only lua 5.3 supports 64bit integers...
it would be nice to have something like minetest.mmh64( "basestring", seed=0xdeadbeef ) that returned the appropriate hash :)
EDIT: also I just want to note SHA1 should be deprecated for at least SHA256 after word finally started spreading of it's poor security a few years ago.
it's been officially insecure for quite a few years now, just nobody's paid any attention until recent.
(how so? different values (sometimes many) return the same hash)
... I know it's just a game, and security isn't really that important... but still
I've been into muh-security for a few years now, and would at least like to see an implentation...
so yeah, hope to see minetest.sha256() soon :)
- rubenwardy
- Moderator
- Posts: 6978
- Joined: Tue Jun 12, 2012 18:11
- GitHub: rubenwardy
- IRC: rubenwardy
- In-game: rubenwardy
- Location: Bristol, United Kingdom
- Contact:
Re: [Solved] Simple non-cryptographic numerical hash in Lua
Security depends on the use case. Sha1 runs the risk of malicious collisions, but that may not matter if you're just generating colored names or inventory hashes
- Tcll
- Member
- Posts: 87
- Joined: Thu Jul 25, 2019 21:43
- GitHub: Tcll
- IRC: Tcll Tcll5850 DarkPikachu
- In-game: Tcll
- Location: The Gates of Darkness.
Re: [Solved] Simple non-cryptographic numerical hash in Lua
right, exactly :)
the problem is that's all we're limited to currently
murmur shouldn't be hard to wrap for something better, but sha256 needs to actually be written
but even still, hash functions are decently simple to wright :)
the problem is that's all we're limited to currently
murmur shouldn't be hard to wrap for something better, but sha256 needs to actually be written
but even still, hash functions are decently simple to wright :)
- sorcerykid
- Member
- Posts: 1847
- Joined: Fri Aug 26, 2016 15:36
- GitHub: sorcerykid
- In-game: Nemo
- Location: Illinois, USA
Re: [Solved] Simple non-cryptographic numerical hash in Lua
My Simple Cipher mod offers a variety of industry standard hashing functions:
https://github.com/sorcerykid/cipher
https://github.com/sorcerykid/cipher
- Tcll
- Member
- Posts: 87
- Joined: Thu Jul 25, 2019 21:43
- GitHub: Tcll
- IRC: Tcll Tcll5850 DarkPikachu
- In-game: Tcll
- Location: The Gates of Darkness.
Re: [Solved] Simple non-cryptographic numerical hash in Lua
nice job using the slow workaround of formatting numbers to bytes (strings) to get around the 32bit limitation of LUA 5.1 ;D
I'm trying to get the minetest devs to integrate a few of these into faster native C here :)
though on that note, why are we still using LUA 5.1 in 2021 when LUA 5.3 solves a ton of these issues??
minor rant:
EDIT: btw, I've found I actually don't need these hashes for my issues
LUA can hash anything in tables... which leads me to ask why we haven't deprecated SHA1??
I'm trying to get the minetest devs to integrate a few of these into faster native C here :)
though on that note, why are we still using LUA 5.1 in 2021 when LUA 5.3 solves a ton of these issues??
minor rant:
Spoiler
that'd be like me using Python 2.7 in 2021 when Python 3.4 is a much better choice
oh and uhh, btw, python has array.array(), which is muuuuuuuuuch faster for doing this same stuff ;)
though I can understand security reasons for not using python over LUA, even with restricted namespaces, despite having much more functionality and speed...
(I'm dealing with my own numerous issues here, though at least I hide your IP)
oh and uhh, btw, python has array.array(), which is muuuuuuuuuch faster for doing this same stuff ;)
though I can understand security reasons for not using python over LUA, even with restricted namespaces, despite having much more functionality and speed...
(I'm dealing with my own numerous issues here, though at least I hide your IP)
LUA can hash anything in tables... which leads me to ask why we haven't deprecated SHA1??
- sorcerykid
- Member
- Posts: 1847
- Joined: Fri Aug 26, 2016 15:36
- GitHub: sorcerykid
- In-game: Nemo
- Location: Illinois, USA
Re: [Solved] Simple non-cryptographic numerical hash in Lua
Evidently you are new to Lua, because 5.1 uses 64-bit doubles for all numbers.
Moreover, both the Adler32 algorithm and the Fletcher32 algorithm literally operate on character strings (i.e. bytes) when written in C. So I'm not "getting around" any limitation. You really need to do a bit more research before coming into these Minetest forums acting so arrogant and smug.
- Tcll
- Member
- Posts: 87
- Joined: Thu Jul 25, 2019 21:43
- GitHub: Tcll
- IRC: Tcll Tcll5850 DarkPikachu
- In-game: Tcll
- Location: The Gates of Darkness.
Re: [Solved] Simple non-cryptographic numerical hash in Lua
I wasn't trying to be arrogant and smug (a little self-righteous maybe, but nothing offensive), I'm sorry that comment sounded as such to you... :/
I'd gotten my info from these (primary) sources: (there were others I'd derrived info from as well)
https://stackoverflow.com/questions/310 ... t-integers
https://github.com/tst2005/lua-bit64/bl ... /bit64.lua
https://gist.github.com/tkaemming/25dfcd4998c470d630e1
as well as numerous individuals (which btw, someone mentioned LUA runs off 32bit floats, though I know for certain 64bit was introduced in 5.3)
yes I'm still a bit new to LUA
but I don't speak out unless I have at least some knowledge of what I'm talking about
again, I'm sorry if I sounded offensive, it wasn't my intent.
EDIT: btw, jsyk, if I had the option, I would've given your post an upvote before making that comment.
I'm glad to see more work being done to bring in more functionality, despite the shortcomings of LUA
I'd gotten my info from these (primary) sources: (there were others I'd derrived info from as well)
https://stackoverflow.com/questions/310 ... t-integers
https://github.com/tst2005/lua-bit64/bl ... /bit64.lua
https://gist.github.com/tkaemming/25dfcd4998c470d630e1
as well as numerous individuals (which btw, someone mentioned LUA runs off 32bit floats, though I know for certain 64bit was introduced in 5.3)
yes I'm still a bit new to LUA
but I don't speak out unless I have at least some knowledge of what I'm talking about
again, I'm sorry if I sounded offensive, it wasn't my intent.
EDIT: btw, jsyk, if I had the option, I would've given your post an upvote before making that comment.
I'm glad to see more work being done to bring in more functionality, despite the shortcomings of LUA
Who is online
Users browsing this forum: No registered users and 9 guests