[Question] meta:set_float behaviour/precision [Answered]

Post Reply
User avatar
sirrobzeroone
Member
Posts: 593
Joined: Mon Jul 16, 2018 07:56
GitHub: sirrobzeroone
Contact:

[Question] meta:set_float behaviour/precision [Answered]

by sirrobzeroone » Post

Hoping someone with more expertise can explain this one :)

Very simple I set a variable to 2.06
I then send this into a nodes meta as a float
Then retrieve the float later from meta

The float changes from 2.06 to 2.0599999427795

I think this relates to float precision and somewhere in the back and forth conversion it's getting slightly messed up?

I can workaround the above by I think storing as string and converting to number on return. But wanted to check and see if anyone else has noticed this behaviour? or if I'm using meta float incorrectly?

MT version 5.5 (yes havent jumped to 5.6 yet).

Thanks again for the help.
Last edited by sirrobzeroone on Sun Aug 21, 2022 11:24, edited 1 time in total.

User avatar
Wuzzy
Member
Posts: 4786
Joined: Mon Sep 24, 2012 15:01
GitHub: Wuzzy2
IRC: Wuzzy
In-game: Wuzzy
Contact:

Re: [Question] meta:set_float behaviour/precision

by Wuzzy » Post

You should probably first learn what floating-point numbers in computing are, actually. This is very important to understand to know what is going on (and why).

https://en.wikipedia.org/wiki/Floating-point_arithmetic

(just one of many resources on the Internet about floats)

If you need to store a value EXACTLY as it is, don't use floats.

User avatar
sirrobzeroone
Member
Posts: 593
Joined: Mon Jul 16, 2018 07:56
GitHub: sirrobzeroone
Contact:

Re: [Question] meta:set_float behaviour/precision [Answered]]

by sirrobzeroone » Post

Wuzzy wrote:
Sat Aug 13, 2022 00:55
You should probably first learn what floating-point numbers in computing are, actually. This is very important to understand to know what is going on (and why).

https://en.wikipedia.org/wiki/Floating-point_arithmetic

(just one of many resources on the Internet about floats)

If you need to store a value EXACTLY as it is, don't use floats.
Thanks Wuzzy,

I've been equating floating point to scientific notation mentally as per wiki...what I failed to allow for is that in scientific notation the old human brain auto rounds the same each time as we essentially just jump decimal points around, naturally computer/code dosen't work that way.

I'll use num >> string >> num conversion to keep the value fixed.

Post Reply

Who is online

Users browsing this forum: No registered users and 4 guests