VEX Problem with precision comparing attributes

[antoniosouza3d]

Hello, guys!

I keep having an issue when I try to compare attributes from two distinct points.

The VEX program tests if they are equal, like this:

     if(ptu == t_start)

And although I see in the geometry spreadsheet that they are, the VEX code say they aren't.

I thought It was a precision problem, so I rounded both values to a three float precision value

//round
t_start = rint(t_start * 1000)/1000;
t_end = rint(t_end *1000)/1000;
ptu = rint(ptu * 1000)/1000;

This was the solution for some points and for others, the problem persisted.
Some still have the same values in the spreadsheet and the VEX code interprets them as different values.

Edited October 16, 2017 by antoniosouza3d

[acey195]

in general you do not want to do an equal tests on floats (other than for 0 maybe)

I would suggest doing:

if(abs(ptu - t_start) < 0.0001) //add how many zeros after the decimal point for how much precision you need

alternatively, if you really require the numbers to be exactly the same, the best way is to use integer values only, note that the rint() function still outputs a float,
so you may need to also cast it like int(rint(floatValue));

Edited October 16, 2017 by acey195

[antoniosouza3d]

Man, what a great idea!

Now everythin's gonna be easier! rs

Thank you very much!