Jump to content
antoniosouza3d

VEX Problem with precision comparing attributes

Recommended Posts

Hello, guys!

I keep having an issue when I try to compare attributes from two distinct points.

The VEX program tests if they are equal, like this:

     if(ptu == t_start)

And although I see in the geometry spreadsheet that they are, the VEX code say they aren't.

I thought It was a precision problem, so I rounded both values to a three float precision value

//round
t_start = rint(t_start * 1000)/1000;
t_end = rint(t_end *1000)/1000;
ptu = rint(ptu * 1000)/1000;

This was the solution for some points and for others, the problem persisted.
Some still have the same values in the spreadsheet and the VEX code interprets them as different values.

Edited by antoniosouza3d
  • Like 1

Share this post


Link to post
Share on other sites

in general you do not want to do an equal tests on floats (other than for 0 maybe)

I would suggest doing:

if(abs(ptu - t_start) < 0.0001) //add how many zeros after the decimal point for how much precision you need

alternatively, if you really require the numbers to be exactly the same, the best way is to use integer values only, note that the rint() function still outputs a float,
so you may need to also cast it like int(rint(floatValue));

Edited by acey195
  • Like 2

Share this post


Link to post
Share on other sites

Create an account or sign in to comment

You need to be a member in order to leave a comment

Create an account

Sign up for a new account in our community. It's easy!

Register a new account

Sign in

Already have an account? Sign in here.

Sign In Now

×