setPointGroup behavior

[sneaky]

Hey,

I have 2 scatter node with 30 points on each of them. I wired the two in a pointwrangle.
I used this code and I am wondering why the setpointgroup is running only once and not as the doc suggest : "This node runs the snippet on the detail or every point/primitive/vertex (depending on the Class parameter) in the input geometry."
This is probably something I missunderstood but I would be grateful if someone can explain me how it's really behave.

Here is my code :
 

@group_basePoints = 1;

vector npos = point(1, "P", @ptnum);
int newpt = addpoint(0, npos);

setpointgroup(0, "basePoints", @ptnum+@numpt, 1);

 

[Skybar]

It will run the code for every point, yes. Geometry will only be created after the code has run though, so @numpt won't update - it will only show the number of points in the input.

If you are trying to add the new points to the group, you can simply do:

setpointgroup(0, "basePoints", newpt, 1);

[sneaky]

Oh I see, thank you for the explanation. Two more questions in regard to what you have said.
Is it the same thing to use the new point number (the return value of the addpoint) as the @ptnum+@numpt ? Which in my opinion should be the same thing isn't it ?
How, if the geometry is created after the code has run through, it is possible that setpointgroup is working ?

[Skybar]

17 hours ago, sneaky said:

Is it the same thing to use the new point number (the return value of the addpoint) as the @ptnum+@numpt ? Which in my opinion should be the same thing isn't it ?

Well, no. The wrangle loops over all input points, and @ptnum will return the point number of the current point. So when it runs the code over the first point, @ptnum will return 0 - over the second point, 1 - over the third point, 2, etc. Up to 29 since we got 30 points that have point numbers 0-29. We established before that it will only loop over the input points, not new ones created in the wrangle. So if we use @ptnum when trying to set an attribute for a new point, it will return 0 because we are not currently looping over a point (essentially). So for all new points @ptnum will always return 0 and @numpt will return 30 (number of input points). We can see this because with your original code we actually get 31 points in the basePoints group - thats because @ptnum+@numpt is currently 0+30 - so point number 30 will be added which happens to be the first new point. But for all the other new points, it will still evaluate to 0+30 so none of those will be added. 

17 hours ago, sneaky said:

How, if the geometry is created after the code has run through, it is possible that setpointgroup is working ?

I'm not exactly sure of the specifics, but I'd imagine it first runs the code, then creates geometry, and then creates attributes. Or if it creates attributes on new geometry at the same time it is created. Something like that

[sneaky]

Thank you for taking the time to explain me in detail all of that. Sounds pretty clear now 

[djiki]

Modifying geometry inside wrangle/VOP node has nothing with internal node's local variables which refer input geometry . So, @numpt  is the number of points of connected geometry to first input and it is used to form an internally hidden a FOR loop with iterator @ptnum. So think of your code in point wrangle node as code inside FOR loop which is hidden. Adding or removing points will not impact values of @ptnum or @numpt inside that loop (whole your wrangle code). Function for adding point returns an index of new added point and that has nothing to do with @ptnum or @numpt values. However such index can be forwarded to almost all functions which require point number. Adding point occur immediately (not in visual manner but in internal data structure it exists as regular point) but removing point not. Function remove point only mark points for deletion which occur internally after whole loop is finished. That apply to all wrangle node types with one exception. DetailWrangle node doesnt form internal loop of any kind so iterator value like @ptnum or @prnum doesn't exist. Values of @numpt or @numpr are still valid because they are referencing input geometry.

Those hidden FOR loops can be easy "visible" by comparing a code inside PointWrangle node:

 setpointattrib(0,"myattr",@ptnum,1,"set");

with a code which has identical behavior but written in DetailWrangle node:

for(i@ptnum=0; @ptnum<@numpt; @ptnum++){

 setpointattrib(0,"myattr",@ptnum,1,"set");

}

This is valid code. @ptnum doesn't exist in DetailWrangle as reserved variable so you can use it like you would use any other variable. I chose it just for illustrative purpose because those two lines for comparison are identical that way. Any context insensitive code written in PointWrangle node can be copy/pasted in such loop in DetailWrangle node and it should work fine. For example, context sensitive code is: 

i@myattrib=5;

and that will not work because that line of code if executed in point context will create/assign point attribute "myattrib". If it is executed in primitive context it will create/assign primitive attribute "myattrib" etc.

But if that is written as context insensitive function call 

setpointattrib(0,"myattrib",ptnumber, 5,"set");

that will work doesn't matter in which context is executed as long as you can provide handle (the very first argument in most context insensitive functions) and index (not required in all functions).

Now, consider your original code inside PointWrangle 

@group_basePoints = 1;

vector npos = point(1, "P", @ptnum);
int newpt = addpoint(0, npos);

setpointgroup(0, "basePoints", @ptnum+@numpt, 1);

if you change it to be context insensitive it would looks like this:

setpointgroup(0, "basePoints", @ptnum, 1);

vector npos = point(1, "P", @ptnum);
int newpt = addpoint(0, npos);

setpointgroup(0, "basePoints", @ptnum+@numpt, 1);

Copying such code in DetailWrangle loop from previous example and voilaaa .... it works. 

Now you are probably more confused  Same code works one way and not the other way. To keep track what actually happened modify code like this

setpointgroup(0, "basePoints", @ptnum, 1);

vector npos = point(1, "P", @ptnum);
int newpt = addpoint(0, npos);
setpointattribute(0,"np", @ptnum, newpt, "set"); // this is new line
setpointgroup(0, "basePoints", @ptnum+@numpt, 1);

Now, watch spreadsheets on both DetailWrangle and Pointwrangle nodes with this same code. You can see results of addpoint() function in that np attribute on each point.

In the case of DetailWrangle, all execution is done on single thread and result of addpoint() is incremented each time, as expected, and that's why that last line of code works.

In the case of PointWrangle node, different values are consequence of multi threaded execution. You should always have on mind, that execution of PointWrangle will start on multi threads in parallel. So from the aspect of single point on some execution thread, result of single calling addpoint() will always return @numpt. If you modify your code in such way that, single point calls more than one addpoint(), their results will be @numpt+1, @numpt+2 and so on. In the moment of execution, result of addpoint() function is not synchronized among other threads, thats why each execution thread get @numpt value as the result of first such call, @numpt+1 for second call etc . First synchronizing barrier for them is at the end of thread execution and that is end of your code. That part is hidden from user. So, as long as you use returned value in the rest of code, you are sure you are addressing just added point(s), doesn't matter if those values are same among different threads. So line:

setpointgroup(0, "basePoints", @ptnum+@numpt, 1); 

will fail in each particular thread because, each thread added only one point and each thread gets @numpt as the result of addpoint() and your code is trying to address point with larger index. You can change that line only in PointWrangle node just to see result:

setpointgroup(0, "basePoints", @numpt, 1);

and it will work. BUT,

whole this is written only for behavior testing purpose so you can see differences in execution logic. In your real code you would NEVER use such "tricks". Especially because such things depend on internal design which could vary with every new version of Houdini.  So as long as you stick to rule to use returned point index of addpoint() function, you are sure it will work regardless of "what is behind". So proper line would be:

setpointgroup(0, "basePoints", newpt, 1);

as Skybar already pointed.

cheers 

 

[sneaky]

@djiki First of all, thank you for ur very detailled answer that help me better understand those nodes. I am very curious to know where u found all that piece of information which is really usefull and I am sure others that will read you will also wonder.

Your answer was very very clear and I will probably return to it if I have a pb with this node. 

     If I did understand you words correctly, you also implied that as pointWrangle is multithreaded the index of the newPoint will always be in this iteration @numpt. If we create a second point in this same iteration it will be @numpt + 1.
     Detail, as I understand (tell me if I am correct) is not multithreaded, so it will run the iterations one after another. As a consequence to this the index of the new point will be eachtime incremented.

     So when we do this hacking thing that you were proposing for demonstration purpuses :

     setpointgroup(0, "basePoints", @numpt, 1);

     It is actually working because, as we are creating only one point it is equal to the indew stored in newpt. If I am creating a second point I should write (always as demonstration purpuses):

     setpointgroup(0, "basePoints", @numpt + 1, 1);

     To put my second point in that group.

     That is why everybody is using the newpt, is clearer and more convenient as it is I guess in the back secret garden taking care of this math for us.

I am now wondering why are we using (appart for sensitive contexte which are certainly handy) the pointwrangle, is the multithreading thing making it faster ?

Thanks again

[djiki]

Most information are not Houdini specific but simply concept of programming for multi threaded execution. Primer is just an example of how following human logic can lead to wrong results. 

Quote

 Detail, as I understand (tell me if I am correct) is not multithreaded

As it runs only one pass of your code and there is no mechanism to start multi threads from code, Detail is single threaded. But, in new version of Houdini you will see one more "Run over: NUMBERS" method which would allow multi-threaded execution.

On 3/27/2020 at 10:13 AM, sneaky said:

So when we do this hacking thing that you were proposing for demonstration purpuses :

     setpointgroup(0, "basePoints", @numpt, 1);

     It is actually working because, as we are creating only one point it is equal to the indew stored in newpt. If I am creating a second point I should write (always as demonstration purpuses):

     setpointgroup(0, "basePoints", @numpt + 1, 1);

     To put my second point in that group.

     That is why everybody is using the newpt, is clearer and more convenient as it is I guess in the back secret garden taking care of this math for us.

Well, that's depend on how coders of Houdini implemented synchronization of threads in that hidden part of code . At least, if your overall number of points is less than thread pool block size, that should work. First time sync among finished threads is done, rest (still unfinished) threads could have different result. But as I said, it depends on code implementation which is hidden from user. So things like: decision about of thread pool block size which will run in parallel, and possible synchronization barriers and possible different algorithms of handling divergent execution paths etc are not known, so you can not guarantee that such code will work in all cases. That is the reason why you should use number returned from that addPoint() function. Even if you know all of those things behind, that doesn't mean they can not change with every new version of Houdini so such "dirty" code wouldn't run properly in newer versions.

On 3/27/2020 at 10:13 AM, sneaky said:

I am now wondering why are we using (appart for sensitive contexte which are certainly handy) the pointwrangle, is the multithreading thing making it faster ?

These days almost every processor has several cores. Also modern GPU has thousands of cores. Separating an execution on many of them and execute in parallel usually means faster execution. How much faster, depends on nature of problem and code optimization for hardware specific advantages/limitations etc. If you want to learn more on that topic, google for CUDA, OpenCL, PTX, Parallel programming algorithms etc.     

Edited March 29, 2020 by djiki
Text missing