PDG - Remove work item

[Alain2131]

Hello !

I'm trying to learn a bit of PDG since it might be useful for some of my projects.
But I'm having a few issues
For one, I wasn't able to find how to remove a work item.

What I'm trying to do is search for a string in a bunch of files, and based on that I'll do some other things. Right now I'm simply trying to get to find the string in the files, but first I need the relevant files.
So I use the filepattern node for that, but there is a few files for which I don't want.
I use a pattern for the extension (*.txt), but there is some files with a specific string (in this case "proxy") in the filename that I would like to ignore.
I tried something like "thePath\*.txt ^*proxy*" to get all the txt files without the ones with "proxy" in the name. Didn't work, maybe the syntax is wrong, but moving on.

So I have a bunch of files (aka work item) that I want to get rid of. I tried using the split node, but I don't know how to work with PDG attributes. It always either error out, or gives me this

Quote

Warning on child node /obj/topnet1/split1/splitattrib:
Missing or empty attribute '@filename' when evaluating '/obj/topnet1/split1/splitexpression' with work item 'split1_splitattrib3'

I tried "@filename="file.txt" | @pdg_filename="file.txt" | I tried without the quotes, I tried with backticks around the filename part, without any success.
What's the correct syntax for this ? I tried reading this, but without much results. I would need an example to understand it.

So using expressions didn't work. I then moved on trying it out with Python. There is the "work_item.dirty(True, True)" command, which works as I want, I see the work item being removed in the node UI, but then Houdini freezes, then crashes.

So I'm seemingly out of options.
There is multiple questions, which are as follows :

1 - Is there a syntax for the file pattern that allows to get all .txt files, without the ones with a specific pattern in the name ? (mockup - "*.txt ^*proxy*")

2 - How to use PDG attributes in a node, as part of an expression ? Kinda like something simple like "@Frame=2", but in this fashion '@filename="file.txt" '

3 - Why does the work_item.dirty() python command crashes Houdini ? Am I doing something wrong ?

 

Test file attached. The zip contains some random .txt files with names to test out what I want, along with the hip file.

pdg_test.zip

[Alain2131]

I wasn't able to answer any of my questions yet, but I found a workaround. This doesn't mean that I don't need an answer to the previous questions, so feel free to reply 

My workaround for getting rid of those pesky work_item that contains the dreaded "proxy" keyword is using the Python Processor node.
I noticed that placing this bad boi in a chain blocks all traffic from coming through, and that I would need to police each of the work_items to decide which one I let through. And this is exactly what I want to do !
So the concept is really simple, for each of the upstream work item, check if "proxy" is NOT in the filename. In that case, yeah ! Let 'im through !

for item in upstream_items:
    if "proxy" not in item.attribValue("filename"):
        newItem = item_holder.addWorkItem(parent=item) # parent copies the original work_item's attributes

So instead of deleting the ones I want to remove, I'm actually regenerating the whole graph, copying only the ones I want.
I feel like this is really not optimal, so using an actual node made for that purpose (like split) would surely be a lot better.

Attached is the updated hip file, along with a quick mockup of that "find string in files" idea I mentioned.

pdg_test.zip

[jamesr]

I've been playing with PDG for a little bit, and am by no means a wizard, but if you want to remove work items, you can try using the "Filter By Expression" TOP. Set the operation to "Remove Work Item".

7 hours ago, Alain2131 said:

So instead of deleting the ones I want to remove, I'm actually regenerating the whole graph, copying only the ones I want.

I feel like this is really not optimal, so using an actual node made for that purpose (like split) would surely be a lot better.

Not sure what exactly the filter node does behind the scenes, but it might actually be similar.

 

Quote

2 - How to use PDG attributes in a node, as part of an expression ? Kinda like something simple like "@Frame=2", but in this fashion '@filename="file.txt" '

In a Python Script TOP, you can use the handy shorthand syntax

`@myattribute`

or for strings

"`@mystringattribute`"

There is an equivalent in python expressions, but it's not as elegant (I could be missing something similar, but this has been working for me). It's basically just the long way you'd typically use in a Python Script TOP:

pdg.workItem().stringAttribValue("myattribute")

In the netbox I'm just faking the result of a file pattern top. I have 2 generic generators making work items that get assigned a file name. The left branch makes "file#.txt" and the right "file#_proxy.txt". They get merged together to emulate something like the result of a file pattern. The Filter by Expression top has the following for its Filter Expression:

"proxy" in pdg.workItem().stringAttribValue("myfile")

filter_by_expression.hip

Edited August 14, 2020 by jamesr

[Alain2131]

Hi James !
So you're basically using Python in the expression box. I totally didn't think about that, and yeah, that works !
Thank you for that insight, much appreciated

Oh and I figured out the second question !
Turns out I was trying to match strings, which has to be through the "strmatch" function. I was doing an "==" comparison, which doesn't work for strings.
So strmatch("*proxy*", @filename) for string comparison and @intAttr=4 for int/float attributes works fine.
There are some weirdness from attributes created within a Python Script node (has to set the subsequent nodes trying to access the attribute to Dynamic, else it's as if the attribute doesn't exist, and other disparities), so that's something to keep in mind (see attached hip file).

pdg_test.zip

Edited August 14, 2020 by Alain2131
Added attachment

[jamesr]

Not sure what you mean by attribute weirdness, but if you're finding that sometimes the python nodes aren't cooking how you want, try setting their evaluation to "generate" and see if it works better. This works fine if your graph doesn't need to be dynamic. Also, double check your attribute names. You're calling it "strAttrib" in the python script top, and "strAttr" in the attribute create. (Not sure if this is on purpose)

[Alain2131]

Welp, that sure wasn't helping =P

My bad - but looks like they fixed the weirdness in .532, vs .348
Lemme explain - I'm testing this out on my work PC (that has 18.0.348 installed), but can't upload a  file from it, so I recreated the same setup on my personal computer (with 18.0.532). I messed up writing that part, but here's the behavior attached I was having on my work PC

And when redoing the setup on my personal one, I mis-typed this part, and assumed that the problem just got worse in .532, but no ! Without the typo, that weird behavior seen in the attached video is no more !

.. Or is it ? There is some more weirdness coming from the other branch. The discard_files_with_no_match1 shows the same behavior when set to Automatic. Set it to Dynamic, and now it works.
Sorry for the multiple edits. Here's the video of it happening in the .532 version.

Edited August 14, 2020 by Alain2131

[Drunkenpanda]

"something" in pdg.workitem["files"][0]

Hello, I'm trying to filter on the work item name, the code above hower did not work.
Anyone know of a way ?

Edited September 16, 2023 by Drunkenpanda