Houdini Pattern Matching Mechanism/Library

[timmwagener]

Hey guys,

---

I'm sure that everyone who has ever attempted rendering in Hou has experienced the pleasures of using expression functions and pattern matching.
The workflow of formatting your target content (geometry, lights etc.) together with this syntax is truly powerfull.
 
For example:

---

I want to render a character and everything else to mask him. Group and expression operators make that easy and dynamic:
 
Render Objects: @character
Mask Objects: !@character
 
However at some point the group of @character has to be expanded and the ! negation-operator has to be resolved before the concrete pathes are used to generate the
information consumed by the renderer. I guess this is the job of the Pattern Matching and Expression Library I am referring to in the title.
 

---

Now on an abstract level, this is just a powerfull language for users to dynamically describe (sub-)sets of data which they wish to use for something.

---

 
Same principle - different data sources:

---

 
Now what if your Object Names in the Mantra node whose patterns and expressions will be evaluated would not refer to objects in networks but to entries in databases?
@group_name string expansion could result in all database entries whose "Group" field matches "group_name". (and of course !@group_name would result in the opposite...)
What if we are not talking Mantra ROPs anymore, but maybe a Pyside Standalone or Web Application that lets you enter these patterns and expressions to describe the datasets to pull from any database?

 

How to ?:

---

Does that sound realistic/reasonable/doable at all?
If so, do you know any libraries that define expression languages/operators and expose solvers for them? (Sympy!?)
Are there some details on how Houdini handles that?
How would you go about implementing a Mantra ROP like application that exposes string fields where people can use pattern matching and expressions to describe data that is then pulled from databases? Edited January 14, 2016 by timmwagener

[koen]

I am not sure about the cases you debribe here, but in the past I have used UT_String::matchPattern to do similar things. All this does is check if a string matches the pattern, so you'll need to build the list and check them.

 

"matchPattern(UT_WorkArgs &) assumes that the arguments contain the components of a pattern to be matched against. The method returns true if the pattern matches, false if it doesn't. This matching process handles ^ expansion properly (and efficiently). If the string doesn't match any components of the pattern, then the assumed value is returned."

 

From:

http://www.sidefx.com/docs/hdk15.0/class_u_t___string.html#a158f3b7b5fab18af745f218a8d2fdc68

 

Hope that helps,

 

Koen

[timmwagener]

Thanks for your answer! I feel like I a good example of what I want to achieve might be very helpfull.

 

Lets think of a web application

---

We move completely away from Houdini as a platform here. What we want to do is create a web application that lets people create and render genealogical trees of animals in their browser like so, so or so.

When they enter the adress of our website, they are presented with a form that has just one textfield called animals. Here they enter their patterns and groups in Houdini exression language to express for which set

of animals they want to draw the genealogical tree. The backend solver expands the groups, solves the patterns and operators and delivers the valid subset of animals to the gen. tree render engine. The

backend also has a big database where all animals of the world are in and they are also linked with groups like mammals, reptiles, pokemon etc.

 

Program flow

---

1. User enters an expression in the animal web form like: @mammals ^elephants firebellytoad , and hits submit in the form.

2. An url is submitted like https://genealogical-tree/get/animals=@mammals^elephants firebellytoad

3. Via REST API we call the function solve_aimals(expression=@mammals ^elephants firebellytoad) in the backend.

4. The solver a) expands the groups (@mammals) evaluates operators like ^ (except) 

5. After the matching animals are solved (In this case all mammals (horses, bears, whales...) except elephants and the firebellytoad, which belongs to amphibia) the results is returned

6. Call render_genealogical_tree(resulting_list_of_animals) and display the result in the browser.

 

Where do the animals come from

---

The possible animal names are stored in a big database, like possible geometry names in Hou are in networks.

Groups of animals are pointed to in database tables, as groups of geo objects are linked in the bundle editor.

 

How To?

---

Are there libraries that define pattern matching, a possible set of operators and according solvers (maybe even some grouping/expansion mechanisms) ?

I feel like supporting some sort of well established standard here would be much preferable over building something yourself with regex etc.

Does Hou use a third party library for this by chance or is it at least oriented on some well known libraries?

In general, how would you approach this undertaking?

 

...hope this makes my intentions a bit clearer!

Edited January 15, 2016 by timmwagener

[Stalkerx777]

Well, houdini pattern matching is almost the same as unix shell pattern matching, except that @ symbol. Most programming languages have support for it, and i'm pretty sure it's not very hard to extend this to your needs. But i don't understand the intention of making this type of interaction with user available in web browser. Wouldn't be nicer to have more pleasant UI for end user, say in JavaScript?

[timmwagener]

Well, obviously this is just the first best example I could think of However the basic idea behind this approach is for the UI to be data agnostic while providing maximum flexibility for the user.

As a programmer you would provide the data (databases) and the rules/language to interact with it (which I hope to find very well implemented/standardized in third party libraries). I expect users to come up with expressions

quickly that far exceed the possibilities of any UI. From my experience, once you used string based pattern matching, group expansion and expressions (like in the Mantra ROP) you never want to return to applications that force you to wrestle with the UI to format your content together (...think renderlayer/overrides in Maya as an example). Additionaly a simple string value is nice and easy to query/store/process.

 

Also think of cases where the dataset in question is subject to constant updates. An * operator would always take care of that, which would be pretty hard to achieve in UIs that change based on data.

 

Concerning libraries, Pythons fnmatch package might be worth a look....however it does not seem to suport nice operators like ^ (except).

Edited January 19, 2016 by timmwagener

[edward]

Just convert the input to a regex and then match against that.

[timmwagener]

Hey Guys,

---

work went along pretty well and I feel like I got the basics in place (expansion solver, pattern matching solver).

I'm currently in the process of writing tests to verify the solver and I use Houdini as the initial source of truth.

And therefore I have been stumbling upon something that seems kind of strange. The following expressions are

in the Mantra ROP Candidate Objects field:

 

A ) * ^@bdl_spheres sphere_a sphere_b <-- Renders everything in Houdini

B ) * ^@bdl_spheres sphere_* <-- Renders everything except sphere_a sphere_b in Houdini

 

In my solver B ) renders identical to A ) (everything) and therefore the test fails as I test against Houdini ground-truth.

---

• I attached my test scene if you want to give it a try.
• Please keep in mind these expressions are for programmatic unit tests and not actual expressions that would be used in production.
• @bdl_spheres contains sphere_a and sphere_b

---

Anyone any ideas? Is there something particular to know about operator precendence?

houdini_pattern_test_a_0001_tw.hipnc

Edited January 31, 2016 by timmwagener