Search the Community
Showing results for tags 'pipeline'.
-
Currently facing issues with file attachement //////////////////////////// OpenUSD Guide for (Student) Games //////////////////////////// Hello everyone! For the several weeks I have been looking into implementing OpenUSD into the pipeline of student games. The information that is currently available is more technical than practical or software-specific, instead of guiding you how to set up a pipeline. To fix this problem, I decided to write my own high-level guide for those who will follow in my footsteps, whether it be student like myself or individuals who have taken an interest in OpenUSD and its capabilities! I will leave out the in-depth research, as most of the information can be found on OpenUSD.org. Disclaimer: I will restrict this guide to Maya and Unreal. Houdini and Blender's approaches to the pipeline will be shortly discussed. This guide is also currently aimed at environment art, due to a lack of availability of other disciplines at the time of writing. //////////////////////////// Before we start, here is a quick introduction on the author of this post. My name is Sikandar Maksoedan, 20, I'm a 3rd year student at Breda University of Applied Sciences specializing in Tech art. I am currently looking for an internship starting September 2025. I am currently working on broadening my skillset in terms of Shaders, VFX and Pipeline optimization. I got interested in USD after my teacher Zoran Arizanovic mentioned the possibilities of the file format during a lecture some time ago, but only really picked it up after looking for a way to transfer a unreal scene to a different DCC while retaining all the data. //////////////////////////// Contents 1. What is OpenUSD (& Why should you care)? 2. How does USD function for Artists? 3. Introduction to Pipeline 4. Practical showcase of Pipeline 5. Closing statements //////////////////////////// 1. What is OpenUSD (& Why should you care)? USD, Universal Scene Description, is a file format that allows for the transfer of entire 3D scenes. This includes data from Geometry to Materials, from Lighting to Cameras, and more within this single file format. It shouldn't be grouped together with OBJ and FBX, it is closer to .MA and .Blend files. USD as a single file can be read in almost any major DCC with most of its features intact. USD allows the referencing of different USD files, allowing multiple people to collaborate on a single project at once. By layering USD files in sublayers, you are able to have your team work in tandem. Your level designer could be moving around props using a proxy mesh while your Prop artist is still modelling. The difference between using USD and a normal approach in team settings would be the latency. By updating a single USD file, all layers referencing this file will adjust their proxy on a quick refresh. Then, why should you as a (Tech) Artist care about USD? The advantages for both teamwork as personal work that USD provides are too big to be ignored. USD is like the PDF of games, a universal format that we won't be able to live without in due time. The scale of these advantages is big enough to have game studios such as Remedy Entertainment switching over already. Within the next few years, USD will be slowly adopted within the games industry. This means being able to work with it will be a skill required, especially for students. For all the advantages of USD, I will redirect you to the OpenUSD website: https://openusd.org/release/intro.html //////////////////////////// 2. How does USD function for Artists? USD for Artist should be kept as simple as possible. As stated before, this article isn't aimed at explaining all the technical details. USD can be used in 3 major ways for Artists 1. As a Container for single assets 2. As a Sublayer for a collection of Containers 3. As a Shot containing a collection of Sublayers Not all 3 have to be used to succesfully implement USD into your pipeline, but having all 3 gives you the most advantages. Lets go over usage 1, as a Container for assets. USD stores data in containers called 'Prims'. These Prims can have Parent-Child relationships with other Prims. Prims can contain Property, which can be divided into Attributes and Relationships Attributes consist of data such as the Vertex coordinates to form a mesh Relationships point to other data, such as which Material needs to be used In the proposed pipeline, you would create your Assets and store them in individual USD files. These USD files would be referenced in this Prim. Now onto usage 2, Sublayers Prims can be stored in Layers. Layers are akin to the scene hierarchy you would have in most DCC. Layers is where the Scene building happens. You can instance a Prim. This instancing references the USD file using it's local path, allowing us to change the Prim itself while keeping the reference. I have chosen to name the layers 'Sublayer', as you can have layers consisting of layers. This would allow you to have seperate layers for seperate parts of a level, each part consisting of more layers. That allows you to disable and enable parts of a level with ease, while also allowing you to for example work on the lighting or set-dressing of the entire level at once. You could also have Sublayers containing the Blockout of a level and work on a reference of that in your own DCC, lowering the latency between departments. Each Layer will be stored as a individual USD file containing reference paths to the lower levels. Usage 3 is last, The Final layer containing information of all the Sublayers is called a shot. Shots are the file you can use to preview all Sublayers and Prims at once and would contain the entire level. With this covered, we can go over to the actual pipeline setup that I propose //////////////////////////// 3. Introduction to Pipeline The USD pipeline that I have tested out for Games is fairly simple. But first, we need to discuss the role that USD plays in the pipeline. USD is practically an asset library in the way we will use it. We are creating assets that will be referenced by other people in their scene. The difference here is that we have automatic references to these assets, and once they are updated all scenes containing the file will also automatically update. Another advantage of the USD pipeline that we can open the files in any DCC to adjust it, whether it be a full scene or a single assets We have 3 hierarchies of files, the Shot, Sublayer and Container. Container will almost always be an Asset, so I will use these terms interchangeably. See Image 1 Containers will be the hierarchy where you will save your work to most of the time. These files are what in a normal pipeline your FBX files would be, as these files contain singular assets. Every asset gets an individual USD file so beware of naming schemes and folder hierarchies! If you want to work on a single USD file, you can. For texturing this can be nice. Most of the time however, you would want to work in a Sublayer. Sublayers will be the hierarchy which you will open most of the time. The layers are what in a normal pipeline a folder would be, whether in Maya or Unreal. Sublayers have the widest usecases. They can be used to store your own work like a .MA file, or as a part of a scene. Ideally you would have a Sublayer replace your .MA file, and use a separate sublayer to build part of your scene. Having multiple sublayers allows you to work on the same scene with other artists, since they are all individual parts of the same Shot Multiple Sublayers can reference the same Containers, and multiple Shots can reference the same Sublayers. Shots will be total sum of Sublayers and Containers. This is the file that will be loaded into your Game Engine. This file only needs to be accessed when adding or removing Sublayers. Proper naming schemes is key when using USD files, as there is little distinction to be made between the 3 types when they are on your disk. naming them with a prefix such as Co_, La_, Sh_ will already bring you a long way. I personally order the files like the following example: ------------------------------- -Level_01 --Shot_Level_01.USD ---Sublayer_Setdressing_01.USD ---Sublayer_Lighting_01.USD ---Sublayer_Buildings_01.USD ---Sublayer_BuildingBlockout_01.USD ---Sublayer_BuildingBlockout_02.USD -Assets --UserScenes ---Sublayer_UserScene_02.USD ---Sublayer_UserScene_02.USD --BuildingKit ---Asset_SM_ModularWall_01.USD ---Asset_SM_ModularWall_02.USD --Foliage ---Asset_SM_Tree_01.USD ---Asset_SM_Bush_01.USD ------------------------------- As you can see, We have seperate folders for our Assets and for the levels For a better overview different assets have been seperated based on types Users have their own scenes that reference specific assets So to quickly sum up what we have gone over up until now: You model assets and save them in Container-level USD files You assemble parts of the scene and save them in Sublayer-level USD files You combine all parts of the scene and save them in the Shot-level USD files Below you can see a more expansive overview of my USD pipeline. See Image 2 As you can see, we divide it up in Shots, Layers and Assets as mentioned above. We model and texture in the Asset stage, we can texture and assemble our scene in the Layers stage, and we can combine our Layers in the Shot stage. We Import the Shot stage into Unreal, but Unreal can both read and write to the USD files to edit the data. I have added OpenUSD viewer to the pipeline as I often use it to verify that my USD changes have actually saved. It also makes it easier to preview individual files and check if they are correct. With this info, we can go on to the practical pipeline example //////////////////////////// 4. Practical showcase of Pipeline The first step in the practical pipeline would be the setup of your Main USD file, the Shot, and its first few Sublayers. In Maya, enable the USD plugin to get started. By using the USD layer editor, we can create several layers and save these. They will each save as a separate USD file. See Step 1 You can easily add Geometry to a layer, but this causes it become a single mesh. See Step 2 A better approach that works for larger setups is using references to other USD files that contain the desired assets, whether it be boxes or horses. See Step 3 We are now saving the horses to a separate layer, allowing us to completely remove the boxes from our sight just like you would in the outliner. As you can see, the material setup and positions get correctly imported into Ue5 when using the Ue5 USD plugin. See Step 4 Changing the position of meshes in Unreal and saving will update the USD file, and refreshing in Maya showcases these changes. See Step 5 Changing your material setup in Maya changes the MaterialX parameters, thus changing it in Ue5 by extension. See Step 6 And that concludes the short practical demonstration. Once I get my hands on some more time and more assets to create a proper demonstration, I will update this guide. This concludes the basic pipeline, I will list some extra resources at the end of this topic. This is just a basic demonstration of the basic concepts, on how we can use different USD files as an assetkit to build larger scenes. Now some highlights as to why we are not using Blender. Blender lacks the simple UI that Maya has where you can easily create USD layers, Blender can still be used in the USD pipeline for individual asset creation, while the assembling currently has to be done in Maya, Houdini or Unreal I have left Houdini out of this pipeline on purpose as Maya is the most user-friendly, and thus easier to use in a student-game pipeline. Houdini also has plenty of documentation online on how their USD workflow and has plenty of support for the features in Solaris. You can check out Project Greylight for a practical example, which you can find on the SideFX website that will be listed below. I personally use Houdini as my main DCC, so I will update this guide later with a proper Houdini pipeline setup. //////////////////////////// 5. Closing statements Well, that marks the end of my short introduction to USD for student games. I have purposefulyl left out Variants and Classes, just to make the key pipeline of USD easier to understand. USD is slowly creeping into the head of students at my school, but sadly none seem too eager to try it out in actual production. With USD finally making it's way into the games industry, we students should be the first to jump on the OpenUSD train and learn how it works before entering the Games industry. I doubt this guide will get much attention on the forum itself, but for future students it would be nice to have some form of starting point. Currently I'm looking for industry people that have used USD daily and can help me improve the pipeline a bit more, and for Internships where I could possibly improve my understanding of Pipeline integrations. I am also looking into replacing GLTF in a custom engine pipeline with USD, giving not so tech savvy artists a chance to help out in these projects by using their preferred DCC. Thank you to anyone who read this through to the end, If you have any feedback to improve my pipeline or see any mistakes then please reach out!
-
Hey there, We're developing our own pipeline that will be set up as a saas product and are looking for another experienced developer to join our team of talents. Find out more info here: https://arxanima.com/career/pipeline-developer2 All the best, David
-
HI ! We have just started small new studio and we are at the point of choosing proper pipeline for us. we are free from any legacy and are able to build our pipeline from scratch we plan to switch to usd, but not sure what to use - Karma or Arnold. we have experience using both arnold and redshift in the past ....but think about using karma as the best integrated rendering solution (seems like) sidefx says that it is production ready. but does it make sense ? wouldn't we regret ? so maybe someone has already walked this way and could share some opinions ? what is better option to choose ? Thanx in advance fir any info regarding this matter
-
Hi All, I've been working on the .vox exporter for a while. After some improvements I decided to publish it on github. The HDA generates .VOX Magicavoxel file from given inputs. If interested you can get it from here; https://github.com/caglarcomez/Houdini_to_Magicavoxel
-
- magicavoxel
- pipeline
-
(and 10 more)
Tagged with:
-
When trying to close Houdini, a dialogue window pops up asking the user if they want to save the file or close Houdini right away. It seems like the saving (using this dialogue, not saving in general which is working fine) has lead to several corrupted files in our internal network, therefore I was wondering if there was an option to de-activate this? When closing Houdini, the scene should just close and neither try and save anything nor should the user have the option of doing so as this might result in corrupted files. Is that possible somehow?
-
Hey, This is a very broad question but being somewhat new to Houdini (not vfx in general) I thought it would be worth asking people with more Houdini knowledge than myself. We are a small VFX studio and we have recently switched to being fully Houdini, as in, everything but some modelling, rigging and animation will be done in Houdini. We are also using Arnold for rendering. I've been looking at ways we can handle things like asset management through houdini, for instance creating a chair asset from modelling to lookdev and ending up with a final asset to then be referenced in shots for rendering. Obviously there are many ways of doing this, I've seen houdini HDAs which seem like they could be the way to go but I'm wondering how other studios are working with Houdini, not just single artists. Any links or information to research would be much appreciated.
-
DupeVFX are looking for a Mid-Senior Houdini artists to join our small growing 3d team in East London. We are currently at the stage of switching to an almost full Houdini pipeline, wanting to take advantage of using Solaris and USD to drive the pipeline backbone and PDG for automation. The two roles we are currently interested in are: - Lighting artist with production experience using Arnold for Houdini. Solaris experience a bonus. - Pipeline oriented Houdini artist to help develop our Houdini/USD/PDG pipeline and integrate with our internal pipeline tools. We would also welcome very experienced supervisors that have run multiple shows using Houdini as the core DCC. To apply send an email to joinus@dupevfx.com
-
I'm trying to figure out if a TOP's/PDG network can be made to keep alive while waiting for data from a database. Basically I want something like a while loop in python that keeps the TOP network waiting for new items to appear in the data base and then start processing once a new work_item is available. Is TOPs/PDG even meant for such a thing? Thanks! -Johan
-
I made this HDA to streamline the process of versioning caches. It will automatically produce a file path and file name for your cache, and load it back in once it is exported. You can flip through different versions easily by using the version slider, or using the 'Create File Node for This Version' button and wiring the file nodes up to a switch node. You can write detail attribute strings to store notes about the cache such as simulation parameters - very useful when referring back to old sim caches. At the moment this is a non-commercial HDA. Download link down this page beneath the video. Get at me with thoughts, comments, questions etc. SOP_MI_version_filecache.hda
- 5 replies
-
- 3
-
-
- version
- versioning
- (and 9 more)
-
Hey party people, Has anyone developed a solid worklflow for getting .sbsar files to work with redshift inside houdini? Did some research on it, but it seems like I am not the only one struggling with it! Excited for your opinions Cheers, Felix
-
Hi guys, I'm currently experimenting with packages for setting up Houdini's environment variables. Is there a way to update the HOUDINI_PACKAGE_DIR while Houdini is already open or do the packages only get scanned during startup? It looks like the packages don't get processed if I just change the HOUDINI_PACKAGE_DIR from inside Houdini via hou.putenv() or hou.hscript('setenv …'). What I'm practically trying to do: I'd like to switch project specific packages with the corresponding env variables being updated when the user opens a file that's located in another project. I also actually don't want to relaunch Houdini. I guess another more basic way would be to just set all the variables being inside the package manually in the 456.py?! Thanks for your help! Michael
-
Some strings in my CSV have white space (spacebar) and some have unusual characters that Windows and Houdini don't like, such as Å. In a TOPs network, how can I create an attribute `@cleanName` without these characters? ie: MY HÅT becomes MY_HAT Something like this, but dynamically. mystring.replace(" ", "_") If someone could help with the whitespace I can work out the rest. Thanks!
-
Hey all, I'm trying to set the environment variable HOUDINI_USER_PREF_DIR - to a specific folder. I have other environment variables working perfectly, but I cannot get this one to work regardless of whether I use a windows environment variable or place this inside the Houdini.env, Everytime I open Houdini and check the shell using hconfig, it says my temp directory is set to the default directory, does anyone have any ideas? My only guess is this is being set elsewhere and their fore overriding my manually created env varaibles. (Ive checked spelling etc. and this is what my windows env variable looks like; name = ‘HOUDINI_USER_PREF_DIR’, value = ‘D:\Houdiniprefs’ ) Thanks Chris.
- 3 replies
-
- environment
- variables
-
(and 2 more)
Tagged with:
-
Hi, Is there any way to export from Houdini procedurally and working in Maya with that model, I heard that procedurally doing this way doesn't give any issues and works fast in Maya while working with large super sets or even smaller one... I wanted to model a super set in Houdini and import in Maya and shade, light it... Please text If you do any exports from Houdini to Maya...
-
Hello folks, so, i have a bunch of houdini file with a top node at OBJ level, all file have the same structure and are in the same folder I would like to cook the TOP node from external, Python or another TOP graph. I tried in a background script to access with hou.node('/obj/topnet').parm('cookbutton').pressButton(), but it seems that doesn't cook anything Any idea ? Thanks in advance
-
Hello folks, i'm reading a .json in TOPs, creating string attributes, and collecting. Now, i would like to read those attributes that i created in TOPs from SOP, i'm tryng with @myattr, and pdginput, but no luck. Basically it's the same situation described here: https://www.sidefx.com/forum/topic/66408/ Any idea ? Thanks in advance
-
Hello everyone, My name is André and I’m researching how to design a 3D (environment) Production Pipeline for Virtual Reality training. The goal is to have a pipeline that would allow us to build training environments as efficiently as possible. Due to the nature of the training, our environments are relatively small max 200m x 200m. Our team is small 3-6 people but can change depending on the project. I'm especially interested in Houdini3D integration in the Pipeline for generating procedural assets. Online I've done quite some research but don't find much information about this topic in detail. Lots of examples of output but not the actual production process. I need some help with designing our pipeline. Love the GDC talk by Insomniac about procedural generation in Marvel's Spider-Man (see video below). Gives some great insights but would love to learn how to do this on a small (indie) scale. Could you share your Production Pipeline in a diagram? What do you produce with Houdini and what do you produce with other tools? Would love to learn about how you developed your pipeline and why you made certain choices.
-
Hi everyone, I've been tasked to lay down a decent workflow pipeline plan for my team revolving around creating motion graphics or key visuals for advertising in houdini and I was hoping to find some help/suggestions for it. I chose houdini as the core element just to avoid extra plug-ins that from my experience were creating a wast amount of issues in the long run, but I feel I won't be able to do everything I need quick enough using it alone. My first idea was to get license for zbrush, substance and redshift to complement houdini, so basically it would be like this: start by modeling non procedural things in zbrush, texturing them in substance, animating everything in houdini and finally render in redshift. When I started testing the workflow, I immediately noticed that substance designer plugin is not compatible with latest versions of houdini and my biggest fear came back, since I was hopping to achieve something at least slightly more seamless. I am now considering making all maps and textures through photoshop and zbrush only, skipping substance altogether and sticking as much as possible to procedural shaders in houdini if I can produce them quick enough. Anyway, what kind of workflows do you guys have or recommend? Could you give me any tips on how I could make things go smoother and avoid unexpected compatibility issues? How was your experience using Substance painter or designer within the workflow so far? Cheers!
-
My Effects Pipeline Destruction Constraint toolset Tree Rig Lib and much much more Viemo Playlist : https://vimeo.com/album/4484231 password : 6h45 please if you have any comments tell me !!
-
Hey All! The first part of this tutorial has been available for almost a year now, but because of the sad news that hit CMI, I was unable to upload the 2nd "half" there. So instead I just made it available on Youtube for everyone, to make up for that I suppose. I'm considering putting the first part on there as well, if enough people want that. In this tutorial covers the following, using Houdini * Generating water meshes * Updating the terrain based on the water * Generate walking paths on the terrain * Create some basic instances * Build a flexible system using external files to place these instances. Built on Houdini 16.5, but should work on 16 too, or all the way back to H14 if you skip the heightfield part. Recommended specs: at least 16GB of RAM, reduce the terrain size if you have less. Disclaimer: Work files are as is, and do not contain the cached geometry to save on space, this may explain node errors on the various "File Cache" nodes. It does however also contain the work done in the first half of the tutorial, albeit mostly undocumented: https://tinyurl.com/y89egjvq Hopefully its of some use! Twan
-
Hey there. I am trying to read a heightmap, do some changes to a specific area and then write it back out again. I am making it a specific area because an 8x8k heightfield seems a bit too heavy. Unfortunately, when trying to use a Composite COP Node to compose the changed area back on the original image, it leaves big black bars, freezes Houdini, or just instantly kills Houdini. On trying to export it all with the ROP File Output node, it just g ives a bake error. Could anyone have a quick peek and see if there is something obviously wrong with my setup? I imagine it is because I am new to the program and am doing something fundamentally wrong with the way I build graphs. The attached HDA needs a heightmap and that's about it. The Composite node is in [HDA]/TerrainTileOut/Output_Texture/ where it's the second last node. Thanks. Terrain_Spawner.hda
-
- heightfield
- terrain
-
(and 3 more)
Tagged with:
-
I want to start working on a short film next month and my plan is to create as much of it as I can in Houdini. If my film has characters with, hair, and clothing dynamics, what is the best workflow to handle this? Is it possible to have all the dynamics inside the character asset with a switch that turns them on and off? like an animation mode, and then a simulation/rendering mode?
-
Hi guys, I hope I can pick your brain a little bit. I would like to know how you guys manage your effects versions. My concrete scenario is a couple of strung effects. It starts with a RBD explosion sim, then a pyro sim off of that and finally advecting particles along the sim. I like to hold all these in the same geo which poses a bit of a problem. How do I correlate the cache version to the hip file version? My approach is to use this in the filecache node: $HIP/geo/$HIPNAME.`opname("..")`.$OS/$HIPNAME.`opname("..")`.$OS.$F.bgeo.sc (windows guy, so I like to keep all sequences in separate folders). This complicates if I don't want to redo the RBD sim for example, then I have to replace the $HIPNAME with the absolute name of the file which simulated the RBD version. I know it would be much cleaner if every sim would reside in its own file, but that is a little too small of a granulation for single person workflow. If you could shed some light/share your practices, it would benefit me immensely
-
Hello hive mind! I'm looking for a better solution to saving out my flipbooks as .mp4s. Right now I have ffmpeg installed so I can go directly from mplay and save an .mp4, but I cannot figure out how to change the preset for the menu so that the default file name matches my naming convention. Which is really annoying. I'm looking for any of these solutions! 1) Does anyone know how to update this ffmpeg window with a custom string in the "Output File" field? 2) Does anyone know where mplay stores the tmp images, so I could grab them straight from there and go to ffmpeg from a shell? 3) Does anyone have a better solution for flipbooking? Thanks!
- 2 replies
-
- ffmpeg
- naming conventions
- (and 6 more)
-
Hey magicians, I'm working on a project with a client that involves some large simulations, around 400gb stuff, my internet connection isn't the best on my city (10mb down, 5mb up), so I wanted to ask what's the best pipeline in this cases. The client gave me the idea to save the sim in a hard drive, and send to them over fedex or something, so they got all the stuff within some days and they can render there. Another solution I think can work is using some service like Gridmarkets where I gave them the files and they render out. Any ideas will be more than welcome Thank you!