Modular Level Analysis Tool

Step 1: Learning Python

Time to teach myself some scripting! This is going to be fun :)

Coming up with a plan

This tool is a little bit harder to plan than my Houdini tool as Python's capabilities in UE5 are still relatively unknown to me at this point. While I'd love to come up with a detailed brief for myself to work towards from day 1 I think in this case some upfront research will be necessary. However, despite not knowing the details of how it's going to work I do know what I want my script to achieve; I would like to have a tool that will put all the useful optimisation information I am likely to use regarding my modular pieces in one place. This will likely mean gathering a bunch of data and then outputting that data to a file or a spreadsheet - so one specific thing to research is how to write files in Python. Some examples of useful data (which I have no idea if it is possible to get via Python) would be the number of instances of a specific mesh, the name of the mesh or its location, it's triangle count, if it has LODs and maybe the number of materials. I could use all this information to then get a good idea of relative cost, maybe even coming up with a formula to automatically calculate the number of drawcalls or something based on the material slots.

Initial research

For my initial research I looked over the Unreal Python API documentation and then watched a great course from Epic Games about using Python for Unreal Engine Editor Scripting. I discovered there's a method that be used to count instances and I realised this could be really useful for my level where there would potentially be a lot of instances of the corridor modular pieces. If I can create a tool with a nice UI that allows an artist to get instance data like the name of the actor, the mesh and the number of instances in the scene, and then output that data to a spreadsheet so it can all be viewed in one place that would be really useful from a scene analysis and optimisation standpoint.

Testing out what I learnt

Interestingly, at this point despite its limited functionality this tool helped me discover a significant issue in my level - one of the instancers on my corridor tool was creating duplicate walls. Without this tool I might not have noticed! So that was serendipitous. I fixed that issue and you can read about it here.

Counting instances & exporting data

Based on this first little function I feel like I now understand the basics of Python scripting pretty well. I like that the syntax is really straightforward although I do keep putting semicolons on the end of everything out of habit. As my scene in Unreal continues to develop I am going to come back and add more functions to this script. My plan is to then work all these snippets into a UE5 editor utility/blueprint widget which should allow me to make this script into something with a useable UI rather than my current method of calling it straight from the command line which I imagine wouldn't be very appealing to a non-technical artist or level designer.

Bonus Script #1: Bulk Material Assignment

While creating my modular pieces I realised I needed to change their basic material to a metal material to check how my weighted normals were looking in-engine. Now I have about 15 basic modular pieces and manually changing all those materials and then manually changing them all back seemed like it would waste a lot of time. So, I thought to myself can I work this into my Python tool? The answer was yes. The only part that was tricky about creating this was figuring out how to reference the material. As you can see in the code below Line 77 I initially tried to get the reference you can copy from the editor library and then convert it into a usable reference but this didn't work. Thankfully I discovered the load_asset() function which meant I could quite easily get a useable reference from the original input.

I am really enjoying the freedom learninig just a bit of basic scripting has given me. Creating this little function is going to save me so much time on this project - I'm imagining scaling that time-saving up to the size of a AAA project and realising just how useful this is to know.

Integrating into the Editor

Up until now I have been calling my python scripts directly from the command line in engine; this is fine by me but for other users a slightly more friendly approach is warranted. I decided to use Unreal's Editor Utility Action blueprints to integrate my code into the editor - this allows for seamless execution with a simple right click and menu selection. I can even set appropriate contexts for when I want each script to appear (for example changing materials in the asset browser, counting instances in the level outliner).

I used this GDC talk by Matt Oztalay to get to grip with the basics of how these blueprints work and then simply used the premade Execute Python Script node to run my code. However, it didn't work. Trying to integrate this code seamlessly was slighly more complex than just copy pasting. Because I wasn't typing in my commands manually I wasn't able to input the data I needed in the same way by typing in the arguments. This meant I needed a new way to get that data.

Another change I made was changing the way the data exports. Previously I had one array with all the data which I ran through in a loop to write out a csv all in one go. This was messy and inefficient and I knew it but it did work for testing purposes. However, I wasn't happy with it and I knew I could do it more efficiently and in a way that is more helpful for anyone else who might look at or use this code. This is what I did:

• I separated out the initial creation of the csv file into its own executable script so that a blank file would be created at the beginning of each execution of this blueprint.

• I changed the writing of the rows to be in append mode rather than write mode so that no lines could be accidentally overwritten

• Instead of writing all data to one big array of varying types (which UE was not a fan of) I separated out each piece of data into its own variable which is named accordingly

• For each asset that is analysed, the data pieces are now saved to their variables and then immediately appended to a new row. The values are reset at the beginning of each loop to "error" so any missing data in the file is clearly noted.

In addition to this I also figured out how to get a file path input from the Unreal Editor as file path is already a datatype in engine. This was a big improvement on the original hard coded file path (which would break if anyone other than my user was using it) and an improvement on having to manually copy/paste the file path AND remove the quotes around it. File paths were surprisingly complex to handle but now I know how important distinguishing between absolute and relative file paths is.

Bonus Script #2: Bulk Renaming Assets

Looking at the spreadsheets above you may, like me, notice that I neglected to follow some naming conventions on my static meshes. The wall modules don't have the prefix SM_ as they should. This needed to be rectified immediately! But manually changing names sounded laborious and I am enjoying Python too much to stop now so I decided to try and create myself a simlpe batch renaming script. This would work similarly to my material reassignment tool but instead of chaning materials I would change names. I needed a way to specify just changing or adding a prefix rather than just renaming eveything all at once which would be less helpful. So I did some research and found it to be suprisingly straight forward yet again as Python has an inbuilt replace function!

Refining the Script (Minimizing Blueprint Reliance)

Last week when trying to transfer my scripts from using command-line input to utilising Unreal's Editor Utility Action blueprints I was having difficulty getting the input parameters to pass into the function correctly and so became reliant on blueprint nodes to do it for me. However, this is not what I wanted! This is a project to teach me scripting and it would be remiss of me to give up and fall back on blueprints at the first sign of a challenge. So, this week I dedicated my time to figuring out how to integrate my script fully and eventually I got it working! I had learned how to pass in string variables from creating my bulk renaming tool and expanding that logic I was able to get the rest of my scripts working in the same way.

After getting to this point I wanted to add in some of the extra functions I had drafted up in blueprints such as getting the number of materials. Now that I am more familiar with Python and its Unreal API adding new features was a much quicker process.

Bonus Script #3: LOD Analyser

After integrating LOD and Tri count into my initial tool I decided that I wanted to have a dedicated LOD checking tool that would allow me to see the number of LODs as well as the triangle count for each LOD. For a large scale project being able to see this data with a click would be really beneficial when it comes to optimisation. The initial tool would allow to check if there are any LODs at all (which is useful for managing performance) but this new tool would allow me to compare tri count across different LODs which is useful when it comes to disk-space optimisation. If two LODs for an asset only have one triangle difference then we don't need to keep both - it would just be a waste of space for essentially no visual or performant difference.

Creating this tool was relatively easy, however getting it to compensate for different numbers of LODs proved to be a bit of a logical challenge. I knew I needed to run a loop n times where n is the number of LODs (actually I needed to run it n-1 times because LOD0 is the first LOD. I decided to use a while loop for this and basically append the tricount value for each LOD to a list while the condition was true. Here is where I nearly blew up my computer illustrated in pseudocode:

n=0

While n < numlods:

tricount = mesh.get_num_triangles(n)

list.append[tricount]

Can you spot what I missed? Because I did as soon as I tried to run the code and my laptop fans got suspiciously loud all of a sudden. I forgot to iterate my iterator! The loop was doomed to run indefinitely and cook my machine so I killed Unreal Engine as fast as possible to stop the process, and upon restarting it immediately added a line to set n = n+1 so this wouldn't happen again. I had somehow cooked UE5 though and couldn't navigate to or open any assets so I had to reboot my whole machine. Oops! 

Today I learned a valuable lesson. Always double check while loop conditions.

Project Reflection & Analysis

I have learned so much through the execution of this brief and teaching myself Python for Unreal has been a hugely fun challenge. Although, to be honest the hardest part of learning Python was just figuring out how to navigate the Unreal Python API, once I understood that the rest was fairly straightforward. Realising that I could also draft my Python scripts in blueprint functions was also incredibly helpful in allowing me to get a sense of what is possible with this tool.

In terms of the code itself I think it is clearly structured and commented well - I've tried to make it as straightforwardly readable as possible. It is also efficient, I haven't wasted any lines or got any empty or unused functions lying around. However, some of the tools don't incorporate any kind of error messaging system, instead relying on Unreal's native output to call out any errors. I think this is most pertinent to input handling - on a base level this is taken care of by restricting the types eligible to use the Scripted Asset Function. For example, the material script only working for Static Meshes makes sense. However, the Instance Counter Script is only limited to actors in the level hierarchy which is a broader category. Right now, if an actor with no InstancedStaticMeshComponent is selected then the tool will default to outputting 1 for the number of instances, assuming that this is simply a single static mesh. However, I imagine there are cases where this would be wrong - for example what would happen if a light actor or sound actor were accidentally included in the input selection?

There is also the issue of a bug in the material assignment script that I couldn't quite fix before my deadline. The script works fine to assign the materials but occasionally when using the undo function within the unreal editor it only undoes about half of the material assignments, leaving a mixture of assets with the new and old materials. This is especially unhelpful just from a consistency standpoint. By default Unreal does support undoing python functions, and 80% of the time it does function correctly. I have attempted declaring the integrated blueprint version as a distinct transaction to make it clear but it still inconsistently undoes the reassignment. So for now I have no solution to this, but most of the time it works fine and I am going to keep an eye out for any potential solutions in the coming weeks.