Procedural asset creation using lindenmayer-systems

Lindenmayer systems have traditionally been utilized sparingly in content pipelines, but L-systems may be applied across content pipelines adding procedural capabilities to whatever they are used for. Typically L-systems have been utilized for modeling organic growth in the way of trees and shrubs, as well as the creation of two-dimensional fractal artworks. Such use cases have proven beneficial to content creation, yet have relegated L-systems far below their capabilities. L-systems offer content creators an inherent ability for procedural creation to most any part of their content pipelines. This thesis will discuss how L-systems may be applied to flipbook generation, particle system emitters, anatomical modeling, and creation of visual effects. These uses will serve to demonstrate the propensity L-systems have to be applied across content pipelines.

PROCEDURAL ASSET CREATION USING LINDENMAYER-SYSTEMS by Christopher Reay A Senior Honors Thesis Submitted to the Faculty of The University of Utah In Partial Fulfillment of the Requirements for the Honors Degree in Bachelor of University Studies In Engineering Approved: ______________________________ Mark van Langeveld Thesis Faculty Supervisor _____________________________ Jim Agutter Chair, Department of University Studies _______________________________ Erica Rojas Honors Faculty Advisor _____________________________ Sylvia D. Torti, PhD Dean, Honors College April 2018 Copyright © 2018 All Rights Reserved ABSTRACT Lindenmayer systems have traditionally been utilized sparingly in content pipelines, but L-systems may be applied across content pipelines adding procedural capabilities to whatever they are used for. Typically L-systems have been utilized for modeling organic growth in the way of trees and shrubs, as well as the creation of twodimensional fractal artworks. Such use cases have proven beneficial to content creation, yet have relegated L-systems far below their capabilities. L-systems offer content creators an inherent ability for procedural creation to most any part of their content pipelines. This thesis will discuss how L-systems may be applied to flipbook generation, particle system emitters, anatomical modeling, and creation of visual effects. These uses will serve to demonstrate the propensity L-systems have to be applied across content pipelines. ii Table of Contents ABSTRACT ...................................................................................................................... II L-SYSTEM BACKGROUND .......................................................................................... 1 ARTISTID LINDENENMAYER ............................................................................................. 1 UNDERSTANDING THE RULES OF L-SYSTEMS ................................................................... 2 TRADITIONAL USAGE OF L-SYSTEMS ................................................................... 6 MODELING ORGANIC GROWTH ........................................................................................ 6 FRACTALS ........................................................................................................................ 9 DIFFERENT OPTIONS FOR L-SYSTEM USAGE ................................................... 11 MODELING LUNGS AND BREATHING USING L-SYSTEMS ................................................. 11 USING L-SYSTEMS AS EMITTERS .................................................................................... 13 USING L-SYSTEMS FOR FLIPBOOKS ................................................................................ 14 USING L-SYSTEMS FOR PROCEDURAL ASSET CREATION ........................... 15 PROCEDURAL LIGHTNING CREATION FOR GAME USE .................................................... 15 PROCEDURALLY MODELING DNA ................................................................................. 17 TRADITIONAL APPROACHES TO PROCEDURAL ASSET CREATION ......... 20 SPEEDTREE .................................................................................................................... 20 TREEIT ........................................................................................................................... 22 PHOTOGRAMMETRY ....................................................................................................... 24 CONCLUSION ............................................................................................................... 26 WORKS CITED.............................................................................................................. 27 iii L-system Background Introduction Lindenmayer-systems are a powerful tool for the creation and modeling of growth and hierarchical structures within the realms of two dimensional and three dimensional math. This ability to model complex structures using specific growth algorithms allows for the creation of highly procedural assets which can be built on the underlying structure and formulas of the original Lindenmayer-system. Furthermore, the underlying Lindenmayer-system may be further manipulated through the usage of outside model deformation tools while maintaining their core procedural abilities. These techniques allow for the creation of highly complex creations which can be spontaneously rebuilt or reorganized. Lindenmayer-systems have the propensity to be used for most any visual effect, and corresponding with their procedural nature it is possible for individuals without an understanding of modeling or Lindenmayer-systems to still create their own effects through the use of precompiled rulesets that allow specific manipulation of the underlying structure. Lindenmayer-systems are an incredibly powerful tool for the creation of procedural assets. Through a variety of techniques this paper details how these systems may be applied to most any visual effect a creator would wish to make. Artistid Lindenenmayer In 1968 Aristid Lindenmayer published “Mathematical models for cellular interaction in development”. In this he postulated a theory by which cell formation could be modeled through the inputs of their neighbors. These growing filaments would allow various developmental patterns to arise. The belief was that through the creation of these patterns scientists could accurately model the growth of various biological organisms. The creation of a mathematical system by which branching models could be created was a huge milestone for collaborative work between science and graphic technology (Lindenmayer). Therefore, this contribution cannot be discounted when discussing the current approaches to his original idea for modeling complex growth through the utilization of mathematical protocols. Understanding the Rules of L-systems The rule system of L-systems may change based upon the graphical system being used to model the interactions, but the thought behind the creation of these systems can be surmised as follows. L-systems are a formulaic approach to branching with points being created based upon the position of their previous iteration. These points can be filled with geometry, drawn out to make lines, or remain unfilled to allow a nonconnected branch to form. Where these points are placed in relation to the distance from their previous point, how they are orientated in terms of rotation and three-dimensional position, and what color the point is displaying are all controllable tools within the package used to model L-systems for the purposes of this paper. The package that will be used to demonstrate L-systems is Houdini which was created by SideFX. Its use has largely been confined to movie production and recently game production. However, its power can just as easily be applied to scientific situations for visualization of the growth of structures ranging from organs to molecular interactions. The rules for Houdini’s Lsystems are built on turtle command references and are explained in figure 1. 2 3 4 Figure 1: Houdini L-system Ruleset (L-system Geometry Node) Figure 1 details functions which allow for the creation of branching systems by way of determining the placement of a new node and determining whether to draw out to that node. If geometry is created then symbols can be used to determine what color to display over the node. This thought process allows for the basis of what the tools do and how to use them. However, the capabilities of math and programming allow for even more complex relationships to form. Systems can be formed in a variety of fashions such that, so long as they can be understood by the compiler, there is no limit to the ways in which these systems can be formed. An example system can be seen in figure 2. Premise: FF[A]FFA Rule 1: A = +(45)&(45)F Figure 2: Example L-system 5 The system in figure 2 would draw out growth twice through the use of F, then create a branch and end the branch based on the rules of A, then draw out two more growths before calling A again. “A” will draw out a branch in three-dimensional space at an angle of 45 degrees allowing it to separate from the initial structure and then further separate out into three dimensional space. This is the fundamental basis of an L-system with growths being created and ended in a sequence to add geometry to the whole while separating branching geometry from the whole through the use of angling future geometry away from the original structure. Traditional Usage of L-systems Modeling Organic Growth L-systems have been a great asset to game developers who wish to model trees and other such organic growth. The flexibility and speed by which these highly utilized environmental assets can be created has allowed for assets to be extremely variable, so that they can even closely resemble real world locales. To give an example of the ease to which game developers may create a tree through the use of an L-system a tree was created and imported it into unreal with corresponding leaves also created from Lsystems. Figure 3 and 4 detail the ruleset used to create this tree along with pictures for visual reference of the tree. For texturing an autouv was used within Houdini and then brought the asset into Substance where I applied a wooden material. The algorithms for this tree came from SideFX, but have been detailed them below. The creation of these assets did prove problematic, as the models themselves proved to be a little too complex for an autouv system to work properly.. Looking to 6 other programs, such as Speedtree or TreeIt, show techniques that may be applied to fix these issues. These programs seemingly UV the tree in segments as it is built with the primary branching portion of the tree, or trunk in this case, being modeled and UVed separately from the branches and even upper trunk. Such a technique would theoretically allow for complex trees to be created and textured correctly, but is out of reach for this paper. Instead, the current process has some issues wherein textures look stretched and distorted on the model, due to the imprecise way in which their uvs were created. When it came time to add leaves the abilities of L-systems proved extremely helpful. All that had to be done was the creation of a leaf model using L-systems, mainly for the purposes of this paper. Usually, what would be done is billboards would be added where the leaves would appear and those billboards would then be setup to display images of leaves. A billboard system like this can be seen in figure 5. This style of system is also displayed below without the billboards being setup to display images of plant life. The leaves that were created using L-systems used few polygons and allowed a great flexibility when it came time to animate them within Unreal. Creating the leaves and baking them out as models meant a higher cost would be had from their utilization then billboards. The animations that were able to be created did have some advantages though, with the higher geometry amounts allowing for more stretching and movement then would be had from billboards. However, the usage of either billboards or models is not difficult to setup as the creator of the L-system may insert the leaf, in this case J, into the system wherever they might want to display a billboard for their leaves. 7 Premise: A(1,10) Rule 1: A(l,w)=F(l,w)[&(c)B(l*b,w*0.707)]//(180)[&(d)B(l*0.7,w*0.707) Rule 2: B(l,w)=F(l,w)[+(c)$B(l*b,w*0.707)][-(d)$B(l*0.7,w*0.707)] Figure 3: L-system ruleset for modeling a tree (L-system Geometry Node) Figure 4: Multiple angled view of tree with and without leaves. 8 Figure 5: Tree with billboards used to represent leaves before material is applied Fractals One of the most interesting uses of L-systems is their use for the creation of abstract mathematical art, which have been coined fractals. Fractals are a fun way to learn and grow ones understanding of L-systems, because these systems tend to be as complex as the creator wishes them to be. Fractals can be exceptionally artistic and their use of Lsystems for creation lends them a procedural nature. Due to this fractal 2D systems can be utilized for masking effects, creation of maps, art creation like the Koch fractal below, and for spawning of different effects. Abstraction of fractals into the three dimensional space can be just as dynamic as their creation within two dimensional space. Through the use of precision randomness in angles and growth procedural fractals may even be grown in this three dimensional space. The creation of a Koch fractal is shown using the ruleset in figure 6 with visualization in figure 7. This tool being just another extension of the usefulness of L-systems. 9 Premise : F-F-F-F Rule 1 : F=F-F+F+FF-F-F+F Figure 6: Koch fractal ruleset (L-system Geometry Node) Figure 7: Koch Fractal visualization 10 Different Options for L-system Usage Modeling Lungs and breathing using L-systems L-systems have traditionally been confined to mathematical and graphical fields, however there is no reason why this must be the case. L-systems can just as easily be applied to modeling complex anatomic structures for the purpose of teaching and visualizing interactions in the sciences. These visualizations may prove invaluable in the future as imaging techniques become utilized at an increasing rate, with a corresponding need for adequately trained individuals to utilize them. Figure 8 details a ruleset utilized to create the internal workings of lungs. To get the frame by frame effect of breathing the sphere that was used as a leaf had its size set to sin($F) which causes it grow and shrink over the frames following the nature of a sin wave. Using these ideas L-systems may be expanded with relative ease. 11 Premise : !(4)F(.05)AA!(.8)FFF[B] Rule 1 : A=F(.02)!(.5)F(.02)!(2)F(.02)!(.5)F(.02)!(2)F(.02)!(.5)F(.02)!(2)F(.02)!(.5)F(.02)!(2) Rule 2 : B=&(50)[C]&(-100)[D] Rule 3 : C=!(.8)FFF&(30)[L]!(.2)F+(45)[M]-(90)[M]+(45)&(45)[M]&(-90)[M] Rule 4 : D=!(.8)FFF&(-30)[L]!(.2)F+(45)[M]-(90)[M]+(45)&(45)[M]&(-90)[M] Rule 5 : E=!(.8)&(45)[Y]&(-90)[Y] Rule 6 : M=!(.4)F(.05)J Rule 7 : Y='!(.4)FFF+(45)[M]-(90)[M]+(45)&(45)[M]&(-90)[M] Rule 8 : K=!(.8)FF[E]'!(.4)FFF+(45)[M]-(90)[M]+(45)&(45)[M]&(-90)[M] Rule 9 : L=!(.8)&(45)[K]&(-90)[K] Figure 8: Ruleset utilized for the creation of a set of branching lungs Figure 9: Lungs breathing visualization 12 Using L-systems as Emitters To create the initial structure of the effect for figure 10 a Koch L-system was utilized as a base mesh from which particles could be spawned. Utilizing this thought process further abstracts the usage of L-systems by allowing individuals to create dynamic particle systems based upon the makeups of their L-systems. These L-systems thus allow for visual effects artists to create dynamic bases from which to build their effects. Such bases are instrumental throughout the particle creation pipeline wherein they could be used for guiding, spawning, or manipulating particles. Therefore, the use of L-systems for the creation of procedural pathing algorithms could serve as a viable tool for visual effects artists that wish to speed up the flow of the pathing for their sprite animations. Figure 10: Koch system spawning particles 13 Using L-systems for Flipbooks Flipbooks are an asset that directly benefits from procedural tools, because they are just a collection of assets that when displayed properly creates an animation. With this in mind a flipbook was created using lightning and a placement along the curves of an octagon following a half-bend with a forty-five degree turn, this asset is detailed in figure 11. In this way creating an octagon became procedural as once its base form was created that form could be adjusted to do whatever was wished of it. In this case, each vertex of the octagon was utilized to display a model of lightning rather than the typical straight vertices stretch it would’ve contained. Doing this within Houdini meant swapping out the F used to draw the vertex for a J from a lightning L-system, which is detailed later in the paper. Utilizing L-systems in this way allowed for the steady production of flipbooks, as they could be baked and created on the fly with different seeding. Figure 11: Flipbook created utilizing L-systems 14 Using L-systems for Procedural Asset Creation Procedural Lightning Creation for Game Use The use of L-systems for the creation of effects in games cannot be discounted, for they solve many of the issues faced when creating effects. These issues being a need for variance within the effect and scalability of its use. L-systems answer the variance problem through their procedural nature and ability to be scaled dynamically based upon how the creator wishes them to be baked. Therefore, L-systems offer a solid solution for effect artists that wish to create dynamic effects through the utilization of geometry specifically, but other avenues should not be discounted. The following figure details an effect and its ruleset that were built within Houdini, exported using their vertex based animation tools, then imported into Unreal. For the emissive color linear interpolation along with a sin function being driven by time to create a variable alpha was used to lerp between the color of dark blue and light blue. A multiply with a scalar parameter was then used to allow for the adjustment of how emissive the effect would be. Many of these factors are therefore also procedural, so that a designer could adjust how emissive, what colors are being interpolated between, the size of the effect, how long it takes for colors to switch, how fast the effect plays, and things as basic as its metallic and roughness values. Which is to say that these effects are not limited to being procedural just within the sphere of their creation, as they can be further adjusted down the pipeline to make even more dynamic visual effects. 15 To spawn lightning Houdini’s built in lightning L-system was used which follows the ruleset in figure 12. To expand the system into an effect emitting from a sphere in all directions you can use the following ruleset along with two leaf inputs, the L-system lightning and a sphere were used for the example detailed in figure 14. Recall, tilde (~) is used for random angles, so in both of these systems it is used along with a frame reference to get variance across frames. Premise : A Rule 1: A=-F+A:0.45 Rule 2 : A=+F-A:0.45 Rule 3 : A=~(30)[--"!A]A:0.05 Rule 4 : A=[++"!A]A:0.05 Figure 12: L-system lightning ruleset (L-system Geometry Node) Premise : KA Rule 1 : A = ~[J] ~[J] ~[J] ~[J] ~[J] ~[J] ~[J] ~[J] ~[J] Figure 13: L-system ruleset used to randomly spawn lightning across a sphere 16 Figure 14: L-system lightning visualization within Unreal Procedurally Modeling DNA In the same way that effects may be procedurally created for games models and effects may also be created for the sciences. In the following example DNA is created procedurally through the authoring of a formula akin to entering the nucleic acids themselves to create the structure of a strand of DNA. This type of system would allow for scientists to visualize DNA information in an interesting fashion. For they would not only be able to view a strand of DNA, they could author the DNA themselves creating the effect based on the wishes they have rather than the wishes of the developer. The ruleset in figure 15 can be used to create a straight strand of DNA procedurally. There are two inputs used for leafs to create the actual Adenine, Thymine, Uracil, and Guanine bases which are called using J and K. To accomplish the animation of twisting the DNA a twist node within Houdini was used whereby adding a frame reference to the twist factor the DNA would be twisted frame by frame. 17 Premise : &(90)G(-1)&(-90)[A]&(90)G(2)&(-90)[D] Rule 1 : A=EEEEEWWWEEWWWWWWWEEEEEWWWEEWWWWW WWEEEEEWWWEEWWWWWWWEEEEEWWWEEWWWWWWWEEEEEW WWEEWWWWWWWEEEEEWWWEEWWWWWW Rule 2 : D=NNNNNTTTNNTTTTTTTNNNNNTTTNNTTTTTTTNNNNNTT TNNTTTTTTTNNNNNTTTNNTTTTTTTNNNNNTTTNNTTTTTTTNNNNN TTTNNTTTTTTT Rule 3 : C='^(90)F(.5)!(.8)F(.15)J Rule 4 : B='^(-90)F(.5)!(.8)F(.15)J Rule 5 : E=F(.5)[B] Rule 6 : N=F(.5)[C] Rule 7 : Q='^(90)F(.5)!(.8)F(.15)K Rule 8 : R='^(-90)F(.5)!(.8)F(.15)K Rule 9 : T=F(.5)[Q] Rule 10 : W=F(.5)[R] Rule 11 : Y=^(50)F Figure 15: Ruleset used to construct a strand of DNA procedurally 18 Figure 16: DNA animation visualization 19 Traditional Approaches to Procedural Asset Creation SpeedTree SpeedTree was released in circa 2000 as an answer for procedural creation of trees, using an approach very similar to L-systems. Previous thesis’ have argued that SpeedTree uses L-systems, but finding data to substitute these claims has proven difficult as the actual code is safeguarded. The first usage of SpeedTree came in the 2002 release of The Elder Scrolls: Oblivion. 2009 saw the first usage of SpeedTree in films with Avatar (SpeedTree). Therefore, usage of SpeedTree has just recently been usable for games and media and that is largely due singlehandedly to computational performance. Simply put the computing power was not there to utilize procedural assets, with assets instead being created entirely by artists. However, with real time ray tracing being projected for release in video games this year, the idea of computational limits has begun to steadily decline (Lilly). With modern processing power L-systems may be utilized relatively freely and as computational power improves so should usage of L-systems. SpeedTree is a program which was developed with the creation of trees and other organic growth in mind. The program allows for integration into many visualization programs, so it has pretty open usage. Detailed in figure 17 are examples of organic growth which may be created within the program. These trees are noticeable cleaner looking then those detailed earlier in the paper. Largely this is because of their usage of better UV tactics, and better textures which helped both their trees and leaves. The program is procedural in nature allowing owners to create a wide array of organic models and change their models using random seeds to generate great deals of content at a quick 20 pace. Therefore, SpeedTree seems to be extremely capable for the creation of organic branching structures, but must have its limits accounted for (SpeedTree). Houdini offers a more robust underlying system which can be manipulated to suit the creators desire. Yet, SpeedTree offers a suite that is already setup for the creation of specific types of assets. For the creation of these particular assets SpeedTree does offer an inherent leg up on Houdini for procedural asset creation, as most of the underlying rulesets used to modify these trees are built into SpeedTree so that most anyone could make trees within the program. However, SpeedTree does not cover everything as it is currently limited to the creation of organic assets, whereas Houdini offers multiple animation tools on top of these assets. 21 Figure 17: SpeedTree trees TreeIt TreeIt is a program that is free to download and create assets with. It is similar to SpeedTree in that it is utilized for the creation of organic branching growth and can be utilized for the procedural creation of these assets. However, the assets and editor are noticeably weaker than SpeedTree. That is not to say these assets could not be further manipulated in an outside program to achieve the same textures as those found in SpeedTree, it just lacks some of these at the start. TreeIt does supply users with a good 22 deal of starting content to figure out the process of creating textures, but it is far more focused on the asset creation part of the pipeline. This is where TreeIt truly shines, and it is fair for a program to have these priorities with many studios choosing to use their inhouse textures. An example of content that can be created in TreeIt can be found in figure 18. This example details bark textures which compete with SpeedTree along with a modeling system seen in Figure 19 which is more procedural then the one found in SpeedTree. SpeedTree offers more flexibility with what can be created in their program, yet the usage of techniques like sliders make TreeIt a shining example of how one should make a procedural program. TreeIt does not take a great deal to get introduced to as all that a user has to do is toggle sliders, which can also be saved as presets (TreeIt). However, what both SpeedTree and TreeIt really show is that content creators have an appreciation for tools which allow them to procedurally create branching models. If that logic were to be applied to the particle systems detailed earlier then further use cases for procedural modeling would arise. Figure 18: TreeIt tree in Unreal 23 Figure 19: Modeling pane for TreeIt Photogrammetry Photogrammetry is an approach that is becoming increasingly common. Detailed in figure 20 is an image released by Ubisoft to demonstrate their usage of such techniques in the recent Far Cry 5 release (Reparaz). Photogrammetry can be used for the creation of most any real world asset into three dimensional space, so it is procedural in the way that any object may be integrated into computer graphics. However, reference objects are necessary and there are usually touchups that must be done to ensure the content is the correct resolution and shows up correctly. Due to these issues photogrammetry lags in the procedural department. Photogrammetry is still an exceptional tool though, as shown below its usage can allow for real world locales to be represented in game space. Using 24 real world locales is a very powerful tool for developers who seek to develop quality content quickly as they are able to capture the best parts of any locale and integrate them into games. These techniques are becoming increasingly common as developers seek to push out content of larger scopes in less time. Figure 20: Ubisoft Photogrammetry (Reparaz) 25 Conclusion Lindenmayer systems have traditionally been utilized sparingly in asset production, but they can be applied across content pipelines adding procedural capabilities to whatever they are applied to. The typical use cases for L-systems have proven that L-sysems have value for content creators wishing to add procedural capacities to their pipelines. Furthermore, the uses discussed in this thesis show that content ranging from flipbooks to anatomy may be created with procedural capacities in mind. At their base the L-systems utilized in Houdini can run into some issues, these being seen particularly in how they competed against SpeedTree. However, L-systems are not limited to just modeling organic growth. Photogrammetry is another tool that could fill this gap, as these systems are not limited to one particular type of model, except they too run into issues where something needs to be created which cannot be pictured at the details required for creation or the content that is wanted does not exist in the real world. In this way, L-systems can fill the gaps that are left over from these techniques. Such that, L-systems may be applied in the creation of effect and model pipelines, with the added benefit of being able to contain procedural rulesets which could be altered as necessary. The variability of these procedural rulesets, corresponding with the tools contained in Houdini for visualization make L-systems an excellent choice for the procedural creation of specific assets which may be unable to visualize through other procedural toolsets. 26 Works Cited L-system geometry node. (n.d.). Retrieved from http://www.sidefx.com/docs/houdini/nodes/sop/lsystem.html Reparaz, M. (2017, November 15). Far Cry 5 - How Hope County was built into a believable slice of Montana. Retrieved fromhttp://blog.ubi.com/en-GB/far-cry-5how-hope-county-was-built-into-a-believable-slice-of-montana/ Aristid Lindenmayer, "Mathematical models for cellular interaction in development." J. Theoret. Biology, 18:280—315, 1968. Inc, IDV. “SpeedTree.” Speedtree, 8, 23 Oct. 2017, store.speedtree.com. Accessed 21 Apr. 2018. Software, Evolved. “TreeIt.” TreeIt, 12 Feb. 2018, www.evolvedsoftware.com/treeit/treeit. Accessed 21 Apr. 2018. Lilly, Paul. “We Might Be Playing Games with Ray Tracing by the End of the Year.” Pcgamer, PC Gamer THE GLOBAL AUTHORITY ON PC GAMES, 30 Mar. 2018, www.pcgamer.com/we-might-be-playing-games-with-ray-tracing-by-theend-of-the-year/. 27