Radish Trial 5 Custom Entities

From Nexus Mods Wiki
Revision as of 20:24, 29 July 2020 by Rmemr (talk | contribs) (ported trial 5 article to mediawiki format)
(diff) ← Older revision | Latest revision (diff) | Newer revision → (diff)
Jump to: navigation, search
The trial of the radishes is meant as a guided, self-learning tutorial without step-by-step instructions. Instead it focuses on exploratory learning by actively using the tools to solve increasingly challenging tasks.

>> Trail 5 focuses on creating custom entities <<.

This article provides some background information and various tips required or helpful to accomplish these objectives.

See forum post for more information about the trials.

Some background Information

All definitions used in radish modding tools projects are written as text definitions in yml format. As it’s basically “formatted text” it allows easy generation and “export” of radish-encodable definitions from the in-game mods via script log.

But more importantly since it’s human-readable (and arguably self-explaining) it also allows easy editing, tweaking and extending by mod authors in a text editor before the encoding step(s) generate a set of binary files from it.

Entity Presets

Manual tweaking and extending is especially interesting for entitytemplates and layerentity definitions which can get pretty complex and cannot be covered by an in-game mod like radish quest ui: its primary usecases are the interactive placement and (to an extent) configuration of *predefined* property sets (called “presets”) for layerentities based on *existing* templates.

These simplified “presets” can be identified in the dumped layer definitions by the “.preset” setting. But essentially the most simplest preset (with all optional settings not defined) is basically just an entity template path reference and a world position:

def simplest preset

There are only a couple of defined presets, each has a different predefined set of settings (see the example in the trial 5 download for valid settings). Prior generating binary files the encoder expands these settings into more complex entity definitions.

Generic definitions

Although there are plenty of ready to use vanilla entity templates available to choose from, sooner or later it will become limiting for slightly bigger projects.

At this point the more experienced mod authors would turn to editing of encoded files (templates or layer files) in modeditor or wolvenkit. However this requires the files to be already cooked as these edited files cannot be processed by wcc lite anymore. Unfortunately this also makes many things more complicated than necessary (e.g. manually editing compiled bytes, too) and misses the opportunity to let wcc do the heavy lifting.

Generic definitions as supported by the radish tools (quest) encoder allow to define entity templates and layerentities based on the information available from witcher 3 “rtti definitions” directly in the yml definition format (a stripped down version of the rtti definitions is provided in the trial 5 package).

At first this may sound more complicated than editing in modeditor but it has a couple of advantages:

  • all definitions are type checked by the encoder: every attribute name typo or wrongly used type will prevent encoding and thus avoid time consuming search for these unnecessary errors
  • since wcc can process the encoded files the compiled bytes are automatically generated, all referenced templates are merged as necessary, the files are checked by wcc (to a degree) and the result is a (more) consistent file
  • it also allows to catch errors early on by just checking whether wcc prints any errors instead of always verifying in-game if the file works (at all): if wcc complains chances are very high the file won’t work as expected
  • due to the automatic template merging the definitions can become significantly shorter (only the essential parts!) than defining a fully cooked entity (see below for examples of pre and post cooking)
  • and finally the text definitions can be easily backuped and compared with previous definitions with a simple diff tool

Here is an example of a short, generic definition extending a wraith monster with the ability to be used in scenes (something that is not possible with the vanilla template!):

def exended wraith

The “.type” setting in the “entityObject” refers to the class defined in the rtti definition. Which in turn defines all the *possible* attributes (and their types) for this entityObject. If the “.type” is omitted for “entityObject” the encoder will assume the most basic class that is valid for entityObject, in this case “CEntity” - which would restrict the set of valid settings. For example the “isInteractionActivator” setting is not defined for CEntity and the encoding would fail with an error:

encoder error unsupported key

After encoding the definition with the radish encoder the resulting template will basically contain only the above settings and some implicit definitions (like the entityClass):

uncooked entitytemplate

While the final processed output from wcc will contain 96 chunks - all automatically attached by wcc:

wcc-cooked entitytemplate

However a word of warning: it’s important to realize that this flexibility has also its drawbacks. Even if the definition is a manually crafted peace of art, valid based on the rtti definition and processed without any errors by wcc it still does not mean that the result will make any sense or that it will be working.

Mod authors will have to know what they do with this flexibility: this is mostly experimentation area.

And just to be clear: this does not work for arbitrary files - only the game engine entity and component classes are supported by the radish tools. The full list of all the supported classes is provided in the download package for trial 5. Also see below for the limitations of derived scriptclasses!

The intention of this trial is to get familiar with setting up generic definitions. Once these principle are clear it will enable defining new, pretty complex templates and layer entities.

Limited Support For Derived Scriptclasses

The radish encoder checks the attributes (names and types) of all supported classes based on the information from the rtti definitions - which only contain the exposed engine classes. However many of the classes used in scripts and many classes encoded in templates/layer entities are custom, derived classes defined in vanilla scripts (they use the engine classes only as a base class). Moreover custom classes may define custom attributes which are unknown to the encoder.

To reduce this limitation the encoder supports using a “subclass” of a supported generic class in the definition, for example:

.type: "CMyCustomActor:CActor"

This basically ignores the “CMyCustomActor” and checks the definition against the “CActor” class as defined in rtti but will store “CMyCustomActor” in the encoded binary as the class of this object. For the encoded template/entity to work in the game the new class (CMyCustomActor) has to be defined as a new class in the project mod.scripts folder and has to be a direct (or deeper) subclass of the defined base class (CActor).

Since all attributes of the definitions are checked against CActor this does not support usage of custom attributes in the definition for the encoding. As a workaround all custom attributes can be set to some values in the [code]OnSpawned[/code] method of the class directly in the script.

Entitytemplates And Layerentities

Generic definitions are only supported for “entitytemplates” and “layerentities”.

Layerentities are always defined as an individual, statically placed entity instance in a layer and can be either visible (e.g. static world decoration) or invisible (e.g triggerareas or actionpoints). As soon as the layer is activated all layerentities are “visible” to the game.

In contrast entitytemplates only define what components and settings a spawned entity will have. But the actual spawning and placement is defined somewhere else (e.g. in scripts, community definition, scenes…). All npc actors are defined as entitytemplates because these templates are often reused.

Layerentities often do reference entitytemplates but it is also possible (to a degree) to define layerentities in a layer completely from scratch.

Example: Actor With New Appearance

With generic definitions it is possible to define entities with arbitrary components. In fact such generic definitions with explicit component definitions are used in radish quest ui for more complex layerentities like environment areas or ambientsound areas exactly because of this flexibility.

Most of the time it’s easier to leverage existing templates and just tweak some aspects. It’s also possible to assemble a specific appearance from the common building blocks for some special npc.

example npc screenshot

The definition for the above npc looks like this:

example npc generic def

One thing to note is a slight simplification when compared to the rtti definition of CEntityTemplates and for CEntityAppearance.

rtti entity appearance

Strictly derived from the rtti definition the “appearances” setting would have to be defined as a list item and the CEntityAppearance also requires the “name” setting, like this:

def entityappearance-with-name

Using the name directly makes it slightly easier to find with folded subdefinitions. There are a couple of places where the encoder automatically derives the name from the list entry name and sets it as an attribute automatically (e.g. for CEntityTemplates the “appearances”, “bodyParts”, “slots”, “effects” lists). Whenever you get an error like for a list:

encoder error table expected

check if the class of the entries contains an attribute “name”.

Testing Custom EntityTemplates (SBUI)

The build pipeline puts the encoded and cooked entity templates always into the “dlc/dlc<MODNAME>/data/entities” folder (e.g “dlc/dlcanewmod/data/entities/new_actor.w2ent”) and also automatically generates the mounter for this directory.

The new templates can be directly tested after encoding, e.g. by spawning them with scripts (e.g. a helper script saved in mod.scripts-tmp folder).

Using the templates in SBUI (for defining scenes or checking if the set of required animations, mimics, lipsync work) is a matter of adding the template path into the “mods/modStoryboardUi/content/scripts/local/mod_additional_templates.ws” file from SBUI:

sbui add custom template

This will create a new category in the assetsmode for the custom templates:

sbui new actor template

Example: EntityTemplate With New Effect

The encoder also supports the complete set of classes for entity cfx effects. Since wcc will process the template all the effects will be automatically cooked into the cookedEffects attribute. Therefore adding new, custom(ized) effects to entity templates works like the previous generic definition example.

Since effects are slightly more complex (and longer) to define in yml it’s best to use existing effect as a (correct) base yml definition. The radish tools support dumping (parts! not all!) classes from the encoded binary files into yml definitions that can (after some manual adjustments) be used as definitions for encoding with the radish encoders again (see below for the procedure).

However embedded files or compiled databuffers are not directly supported - these need to be exported and saved as individual files prior dumping the definitions first.

Extracting Embedded Files

One possible way to get the effect data is to export the compiled data for a specific effect from the cookedEffects list with modeditor and save to a new file.

But it’s also possible (and more flexible if multiple embedded files should be exported) to use the “w3extract-embedded.exe” utility from the radish encoder package. Starting in a console window with the file to be scanned it automatically exports all (or only a subset filtered by a class search string) of the embedded files and saves them as individual files:

D:\tmp> w3extract-embedded.exe -e mq7023_megascope.w2ent -f CFX
w3 embedded files extractor v0.6.2
INFO - EXTRACT EMBEDDED: SCANNING mq7023_megascope.w2ent
INFO - scanning for files [mq7023_megascope.w2ent]
INFO - opening mq7023_megascope.w2ent...
INFO - > mq7023_megascope.w2ent: extracted 1 embedded files (filter: CFX)
INFO - finished extracting 1 embedded files from 1 source files.

Omitting the “-f CFX” would dump 3 embedded files from this particular w2ent.

Dumping Definitions From CR2W Files

The radish template project contains a batchfile that will try to dump the content from a cr2w file into the **_tmp** folder of the project (CR2W files are basically W3 file that can be opened in modeditor and contain editable chunks):

D:\w3.mods\modANewMod>dump_definitions.bat d:\tmp\mq7023_megascope.w2ent.001.cr2w

# --------------------------------------------------------------------------
# --------------------------------------------------------------------------

w2quest producer v0.6.2

INFO - DUMP DATA: SCANNING d:\tmp[/code][code]\mq7023_megascope.w2ent.001.cr2w
INFO - opening d:\tmp\mq7023_megascope.w2ent.001.cr2w...
WARN - decoding of creation time failed! expected range is [2010-01-01..now]. found: 1900-01-01T00:00:00Z
INFO - creating dumpable objects from red objectgraph...
INFO - saving object definitions in [d:\w3.mods\modANewMod\\_tmp]
INFO - finished dumping data.

Although dumping was just added for debugging purposes it can be used for getting some parts as base definitions instead of writing every part of a generic definition by hand. Since the encoder supports all CFX classes there shouldn’t be any unknown classes errors/warnings. But most other files will not work without errors and warnings - if at all.

All encountered unknown classes will be simply skipped by the dumper and thus the hierarchy of the definition will be broken and will require manual fixing. But the dumping may fail for various other reasons as well… (this was only an experimental debugging helper!)

The dumped output is considerably longer but it looks approximately like this (“object_0” is the “root” object of the file):

def fx-dump-example

Please note: the effects in this example are attached to a slot named “fx” that MUST be available in the template the effect will be added to - or it won’t work at all.

Simply adding the dumped effects definition into a new generic definitions template as a new effect with the name “new_effect” will look like this (look out for indentation errors!):

def fx-in-entitytemplate

However this will not encode fail with errors like

encoder error double write

This is the aforementioned merging of anonymous lists into “named” lists (see above). Since the encoder already derived the “name” of the element (flare) and there is an additional “name: flare” defined it is trying to set it again - which fails. A manual adjustment is required: simply removing all “name: <something>” will fix the problem.

Afterwards the encoding should produce a new working template with a “transplanted” effect that can be checked like in the previous example by spawning it:

entity transplanted fx

The unimpressive small light in the middle is the effect. Since the effect is working now it’s easy tweak the parameters. First let’s define a custom slot for the effect named “fx_abc” to be more flexible with the positioning:

def entity new fxslot

Every reference to the slot has to be updated in the effect definition as well (“componentName: fx_abc”).

Spicing up some parameters (e.g. the dynamic light color, radius and strength), exchanging the referenced particle system and increasing some of the interpolation values (just experiment…) will get something like this (screen captured during nighttime):

entity tuned fx

Testing Custom EntityTemplates As Layerentities (RADUI)

Since layerentities also reference templates radish quest UI supports extending the template selection for layerentities with custom templates. It’s as simply as adding the path to the appropriate section in "**mods_valuelist_additional.ws**":

radui adding custom template
MINOR TYPO IN PUBLISHED RADUI VERSION: MAKE SURE it’s “entity_templates” and not “entity_template” in line 28!

In the template selection list for any editable (not in encoded layers!) static/interactive layerentity a new category should be visible at the top of the list like this:

radui custom template

The docs.quests/test.entity.effect.cooking folder in the radish encoder package contains some dumped effects as definitions that were used to test to check the encoders - but it’s not guaranteed the effects work. You’ll have to experiment…

Tips Generic Definitions

  • the “type” does not need to be defined if it’s the default class as in the rtti definition for the attribute - if in doubt use it, though
  • but every time you see something with a “.type” setting you’ll know it’s a generic definition and can lookup in rtti to see what other variables are available, too
  • if the encoder complains that some attribute is unknown it can mean a couple of things:
    • the name of the attribute is invalid (check rtti definition)
    • the attribute is not in this class so maybe a subclass with more attributes has to be used
    • the indentation may be off and the encoders tries to attach it to the wrong parent/child definition
  • not all effects work on every actor, check component/bone/slot attachments in the original entity templates
  • the interpolation controls some parameters (different ones for different effect types - you’ll have to experiment) that are changed like in a timeline, so tweaking those can get some timed modifications on the effect

Tips Definitions

  • the top level setting “entities” was used in older versions of definitions (e.g. in published quest mod). This was too easy to confuse with layerentities. It is now a legacy setting that is an alias for “templates”.
  • additional mounters are not needed, encoder generates entities in data/entities subfolder and also autogenerates a default mounter for this directory

Tips Radish Quest UI (RADUI)

  • look out for the color of the layername: orange means it is a read-only layer and it or its entities cannot be edited. It has to be “cloned” for editing. (see trial 2)

Tips Build Pipeline

  • Generic definitions tend to be much longer. Putting them in (multiple) separate files is advisable.

Tips Project Template

  • as a reminder: the “additional” folder is really only for files that are manually edited in modeditor and that cannot be cooked afterwards by wcc. Or for files the radish encoders cannot generate at all. DO NOT copy anything from uncooked into additional: you would have the UNCOOKED files in the game. All the time!
  • you can setup a modeditor project to point to the resources/dlc<MODNAME> directory to have quick access to the COOKED files. But beware: don’t save anything there as those files and directories will get overwritten/deleted. Instead in this case put the files into the appropriate subdirectories in the additional folder