Radish Trial 6 Advanced Scenes

From Nexus Mods Wiki
Revision as of 16:49, 1 August 2020 by Rmemr (talk | contribs) (added looks-as section)
Jump to: navigation, search
The trial of the radishes is meant as a guided, self-learning tutorial without step-by-step instructions. Instead it focuses on exploratory learning by actively using the tools to solve increasingly challenging tasks.

>> Trail 6 focuses on creating more advanced scenes <<.

This article provides some background information and various tips required or helpful to accomplish these objectives.

See forum post for more information about the trials.

Some background Information

In the previous trial the focus was set on interactive dialogue scenes and the dialogue flow. Now it’s time to concentrate on the presentation, that is: fine-tune animations, mimics, voicelines and camera framing. This will not only make dialogue scenes more interesting but also allow to create more engaging cinematic cutscenes which present questprogress without player interaction.

Cinematography Techniques

Adjusting animations and camera framing is only one part of the story. A cinematic scene should also have some visual structure to support the narrative. This a very broad topic and way beyond the scope of this article. As a start read at least this short introduction which touches some aspects as this article only focus on the technical side.  

Rule of Thirds

One simple cinematography technique which is also mentioned in the above introduction is the “Rule of Thirds”. It’s a rule of thumb on how to frame the shot to make the composition look more interesting: the camera frame should be dived by two equally spaced horizontal and vertical lines and important elements of the scene should be placed along those lines or their intersections.

Storyboard UI supports this by providing a Rule-of-Thirds overlay in the camera mode:

sbui thirds-rule no guides
sbui thirds-rule guides
 

The overlay can be toggled on and off with a hotkey.

Moving Cameras

In addition to a visually pleasing frame composition and varying between appropriate static shot types, scenes can also benefit from carefully defined camera movements. In the Witcher 3 scene system this works by defining two or more key frames (meaning a specific camera position and its direction) and letting the scene system smoothly interpolate the camera position and direction (and other parameters) automatically between the key frames.

Storyboard UI does not provide any direct support or preview for camera setting interpolation. Instead it is required to define separate shots with static cameras in SBUI and afterwards connect these cameras in the scene yml definition by setting up the first camera as a start cam and the second one as the end cam. In the encoder this is called camera blends and an example definition may look like this:

 

cam blend example
 

In this case the camera blend starts in the element shot_1 and ends in the element shot_3 because it spans multiple voicelines which have to be separate elements. But that is not necessary: it’s possible to define multiple (but not overlapping!) camera blends in one element, e.g. a long pause element without any voicelines. However camera blends cannot span dialog sections boundaries and cannot be defined in choice sections.

Depending on the defined duration (that is the time between the key frames) the interpolation between the camera parameters will be slower or faster. Most of the time only a subtle, very slow movement will improve the scene while fast changes will most likely draw too much attention from viewers and distract from the actual content.

Be aware that smoothly also means that interpolation between more than two key frames (which can be inserted between start and end) will be a curve that may overshot the key frame significantly if not carefully defined. Some experimentation will be necessary.

One simple encode-able example can be found in the docs.scenes/test.examples directory (test_storyboard_cam.blend.yml) which also explains the parameters. The “bigbunny.example” in docs.scenes is an example of a camera flyby with multiple keyframes.

Depth of Field and Focus

An advanced cinematography technique is to adjust the depth of field (DOF). It’s the area in front of the camera that appears sharp and can be used to direct the attention of viewers to certain elements and to blur out unimportant ones, e.g. crowds of npcs surrounding the main actor(s) of the scene.

Here is are three example shots of the same scene with different DOF settings to visualize the effect (click on the image to see the video with rapid switches between the dof settings):

 

dof screens
 

The camera definitions logged from Storyboard UI contain a default DOF setting:

 

dof definition default
 

But in most cases it should be adjusted (or at least tuned down) as it makes the encoded scenes “blurry” if the actors are not in the focused sweet spot area.

Both (blur and focus) settings are specified by two distance values (near, far) which (as a simplified analogy) define the range from the camera position that the blur and focus is centered around. So to put some actor(s) (or the interesting scene element) into focus and blur out the near and far parts of the scene the settings should be spaced roughly like this:

[ ... blur near ... [focus near ... actor(s) ... focus far] ... blur far ... ]

It’s possible to change the values individually in SBUI but most of the time it is easier to use the rough (!) automatic DOF centering (see hotkey help). It tries to adjust the settings to “somewhat adequate” values that put the selected actor into the sweet spot of the current shot.

 

dof definition auto
 

Though the settings can be tweaked individually afterwards, too.

Unfortunately due to technical limitations there is only limited support to preview the effect in SBUI. As a prerequisite for the preview to work at all no active layered environment definitions are allowed (for example no weather environment must be active). Even in this case the preview in SBUI works only partially and only the ‘far’ plane is blurred but not the ‘near’ area. However it is possible to deactivate layered environments with the console command envui_disable_envs() which is part of the envui package. But make sure the depth of field is not switched off in the games post-processing settings!

As a side note it’s also noteworthy that the DOF settings like all camera parameters will be interpolated in camera blends which can be used to create interesting visual effects.

Scene (Definition) Tuning

As briefly hinted in the dialog scene tutorial (see docs.scenes/tutorial) the ‘storyboard’ part of a radish scene definition is responsible for the actual presentation of a scene. Most of the tuning for the presentation has to be done in this part of the definition. But it’s important to understand where settings have to be added or changed, first.

For every section in the dialogscript part of the scene definition, a dedicated, equally named section in the storyboard part can be defined: it allows to attach a variety of different ‘scene events’ for specific “dialogline” or “pause” elements of the dialogscript section.

All dialogscript elements can be referenced in the storyboard part either positional (by their position in the respective section, e.g. PAUSE_1 for the first pause in the section or GERALT_2 for the second textline of the actor named GERALT) or by the name of an additional, prepended named HINT/CUE definition (see example below). Most of the time (e.g. in all Storyboard UI dumped definitions) the named CUE references should be preferred as they don’t rely on the order of elements and also improve the readability of the storyboard.

 

storyboard positional references

 

The previously described camera blends are an example for one type of scene events related to cameras (but there are many other types).

Every scene event type has a different purpose and most have different settings that can be adjusted.

See the short example definitions in the ‘docs.scenes/test.examples’ directory as a reference for of all supported scene events and their respective settings.

Of particular interest are the animation events (normal, additive and mimic animations): although adding custom animations is not supported by the radish modding tools, Witcher 3 has a lot of reusable animations that can be further tweaked by some settings to customize them for the scene. For example animations can be ‘clipped’ to just playback a specific part, have a smooth transition between previous and/or next or idle animation (‘blend’), the intensity can be adjusted by a ‘weight’ parameter and animations can be slowed down or played faster with a ‘stretch’ parameter. In addition a sequence of multiple animations can be setup with different starting positions for any dialogline or pause element in a scene section.

For technical reasons the Storyboard UI in-game mod cannot provide a preview for any of these settings and it also cannot assign multiple animations per shot or setup their starting point within a shot. As a consequence tuning these parameters is only possible in the yml definition and the results have to be checked by encoding and reviewing the scene in the game. Nevertheless SBUI is useful to lay out the rough scene, setup some variations of a shot (e.g. different cameras or animations) or simply to search and preselect animations or mimics.

Debug Timeline

To ease with the tuning of a scene and to get a better “overview” of the storyboard parts of a scene definition the radish scene encoder automatically “renders” a “debug timeline” which visualizes the positions, duration and sequence of animations and other scene events (e.g. camera changes) into a text file.

It looks like this:

 

ascii timeline

 

At this point it is *highly* recommended to use sublime as an editor and to install the radish modding tools sublime support package (from the download section). The package contains auto completions for a couple of scene events, completions for animation names and a coloring scheme for scene timelines. This improves the overall usefulness of the debug timeline, especially with more complex scenes containing many actors and props (note the folded actor timelines in the screenshot):

 

ascii timeline colored

 

With some practice many changes can be prepared using the debug timeline without the need to check every change in the game (see this short example video). Nevertheless make sure you setup the ‘Utility Scene Autostarter’ from trial 4 correctly to playback encoded scenes in the game as fast as possible (when you have to).

Adding/Adjusting Animations To A Scene

Once a scene definition is changed by manual tweaking it becomes impractical to use Storyboard UI to adjust and dump the *full* scene, again. One possible way to add or exchange some animation in an already defined scene requires creating a new (or adjusting an existing) shot in a SBUI scene, logging the definition and afterwards manually merging of multiple parts from the dump into the scene definition yml. These are namely the part of the dumped repository, part of the production assets and the actual animation usage in the storyboard section referencing the new animation in question:

 

def adding anim

 

However an easier way is just to use inline animations in the definition and thus make the manual transfer of the repository and production parts unnecessary. The above example will be considerably reduced to just adding/changing the actual animation and attaching it to the actor directly in the storyboard:

 

def adding inline-anim

 

The only required information from the SBUI dump is the repository name of the animation (in this case "anim_6970_high_standing_determined_calm_enter_frontal").

Poses and Animation Blending

Every actor has always a “pose” defined. Poses are basically the “idle” animations and are active all the time when no specific animation is played, e.g. in scene choice sections. For these reasons pose animations are always looped.

If a specific animation is played and stops the pose idle animation takes over. Most of the usable animations intended for dialogs are named in a way which indicates a compatible pose idle animation (e.g. geralt_high_standing_determined… is compatible with all pose idle animations defined as high, standing and determined). But even in a compatible transition there may be a visible, sudden change of the actors pose at the end of the animation. To reduce this animation jerk, for every anim a blend-in and also a blend-out can be defined, like this:

 

def anim blending

 

These settings are defined as the duration (in seconds) the animation will blend either from the previous (specific or idle) anim to the next anim (again, either the next or the idle one). However making these blends too long will make it look unnatural - so some experimentation will be required. Usually a good starting point is something between 0.3 - 0.7 seconds.

Since this blending is applied to the *played* part of the animation the clipping has to be considered, too:

 

anim blending schema

 

It’s also possible to slowdown or speedup animations (‘stretch’ setting) and to change the ‘weight’ of the animation (which is basically the intensity of the overlay of the animation - try out a couple of different weights to get a feeling for the effect).

Also be aware that the ‘stretch’ parameter is applied on the played range of the animation (that is: *after* the clipping). As seen in the above video it’s much easier to get a handle on the resulting duration by using the debug timeline visualization.

Looped Animations

Some animations like generic walking, running or riding are only very short and have only one or at most only a couple of cycles - then they just stop when used as actor.anim scene events. In addition the actor is moving at the same spot and does not advance.

There are basically two options to loop these animations:

  1. setup a custom pose with the animation as idle animation in SBUI (select the animation intended to be looped in the pose animation list and add an actor.pose event in the appropriate element in the yml definition)
  2. setup a sequence of the same animation with actor.anim events as many times as required

The first solution is easier to setup but does not work reliably if the animation has to be synced with another actors animation (e.g. rider and horse): pose animations seem to start at slightly random positions and this may result in some clipping (e.g. between rider and horse). The second one syncs reliably but the subsequent starting positions have to be set manually which may require more fine-tuning (or some calculations for correct positioning).

Additionally, in both cases the actor has to be moved accordingly by properly defined placement interpolation events.


Placement Interpolation

Most of the actor animations with some movement also move the involved actor while playing. But at the end of the animation the actors position is always reset to the starting position. To reposition an actor (at any time) to a specific position the ‘actor.placement’ scene events can be used.

However for looped animation like walking cycles (or any other animation which do not have actor movement encoded) it is required to manually add continuous, smooth placement updates. In the yml definition this can be done with ‘placement interpolation’ scene events, similar to camera interpolation (see ‘test_storyboard_actor.placement.interpolation.yml’):

def placement blend

Additional key frames can be set between the start and end to define a more specific path. In addition to the position also the rotation will be interpolated. The last parameter defines the ‘ease-in’ and ‘ease-out’ just like for camera blends.

The easiest way to define an interpolation is to create two dedicated shots in SBUI (one for start the other for the end position), place the actor in both shots, log the definition and either change the placement events into interpolation events or just use the coordinates to write the above scene event sequence manually, e.g. attached only to one element (as above). However an appropriate amount of time needs to be between the events or you’ll just get an unnatural sliding - so expect some iterations for fine-tuning.

One thing to keep in mind is that actor movement (from animations) in scenes does not respect any collisions, that includes the ground: actors will clip with any terrain bump or float over terrain dents.

If necessary this can be manually fixed with placement interpolation (Z-Axis) as well. Although most of the time it’s just easier to frame the shot and hide the clipping.

Also, any scene props can be positioned and/or moved along a placement interpolation path with dedicated scene events (see test.examples).


Mimic Animations

Normal animations do not contain any facial movements. There is a dedicated set of animations for this and they are attached with ‘actor.anim.mimic’ scene events. The available mimic animations can be previewed in Storyboard UI.

Unfortunately SBUI does not support assigning a mimic animation and a voiceline at the same time in the same shot. However this constraint does not apply to yml scene definitions.

The settings for ‘actor.anim.mimic’ events are basically the same as for “normal” animation events. And just like normal animations events an ‘actor.anim.mimic’ can be defined as an “inline” animation and directly attached to an actor:

def inline-mimic

Take special care with the weight setting for mimic animations, e.g. if your actors are smiling like idiots you should tune down the weight for the mimic. Here are four examples with anim.mimic weights of 1.0, 0.66, 0.33 and 0.0:

scene mimic 000 small
scene mimic 033 small
scene mimic 066 small
scene mimic 100 small

As a good rule of thumb you should not go overboard with the mimic weight:

the more subtle mimics are the more convincing and natural the result will look. Sometimes even one mildly raised eye brow will be enough to emphasize a reaction.


Look-ats

The encoder automatically generates ‘actor.lookat’ events for scenes with multiple actors and makes actors look at each other. However it’s possible to override these autogenerated scene events by specifying a custom looked-at actor or static point, the speed of change or the involved rotating bodyparts (eyes, head, upper body). It’s also possible to “disable” a specific look-at by defining a ‘none’ target because look-at events are on-top modifiers for animations and some animations may already contain gaze changes, too.

SBUI provides a hotkey to cycle between all actors as look-at target and also the definition of static look-at points (make sure to adjust the distance or the actor will be squint-eyed) but the advanced settings, like speed and the involved bodyparts can only be setup in the definition (see above debug timeline video or the test.examples).

Cutscene Anchoring

Gameplay Scenes

Custom Voicelines

The Big Picture In A Nutshell

Phoneme Extractor GUI

Phoneme Timing Tuning

Warnings and Errors In Steps 2 and 5

Testing Custom Audio In Storyboard UI

Adding Other Languages

Generating Lipsync Animations Without Audio

Tips Scene Definitions

Tips Storyboard UI (SBUI)

Tips Phoneme Extraction/Tuning

Tips Speechfile Packing