Difference between revisions of "Sequence Music Engine in KCD"

From Nexus Mods Wiki
Jump to: navigation, search
m (Introduction)
m
 
(18 intermediate revisions by 8 users not shown)
Line 1: Line 1:
 +
 
== Introduction ==
 
== Introduction ==
 +
 +
''To learn about modding of the SFX, please refer to [[Sound_modding_in_KCD|Sound modding in KCD]]''
  
 
''Portions of this text are taken from a journal article (Sporka and Valta 2017) co-written by the original poster of this article.''
 
''Portions of this text are taken from a journal article (Sporka and Valta 2017) co-written by the original poster of this article.''
Line 5: Line 8:
 
''For any inquiries, please tweet at @adam_sporka.''
 
''For any inquiries, please tweet at @adam_sporka.''
  
''DISCLAIMER: Any information on this page is supposed to be used at your own risk :-)''
+
''DISCLAIMER: Any information on this page is supposed to be used at your own risk :-)''
  
 
Sequence Music Engine is an adaptive music engine. It is a system for playback of pre-existing music material. It is a C++ library with available bindings to CryEngine (via FMOD Studio) and Unity. SQC runs within a host computer game process. The game communicates with SQC via control variables. (Simple set of key–value string pairs.) The current implementation uses the sampling frequency of 48 kHz and a 2-channel output. SQC is intended to work in a sample-pulling pipeline where an audio engine is requesting buffers of audio to play ("getbuffer operation").
 
Sequence Music Engine is an adaptive music engine. It is a system for playback of pre-existing music material. It is a C++ library with available bindings to CryEngine (via FMOD Studio) and Unity. SQC runs within a host computer game process. The game communicates with SQC via control variables. (Simple set of key–value string pairs.) The current implementation uses the sampling frequency of 48 kHz and a 2-channel output. SQC is intended to work in a sample-pulling pipeline where an audio engine is requesting buffers of audio to play ("getbuffer operation").
Line 17: Line 20:
 
It's relatively simple for linear songs, such as menu music. It is way more complex for non-linear pieces, such as combat music, battle music, or even simple exploration pieces with seamless transitions. I will elaborate on this topic later on.
 
It's relatively simple for linear songs, such as menu music. It is way more complex for non-linear pieces, such as combat music, battle music, or even simple exploration pieces with seamless transitions. I will elaborate on this topic later on.
  
I think that the best way to start is to take a look at the ``1980s Synthwave KCD Music in the Menu`` mod (https://www.nexusmods.com/kingdomcomedeliverance/mods/199) and do a diff between the original main_layer.widsh and the modded one. With a bit of help from [https://mh-nexus.de/en/hxd], it is possible assemble the .sqcb file by hand from a raw .ogg file.
+
I think that the best way to start is to take a look at the ''1980s Synthwave KCD Music in the Menu'' mod ([https://www.nexusmods.com/kingdomcomedeliverance/mods/199 https://www.nexusmods.com/kingdomcomedeliverance/mods/199]) and do a diff between the original main_layer.widsh and the modded one. With a bit of hacking and slashing using [https://mh-nexus.de/en/hxd [1]], it is possible assemble the .sqcb file by hand from a raw .ogg file.
 +
 
 +
 
 +
 
 +
 
 +
 
 +
 
 +
 
 +
 
 +
 
 +
 
  
 
== Technical Structure of the Soundtrack ==
 
== Technical Structure of the Soundtrack ==
Line 24: Line 37:
  
 
There are three types of patterns, differing in use:
 
There are three types of patterns, differing in use:
* '''Main pattern''' -- representation of a linear part of the music. The main patterns are the pieces of music material which play most of the time.
 
* '''Branch''' -- representation of a premature, yet scored ending of the main pattern, typically used when the scene should no longer play, and its expedited termination is required.
 
* '''Intro''' -- representation of a scored beginning of a scene. Branches and Intros are used in the mechanism called seamless transition.
 
* '''Cinel''' -- music material which glues two scenes during a transition via cinels.
 
  
Every pattern has a known nominal length, corresponding to the length of the musical
+
*'''Main pattern''' -- representation of a linear part of the music. The main patterns are the pieces of music material which play most of the time.
feature contained in it. For example, a 4-bar pattern at 120 BPM in a 4/4 rhythm has a
+
*'''Branch''' -- representation of a premature, yet scored ending of the main pattern, typically used when the scene should no longer play, and its expedited termination is required.
nominal length of 8 seconds. Its technical duration might be slightly longer in order to enable the playback of overhanging tones or noises. This implements the horizontal re-sequencing technique.
+
*'''Intro''' -- representation of a scored beginning of a scene. Branches and Intros are used in the mechanism called seamless transition.
 +
*'''Cinel''' -- music [https://heymods.net/es/spotify.html Spotify Premium] and [https://modfyp.com/es/spotify Spotify Premium APK] and [https://apkclasico.com/spotify-apk.html Spotify APK MOD] and [https://coimobile.com/spotify-apk/ Spotify APK] material which glues two scenes during a transition via cinels.
 +
 
 +
Every pattern has a known nominal length, corresponding to the length of the musical feature contained in it. For example, a 4-bar pattern at 120 BPM in a 4/4 rhythm has a nominal length of 8 seconds. Its technical duration might be slightly longer in order to enable the playback of overhanging tones or noises. This implements the horizontal re-sequencing technique.
  
Patterns may have multiple flavors, different versions of the same music material
+
Patterns may have multiple flavors, different versions of the same music material between which the engine cross-fades, based on their individual suitability. This enables the paradigm of adaptive music commonly known as the vertical reorchestration.
between which the engine cross-fades, based on their individual suitability. This enables
 
the paradigm of adaptive music commonly known as the vertical reorchestration.
 
  
 
== SQCB Files ==
 
== SQCB Files ==
Line 43: Line 53:
 
This is the structure of the file:
 
This is the structure of the file:
  
<nowiki> Offset      Contents            Comments
+
&nbsp;
0          L"SQCB"            Fixed header prefix; 16-bit characters
 
8          L"1.00"            Version number; 16-bit characters
 
16          <32-bit LE int>    Number of files in the bank (N)
 
20          Bank items desc
 
            -- File name        A null-terminated 16-bit string
 
            -- <32-bit LE int>  Offset of the file in the bank
 
            -- <32-bit LE int>  Length of the file in the bank
 
?                              Binary data of the files, one after another
 
?          L"RDBP"            End of file suffix</nowiki>
 
 
 
=== MINI FAQ 1 ===
 
  
* Q: Is there a public packer/unpacker available?
+
&nbsp;
* A: Not really at this point, sorry, working on it :-)
 
  
* Q: Why are you using .ogg files in Sequence Music Engine
+
&nbsp;
* A: Ogg is a free and open-sourced format
+
<nowiki> Offset Contents Comments 0 L"SQCB" Fixed header prefix; 16-bit characters 8 L"1.00" Version number; 16-bit characters 16 <32-bit LE int> Number of files in the bank (N) 20 Bank items desc -- File name A null-terminated 16-bit string -- <32-bit LE int> Offset of the file in the bank -- <32-bit LE int> Length of the file in the bank ? Binary data of the files, one after another ? L"RDBP" End of file suffix</nowiki>
  
* Q: Why "RDBP"?
+
=== MINI FAQ 1 ===
* A: Because Rainbow Dash is Best Pony
 
  
== .widsh File ==
+
*Q: Is there a public packer/unpacker available?
 +
*A: Not really at this point, sorry, working on it&nbsp;:-)
  
The behavior of the music is described in main_layer.widsh, which is the music script file. Here is a commented excerpt. The file shipped with the game is 3,700 lines long. Apologies for its cryptic look but despite it being a text file, it's generated / compiled from multiple sources of data.
+
*Q: Why are you using .ogg files in Sequence Music Engine
 +
*A: Ogg is a free and open-sourced format
  
<nowiki> SQC MUSIC ENGINE WIDSH FILE VERSION 1.18
+
*Q: Why "RDBP"?
: Any text can be here. I use it for credit information.
+
*A: Because Rainbow Dash is Best Pony
: It won’t affect your music in any way.
 
  
SETUP
+
== main_layer.widsh File ==
// Line comments have to start with '//'
 
// This is run on start-up. Used for variable initialization.
 
  
// A simple assignment looks like this:
+
The behavior of the music is described in main_layer.widsh, which is the music script file. Here is a commented excerpt. The file shipped with the game is 3,700 lines long. Apologies for its cryptic look. Despite it being a text file, it's generated / compiled from multiple sources of data and was never meant to be looked at, let alone edited, by a human being again.
[%string_variable $string_value !]
 
// In a C-like language this line would read as
 
// string_variable = "string_value";
 
  
[%float_variable #3.14 !]
+
&nbsp;
// In a C-like language this line would read as
 
// float_variable = 3.14;
 
  
  // This calculates the circumference of a circle with 10 units radius.
+
  &nbsp;
// Polish Reverse Notation. (Because CD Projekt rulez!)
 
[%C [[#2 %float_variable *] #10 *] !]
 
END
 
  
UPDATE
+
&nbsp;
// This is called before each getbuffer operation. It depends on the host
+
<nowiki> SQC MUSIC ENGINE WIDSH FILE VERSION 1.18 : Any text can be here. I use it for credit information. : It won’t affect your music in any way. SETUP // Line comments have to start with '//' // This is run on start-up. Used for variable initialization. // A simple assignment looks like this: [%string_variable $string_value !] // In a C-like language this line would read as // string_variable = "string_value"; [%float_variable #3.14 !] // In a C-like language this line would read as // float_variable = 3.14; // This calculates the circumference of a circle with 10 units radius. // Polish Reverse Notation. (Because CD Projekt rulez!) [%C [[#2 %float_variable *] #10 *] !] END UPDATE // This is called before each getbuffer operation. It depends on the host // application how often this gets called. FMOD Studio on PC typically // calls this 5 times during 2 seconds. // A music logic is expressed here. END SCENE GUI/MM-BETA-THEME // This is an example of a SCENE block. It provides the metadata for // a single scene. The order of the lines is somewhat loose but not that // loose, so I recommend to keep it the way it is. N MAIN_MENU // N: The section in the SQC Player GUI (which is not publically available, sorry) L [$MM-BETA-THEME] // L: The label shown in the SQC Player GUI E [[%menu $on =] [[%closing_credits $on =] [%closing_credits_type $game ~] &] |] // E: The selection suitability expression. TL;DR: The expression must evaluate as true // if this scene is expected to play and is desired to be chosen. P [#16] // P: Selection priority T [#48000 #600 *] // T: Timeout in samples. The example above corresponds to 600 seconds = 10 minutes. // The scene is guaranteed not to play for 10 minutes after the completion of its // playback. S 1 FADE_UNSUITABLE KEEPQUIET @1 1 // S: Scene mode. // FADE_UNSUITABLE -- As soon as E is false, the playback fades out // KEEPQUIET @1 -- The silence after completion of the playback will last 1 second M 1 0 @132 @133 BRANCHES 0 SYNCPOINTS 65 96000 +96000 +96000 +96000 +96000 +96000 +96000 +96000 +96000 +96000 +96000 +96000 +96000 +96000 +96000 +96000 +96000 +96000 +96000 +96000 +96000 +96000 +96000 +96000 +96000 +96000 +96000 +96000 +96000 +96000 +96000 +96000 +96000 +96000 +96000 +96000 +96000 +96000 +96000 +96000 +96000 +96000 +96000 +96000 +96000 +96000 +96000 +96000 +96000 +96000 +96000 +96000 +96000 +96000 +96000 +96000 +96000 +96000 +96000 +96000 +96000 +96000 +96000 +96000 +96000 NOTIFICATIONS 0 // M <number> 0 <nominal_length_in_samples_or_@seconds> <tech_length> // BRANCHES <number_of_branches> // SYNCPOINTS <number_of_syncpoints> // NOTIFICATIONS <number_of_notifications,_usually_0> END SCENE EXPLORATION/AFLD4 // This is an example of an in-game non-linear scene. N EXPL_FBK E [[%#countryside $on =] [%#quiet $off =] [%#dialog $off =] ^] e [[%#countryside $on =] [%#quiet $off =] ^] // e: The playback suitability expression. As long as this is true, this scene can // play. If not present, the default is true. L [$OPEN_LANDSCAPE_(AFLD4)] S 1 SEAMLESS 1 // S: Scene mode. // SEAMLESS -- the seamless transitions mode E [[%#countryside $on =] [%#quiet $off =] ^] // E: Suitability expression applied for the next patterns. M 1 0 4799052 4943052 BRANCHES 9 21 432000 22 864000 23 1296000 24 1872000 25 2304000 26 2880000 27 3456000 28 3935053 29 4367053 SYNCPOINTS 122 36000 72000 108000 144000 180000 216000 252000 288000 324000 360000 396000 468000 504000 540000 576000 612000 648000 684000 720000 756000 792000 828000 900000 936000 972000 1008000 1044000 1080000 1116000 1152000 1188000 1224000 1260000 1332000 1368000 1404000 1440000 1476000 1512000 1548000 1584000 1620000 1656000 1692000 1728000 1764000 1800000 1836000 1908000 1944000 1980000 2016000 2052000 2088000 2124000 2160000 2196000 2232000 2268000 2340000 2376000 2412000 2448000 2484000 2520000 2556000 2592000 2628000 2664000 2700000 2736000 2772000 2808000 2844000 2916000 2952000 2988000 3024000 3060000 3096000 3132000 3168000 3204000 3240000 3276000 3312000 3348000 3384000 3420000 3492000 3528000 3564000 3600000 3636000 3672000 3708000 3744000 3781732 3824620 3875640 3971052 4007052 4043052 4079052 4115052 4151052 4187052 4223052 4259052 4295052 4331052 4403052 4439052 4475052 4511052 4547052 4583052 4619052 4655052 4691052 4727052 4763052 NOTIFICATIONS 0 // A main pattern 4799052 samples long with 9 branches to endings nr. 21, 22, 23, etc., // with 122 synchpoints placed at 36000, 72000, ... samples starting from 0. // Its number is 1. The corresponding .ogg file in the AFLD4 is called AFLD4-01.ogg E [#1.0] I 10 0 288000 432000 0 SYNCPOINTS 7 36000 72000 108000 144000 180000 216000 252000 I 11 0 288000 432000 1 SYNCPOINTS 7 36000 72000 108000 144000 180000 216000 252000 I 12 0 288000 432000 2 SYNCPOINTS 7 36000 72000 108000 144000 180000 216000 252000 I 13 0 288000 432000 3 SYNCPOINTS 7 36000 72000 108000 144000 180000 216000 252000 // I: An intro of a given color. 0 = white, 1 = red, 2 = green, 3 = blue. // For details please read the paper [1] linked below in References. // Intro 10 would be AFLD4-10.ogg B 21 0 288000 432000 BPOINT 144000 TO_ALEPH 2 SYNCPOINTS 7 36000 72000 108000 144000 180000 216000 252000 NOTIFICATIONS 0 // B: A branch, contained in AFLD4-21.ogg B 22 0 432000 576000 BPOINT 144000 TO_ALEPH 2 SYNCPOINTS 11 36000 72000 108000 144000 180000 216000 252000 288000 324000 360000 396000 NOTIFICATIONS 0 ... END</nowiki>
// application how often this gets called. FMOD Studio on PC typically
 
// calls this 5 times during 2 seconds.
 
// A music logic is expressed here.
 
END
 
 
 
SCENE GUI/MM-BETA-THEME
 
// This is an example of a SCENE block. It provides the metadata for
 
// a single scene. The order of the lines is somewhat loose but not that
 
// loose, so I recommend to keep it the way it is.
 
N MAIN_MENU
 
// N: The section in the SQC Player GUI (which is not publically available, sorry)
 
L [$MM-BETA-THEME]
 
// L: The label shown in the SQC Player GUI
 
E [[%menu $on =] [[%closing_credits $on =] [%closing_credits_type $game ~] &] |]
 
// E: The selection suitability expression. TL;DR: The expression must evaluate as true
 
// if this scene is expected to play and is desired to be chosen.
 
P [#16]
 
// P: Selection priority
 
T [#48000 #600 *]
 
// T: Timeout in samples. The example above corresponds to 600 seconds = 10 minutes.
 
// The scene is guaranteed not to play for 10 minutes after the completion of its
 
// playback.
 
S 1 FADE_UNSUITABLE KEEPQUIET @1 1
 
// S: Scene mode.
 
//   FADE_UNSUITABLE -- As soon as E is false, the playback fades out
 
//   KEEPQUIET @1 -- The silence after completion of the playback will last 1 second
 
M 1 0 @132 @133 BRANCHES 0
 
    SYNCPOINTS 65
 
        96000 +96000 +96000 +96000 +96000   +96000 +96000 +96000 +96000 +96000
 
        +96000 +96000 +96000 +96000 +96000   +96000 +96000 +96000 +96000 +96000
 
        +96000 +96000 +96000 +96000 +96000   +96000 +96000 +96000 +96000 +96000
 
        +96000 +96000 +96000 +96000 +96000   +96000 +96000 +96000 +96000 +96000
 
        +96000 +96000 +96000 +96000 +96000   +96000 +96000 +96000 +96000 +96000
 
        +96000 +96000 +96000 +96000 +96000   +96000 +96000 +96000 +96000 +96000
 
        +96000 +96000 +96000 +96000 +96000
 
    NOTIFICATIONS 0
 
// M <number> 0 <nominal_length_in_samples_or_@seconds> <tech_length>
 
//   BRANCHES <number_of_branches>
 
//   SYNCPOINTS <number_of_syncpoints>
 
//   NOTIFICATIONS <number_of_notifications,_usually_0>
 
END
 
 
 
SCENE EXPLORATION/AFLD4
 
// This is an example of an in-game non-linear scene.
 
N EXPL_FBK
 
E [[%#countryside $on =] [%#quiet $off =] [%#dialog $off =] ^]
 
e [[%#countryside $on =] [%#quiet $off =] ^]
 
// e: The playback suitability expression. As long as this is true, this scene can
 
// play. If not present, the default is true.
 
L [$OPEN_LANDSCAPE_(AFLD4)]
 
S 1 SEAMLESS 1
 
// S: Scene mode.
 
//   SEAMLESS -- the seamless transitions mode
 
E [[%#countryside $on =] [%#quiet $off =] ^]
 
// E: Suitability expression applied for the next patterns.
 
M 1 0 4799052 4943052
 
    BRANCHES 9
 
        21 432000 22 864000 23 1296000 24 1872000 25 2304000 26 2880000 27 3456000
 
        28 3935053 29 4367053
 
    SYNCPOINTS 122
 
        36000 72000 108000 144000 180000 216000 252000 288000 324000 360000
 
        396000 468000 504000 540000 576000 612000 648000 684000 720000 756000
 
        792000 828000 900000 936000 972000 1008000 1044000 1080000 1116000 1152000
 
        1188000 1224000 1260000 1332000 1368000 1404000 1440000 1476000 1512000 1548000
 
        1584000 1620000 1656000 1692000 1728000 1764000 1800000 1836000 1908000 1944000
 
        1980000 2016000 2052000 2088000 2124000 2160000 2196000 2232000 2268000 2340000
 
        2376000 2412000 2448000 2484000 2520000 2556000 2592000 2628000 2664000 2700000
 
        2736000 2772000 2808000 2844000 2916000 2952000 2988000 3024000 3060000 3096000
 
        3132000 3168000 3204000 3240000 3276000 3312000 3348000 3384000 3420000 3492000
 
        3528000 3564000 3600000 3636000 3672000 3708000 3744000 3781732 3824620 3875640
 
        3971052 4007052 4043052 4079052 4115052 4151052 4187052 4223052 4259052 4295052
 
        4331052 4403052 4439052 4475052 4511052 4547052 4583052 4619052 4655052 4691052
 
        4727052 4763052
 
    NOTIFICATIONS 0
 
// A main pattern 4799052 samples long with 9 branches to endings nr. 21, 22, 23, etc.,
 
// with 122 synchpoints placed at 36000, 72000, ... samples starting from 0.
 
// Its number is 1. The corresponding .ogg file in the AFLD4 is called AFLD4-01.ogg
 
E [#1.0]
 
I 10 0 288000 432000 0 SYNCPOINTS 7 36000 72000 108000 144000 180000 216000 252000
 
I 11 0 288000 432000 1 SYNCPOINTS 7 36000 72000 108000 144000 180000 216000 252000
 
I 12 0 288000 432000 2 SYNCPOINTS 7 36000 72000 108000 144000 180000 216000 252000
 
I 13 0 288000 432000 3 SYNCPOINTS 7 36000 72000 108000 144000 180000 216000 252000
 
// I: An intro of a given color. 0 = white, 1 = red, 2 = green, 3 = blue.
 
// For details please read the paper [1] linked below in References.
 
// Intro 10 would be AFLD4-10.ogg
 
B 21 0 288000 432000  
 
    BPOINT 144000 TO_ALEPH 2  
 
    SYNCPOINTS 7 36000 72000 108000 144000 180000 216000 252000  
 
    NOTIFICATIONS 0
 
// B: A branch, contained in AFLD4-21.ogg
 
B 22 0 432000 576000
 
    BPOINT 144000 TO_ALEPH 2
 
    SYNCPOINTS 11 36000 72000 108000 144000 180000 216000 252000 288000 324000
 
        360000 396000
 
    NOTIFICATIONS 0
 
...
 
END</nowiki>
 
  
 
=== MINI FAQ 2 ===
 
=== MINI FAQ 2 ===
  
* Q: Does the file need to be called "main_layer.widsh"?
+
*Q: Does the file need to be called "main_layer.widsh"?  
* A: Unfortunately, yes, for now. At some point I'll add a CVar allowing you to specify a different filename.
+
*A: Unfortunately, yes, for now. At some point I'll add a CVar allowing you to specify a different filename.  
 
 
* Q: What happens when I change the file while the game is running?
 
* A: You need to restart the game in order to hear the changes.
 
  
* Q: What happens when I replace the .sqcb file while the game is running?
+
*Q: What happens when I change the file while the game is running?  
* A: Good question, TBH, I didn't need to try that. In theory, next time the scene is loaded for playback, its new contents should be played back. However, if you need to provide a different timing information, you need to specify that in main_layer.widsh and that one won't be reloaded until the end of the KingdomCome.exe process.
+
*A: You need to restart the game in order to hear the changes.  
  
* Q: Is there a better way how to debug the music than restarting the game every single time?
+
*Q: What happens when I replace the .sqcb file while the game is running?  
* A: We have an in-house playback tool. It hasn't been released to the public yet.
+
*A: Good question, TBH, I didn't need to try that. In theory, next time the scene is loaded for playback, its new contents should be played back. However, if you need to provide a different timing information, you need to specify that in main_layer.widsh and that one won't be reloaded until the end of the KingdomCome.exe process.  
  
* Q: Why is it so god-damn complicated?
+
*Q: Is there a better way how to debug the music than restarting the game every single time?  
* A: These are the internals of the engine. In an ideal world, I'd provide a neat GUI and a WYSIWYG (WYSIWYH?) editor. I did develop a GUI interface for the soundtrack development I'd rather sit on it until it's ready for a public release. Thanks for your understanding!
+
*A: We have an in-house playback tool. It hasn't been released to the public yet.  
  
* Q: Did you really spend all that time typing all the numbers?
+
*Q: Why is it so god-damn complicated?  
* A: Hell no! I wrote a tool which generated those too. Looks intense but once we got used to our workflow, adding more music was relatively easy.
+
*A: These are the internals of the engine. In an ideal world, I'd provide a neat GUI and a WYSIWYG (WYSIWYH?) editor. I did develop a GUI interface for the soundtrack development I'd rather sit on it until it's ready for a public release. Thanks for your understanding!
  
* Q: What does WIDSH stands for?
+
*Q: Did you really spend all that time typing all the numbers?  
* A: Well-Informed Data Structure Header. Actually, that's not it. But it's not at all important. I was serious about Rainbow Dash though!
+
*A: Hell no! I wrote a tool which generated those too. Looks intense but once we got used to our workflow, adding more music was relatively easy.  
  
* Q: Are you a brony or something?
+
*Q: What does WIDSH stands for?  
* A: Yes.
+
*A: Well-Informed Data Structure Header. Actually, that's not it. But it's not at all important. I was serious about Rainbow Dash though!
  
 
== Reference ==
 
== Reference ==
  
* Sporka A, Valta J (2017) Design and implementation of a non-linear symphonic soundtrack of a video game. New Review of Hypermedia and Multimedia. Volume 23, 2017, Issue 4. Taylor&Francis. https://www.tandfonline.com/doi/abs/10.1080/13614568.2017.1416682 Please tweet at @adam_sporka if you wish to obtain a copy of the paper.
+
*Sporka A, Valta J (2017) Design and implementation of a non-linear symphonic soundtrack of a video game. New Review of Hypermedia and Multimedia. Volume 23, 2017, Issue 4. Taylor&Francis. [https://www.tandfonline.com/doi/abs/10.1080/13614568.2017.1416682 https://www.tandfonline.com/doi/abs/10.1080/13614568.2017.1416682] Please tweet at @adam_sporka if you wish to obtain a copy of the paper.  
  
[[Category:Kingdom_Come_Deliverance]]
+
[[Category:Kingdom Come Deliverance]] [[Category:Mod Creation]] [[Category:Documentation]]

Latest revision as of 03:21, 7 March 2024

Introduction

To learn about modding of the SFX, please refer to Sound modding in KCD

Portions of this text are taken from a journal article (Sporka and Valta 2017) co-written by the original poster of this article.

For any inquiries, please tweet at @adam_sporka.

DISCLAIMER: Any information on this page is supposed to be used at your own risk :-)

Sequence Music Engine is an adaptive music engine. It is a system for playback of pre-existing music material. It is a C++ library with available bindings to CryEngine (via FMOD Studio) and Unity. SQC runs within a host computer game process. The game communicates with SQC via control variables. (Simple set of key–value string pairs.) The current implementation uses the sampling frequency of 48 kHz and a 2-channel output. SQC is intended to work in a sample-pulling pipeline where an audio engine is requesting buffers of audio to play ("getbuffer operation").

How to mod the KC:D music

TL;DR: Doable but not straightforward.

You need to (1) prepare a .sqcb file which will contain the actual audio data, and (2) modify main_layer.widsh to refer to the newly created .sqcb file.

It's relatively simple for linear songs, such as menu music. It is way more complex for non-linear pieces, such as combat music, battle music, or even simple exploration pieces with seamless transitions. I will elaborate on this topic later on.

I think that the best way to start is to take a look at the 1980s Synthwave KCD Music in the Menu mod (https://www.nexusmods.com/kingdomcomedeliverance/mods/199) and do a diff between the original main_layer.widsh and the modded one. With a bit of hacking and slashing using [1], it is possible assemble the .sqcb file by hand from a raw .ogg file.

 

 

 

 

 

Technical Structure of the Soundtrack

A SQC soundtrack is a set of scenes. A scene corresponds to a music piece which is able to function as a standalone music composition. Each scene contains one or more patterns. A pattern is a buffer of audio signal, containing a portion of music. Having multiple patterns in a single scene enables a non-linear playback of the scene’s music material by lining up the patterns in a suitable sequence.

There are three types of patterns, differing in use:

  • Main pattern -- representation of a linear part of the music. The main patterns are the pieces of music material which play most of the time.
  • Branch -- representation of a premature, yet scored ending of the main pattern, typically used when the scene should no longer play, and its expedited termination is required.
  • Intro -- representation of a scored beginning of a scene. Branches and Intros are used in the mechanism called seamless transition.
  • Cinel -- music Spotify Premium and Spotify Premium APK and Spotify APK MOD and Spotify APK material which glues two scenes during a transition via cinels.

Every pattern has a known nominal length, corresponding to the length of the musical feature contained in it. For example, a 4-bar pattern at 120 BPM in a 4/4 rhythm has a nominal length of 8 seconds. Its technical duration might be slightly longer in order to enable the playback of overhanging tones or noises. This implements the horizontal re-sequencing technique.

Patterns may have multiple flavors, different versions of the same music material between which the engine cross-fades, based on their individual suitability. This enables the paradigm of adaptive music commonly known as the vertical reorchestration.

SQCB Files

Each scene is stored in a separate SQCB file. Depending on its use, its contents could be one or many SQC patterns. A menu music typically contains just one pattern. A non-linear in-game track (the symphonic parts of the exploration or combat soundtrack) typically contains multiple patterns.

This is the structure of the file:

 

 

  Offset Contents Comments 0 L"SQCB" Fixed header prefix; 16-bit characters 8 L"1.00" Version number; 16-bit characters 16 <32-bit LE int> Number of files in the bank (N) 20 Bank items desc -- File name A null-terminated 16-bit string -- <32-bit LE int> Offset of the file in the bank -- <32-bit LE int> Length of the file in the bank ? Binary data of the files, one after another ? L"RDBP" End of file suffix

MINI FAQ 1

  • Q: Is there a public packer/unpacker available?
  • A: Not really at this point, sorry, working on it :-)
  • Q: Why are you using .ogg files in Sequence Music Engine
  • A: Ogg is a free and open-sourced format
  • Q: Why "RDBP"?
  • A: Because Rainbow Dash is Best Pony

main_layer.widsh File

The behavior of the music is described in main_layer.widsh, which is the music script file. Here is a commented excerpt. The file shipped with the game is 3,700 lines long. Apologies for its cryptic look. Despite it being a text file, it's generated / compiled from multiple sources of data and was never meant to be looked at, let alone edited, by a human being again.

 

 

  SQC MUSIC ENGINE WIDSH FILE VERSION 1.18 : Any text can be here. I use it for credit information. : It won’t affect your music in any way. SETUP // Line comments have to start with '//' // This is run on start-up. Used for variable initialization. // A simple assignment looks like this: [%string_variable $string_value !] // In a C-like language this line would read as // string_variable = "string_value"; [%float_variable #3.14 !] // In a C-like language this line would read as // float_variable = 3.14; // This calculates the circumference of a circle with 10 units radius. // Polish Reverse Notation. (Because CD Projekt rulez!) [%C [[#2 %float_variable *] #10 *] !] END UPDATE // This is called before each getbuffer operation. It depends on the host // application how often this gets called. FMOD Studio on PC typically // calls this 5 times during 2 seconds. // A music logic is expressed here. END SCENE GUI/MM-BETA-THEME // This is an example of a SCENE block. It provides the metadata for // a single scene. The order of the lines is somewhat loose but not that // loose, so I recommend to keep it the way it is. N MAIN_MENU // N: The section in the SQC Player GUI (which is not publically available, sorry) L [$MM-BETA-THEME] // L: The label shown in the SQC Player GUI E [[%menu $on =] [[%closing_credits $on =] [%closing_credits_type $game ~] &] |] // E: The selection suitability expression. TL;DR: The expression must evaluate as true // if this scene is expected to play and is desired to be chosen. P [#16] // P: Selection priority T [#48000 #600 *] // T: Timeout in samples. The example above corresponds to 600 seconds = 10 minutes. // The scene is guaranteed not to play for 10 minutes after the completion of its // playback. S 1 FADE_UNSUITABLE KEEPQUIET @1 1 // S: Scene mode. // FADE_UNSUITABLE -- As soon as E is false, the playback fades out // KEEPQUIET @1 -- The silence after completion of the playback will last 1 second M 1 0 @132 @133 BRANCHES 0 SYNCPOINTS 65 96000 +96000 +96000 +96000 +96000 +96000 +96000 +96000 +96000 +96000 +96000 +96000 +96000 +96000 +96000 +96000 +96000 +96000 +96000 +96000 +96000 +96000 +96000 +96000 +96000 +96000 +96000 +96000 +96000 +96000 +96000 +96000 +96000 +96000 +96000 +96000 +96000 +96000 +96000 +96000 +96000 +96000 +96000 +96000 +96000 +96000 +96000 +96000 +96000 +96000 +96000 +96000 +96000 +96000 +96000 +96000 +96000 +96000 +96000 +96000 +96000 +96000 +96000 +96000 +96000 NOTIFICATIONS 0 // M <number> 0 <nominal_length_in_samples_or_@seconds> <tech_length> // BRANCHES <number_of_branches> // SYNCPOINTS <number_of_syncpoints> // NOTIFICATIONS <number_of_notifications,_usually_0> END SCENE EXPLORATION/AFLD4 // This is an example of an in-game non-linear scene. N EXPL_FBK E [[%#countryside $on =] [%#quiet $off =] [%#dialog $off =] ^] e [[%#countryside $on =] [%#quiet $off =] ^] // e: The playback suitability expression. As long as this is true, this scene can // play. If not present, the default is true. L [$OPEN_LANDSCAPE_(AFLD4)] S 1 SEAMLESS 1 // S: Scene mode. // SEAMLESS -- the seamless transitions mode E [[%#countryside $on =] [%#quiet $off =] ^] // E: Suitability expression applied for the next patterns. M 1 0 4799052 4943052 BRANCHES 9 21 432000 22 864000 23 1296000 24 1872000 25 2304000 26 2880000 27 3456000 28 3935053 29 4367053 SYNCPOINTS 122 36000 72000 108000 144000 180000 216000 252000 288000 324000 360000 396000 468000 504000 540000 576000 612000 648000 684000 720000 756000 792000 828000 900000 936000 972000 1008000 1044000 1080000 1116000 1152000 1188000 1224000 1260000 1332000 1368000 1404000 1440000 1476000 1512000 1548000 1584000 1620000 1656000 1692000 1728000 1764000 1800000 1836000 1908000 1944000 1980000 2016000 2052000 2088000 2124000 2160000 2196000 2232000 2268000 2340000 2376000 2412000 2448000 2484000 2520000 2556000 2592000 2628000 2664000 2700000 2736000 2772000 2808000 2844000 2916000 2952000 2988000 3024000 3060000 3096000 3132000 3168000 3204000 3240000 3276000 3312000 3348000 3384000 3420000 3492000 3528000 3564000 3600000 3636000 3672000 3708000 3744000 3781732 3824620 3875640 3971052 4007052 4043052 4079052 4115052 4151052 4187052 4223052 4259052 4295052 4331052 4403052 4439052 4475052 4511052 4547052 4583052 4619052 4655052 4691052 4727052 4763052 NOTIFICATIONS 0 // A main pattern 4799052 samples long with 9 branches to endings nr. 21, 22, 23, etc., // with 122 synchpoints placed at 36000, 72000, ... samples starting from 0. // Its number is 1. The corresponding .ogg file in the AFLD4 is called AFLD4-01.ogg E [#1.0] I 10 0 288000 432000 0 SYNCPOINTS 7 36000 72000 108000 144000 180000 216000 252000 I 11 0 288000 432000 1 SYNCPOINTS 7 36000 72000 108000 144000 180000 216000 252000 I 12 0 288000 432000 2 SYNCPOINTS 7 36000 72000 108000 144000 180000 216000 252000 I 13 0 288000 432000 3 SYNCPOINTS 7 36000 72000 108000 144000 180000 216000 252000 // I: An intro of a given color. 0 = white, 1 = red, 2 = green, 3 = blue. // For details please read the paper [1] linked below in References. // Intro 10 would be AFLD4-10.ogg B 21 0 288000 432000 BPOINT 144000 TO_ALEPH 2 SYNCPOINTS 7 36000 72000 108000 144000 180000 216000 252000 NOTIFICATIONS 0 // B: A branch, contained in AFLD4-21.ogg B 22 0 432000 576000 BPOINT 144000 TO_ALEPH 2 SYNCPOINTS 11 36000 72000 108000 144000 180000 216000 252000 288000 324000 360000 396000 NOTIFICATIONS 0 ... END

MINI FAQ 2

  • Q: Does the file need to be called "main_layer.widsh"?
  • A: Unfortunately, yes, for now. At some point I'll add a CVar allowing you to specify a different filename.
  • Q: What happens when I change the file while the game is running?
  • A: You need to restart the game in order to hear the changes.
  • Q: What happens when I replace the .sqcb file while the game is running?
  • A: Good question, TBH, I didn't need to try that. In theory, next time the scene is loaded for playback, its new contents should be played back. However, if you need to provide a different timing information, you need to specify that in main_layer.widsh and that one won't be reloaded until the end of the KingdomCome.exe process.
  • Q: Is there a better way how to debug the music than restarting the game every single time?
  • A: We have an in-house playback tool. It hasn't been released to the public yet.
  • Q: Why is it so god-damn complicated?
  • A: These are the internals of the engine. In an ideal world, I'd provide a neat GUI and a WYSIWYG (WYSIWYH?) editor. I did develop a GUI interface for the soundtrack development I'd rather sit on it until it's ready for a public release. Thanks for your understanding!
  • Q: Did you really spend all that time typing all the numbers?
  • A: Hell no! I wrote a tool which generated those too. Looks intense but once we got used to our workflow, adding more music was relatively easy.
  • Q: What does WIDSH stands for?
  • A: Well-Informed Data Structure Header. Actually, that's not it. But it's not at all important. I was serious about Rainbow Dash though!

Reference

  • Sporka A, Valta J (2017) Design and implementation of a non-linear symphonic soundtrack of a video game. New Review of Hypermedia and Multimedia. Volume 23, 2017, Issue 4. Taylor&Francis. https://www.tandfonline.com/doi/abs/10.1080/13614568.2017.1416682 Please tweet at @adam_sporka if you wish to obtain a copy of the paper.