Difference between revisions of "Merged Plugin Guidelines for Personal Use"

From Nexus Mods Wiki
Jump to: navigation, search
m (Errors preventing merges)
(Merging: update)
Line 50: Line 50:
 
=== Merging ===
 
=== Merging ===
 
The latest tool for automated creation of merge files is [http://www.nexusmods.com/skyrim/mods/69905/? "Merge Plugins Standalone"] (MPS), which was released in December 2015.  It is supported [http://forum.step-project.com/forum/99-mators-utilities-support/ here].  It uses the API for [http://www.nexusmods.com/newvegas/mods/34703/? xEdit (aka TES5Edit, FNVEdit, etc.)].  It makes the process very easy, but you do have to exercise some judgement and understand the problem areas it identifies for you as it uses a "merging down" approach.  Be sure to carefully read the included documentation.  Various setup choices may need to be revisited for each merge plugin such as record copying, FormID renumbering, and inclusion of "art assets".
 
The latest tool for automated creation of merge files is [http://www.nexusmods.com/skyrim/mods/69905/? "Merge Plugins Standalone"] (MPS), which was released in December 2015.  It is supported [http://forum.step-project.com/forum/99-mators-utilities-support/ here].  It uses the API for [http://www.nexusmods.com/newvegas/mods/34703/? xEdit (aka TES5Edit, FNVEdit, etc.)].  It makes the process very easy, but you do have to exercise some judgement and understand the problem areas it identifies for you as it uses a "merging down" approach.  Be sure to carefully read the included documentation.  Various setup choices may need to be revisited for each merge plugin such as record copying, FormID renumbering, and inclusion of "art assets".
 +
{{xEdit Exclusivity}}
  
 
There are two potential problems with this automated "merging down" method.  One is that it has the ability to renumber "FormIDs" when they conflict between plugins.  This is often necessary but can cause "duplicate" entities with the same "name" but different IDs.  In the case of "Actors" if the "bash tag" "names" is used this can result in NPCs (such as "companions") sometimes appearing with "(dupe)" appended to their name, and sometimes not, which means there are two such actors with the same name and they will start to differ in particulars like their experience or carried items.  This is not a situation you want to have in your "saved games".  Such "merged down" results are effectively a completely new file.
 
There are two potential problems with this automated "merging down" method.  One is that it has the ability to renumber "FormIDs" when they conflict between plugins.  This is often necessary but can cause "duplicate" entities with the same "name" but different IDs.  In the case of "Actors" if the "bash tag" "names" is used this can result in NPCs (such as "companions") sometimes appearing with "(dupe)" appended to their name, and sometimes not, which means there are two such actors with the same name and they will start to differ in particulars like their experience or carried items.  This is not a situation you want to have in your "saved games".  Such "merged down" results are effectively a completely new file.

Revision as of 23:54, 26 April 2017

Overview

The active plugin cap for "Fallout: New Vegas" (FNV) is between 130-140 of both ESM and ESP file types, varying by system. There is a mod that pops a warning when you hit 139, but people can have problems before that. The solution is to create "merged plugin" files.

The focus here is getting under the plugin cap in the easiest possible manner consistent with a stable game by creating merge plugins for strictly personal use. This means we take a very conservative approach and generally try to avoid any surgery on mods. The resulting merge is highly specific to your personal "Load Order" (LO). Do not attempt to redistribute the result as it will almost certainly not work for anyone else.

There are a number of very similar and confusing terms used in this article. Fear not, they are explained in the Terminology section. But we start with an "overview" of what is involved to give you an idea of what you are getting into. It is worth while to learn how to do this, but it is also a learning experience that requires you to understand what is behind the differences in your choices.

"Renumbering of FormID's" always needs to be done when records are unique, except you do this by injection into the 'parent plugin' that is the object of your merge. Overrides are never renumbered, unless they are in plugins loaded after the parent plugin, and are overrides of unique records which are being injected. "Merging up" preserves and conforms to the FormID numbering in the parent, whereas "merging down" does not preserve the parent's numbering but instead conforms to that of it's new file, the last plugin in the merge list, potentially breaking anything relying upon that parent numbering, especially in the case of "patches". This is the reason Mator's Merge Plugin Standalone tool does not process "patches" (neither "fixes" nor "compatibility"), as it uses the "merge down" technique.

Note that a "merge plugin" reduces the number of plugins that remain active in the "load order". It does not resolve any record conflicts within plugins merged up into the 'parent master' plugin, making it now the sole "winning" plugin according to the "rule of one". Record conflicts between the plugins to be merged need to resolve separately before merging, by way of a "merged patch" file. Any 'dependent' plugins (that require a 'master' plugin you are merging) need to be included in the 'merge' process so they are adjusted to any new record "FormIDs". When 'dependent' plugins are involved, you are better off using the 'merge up' technique.

A "merged patch", like a "bashed patch", resolves record level conflicts (i.e. "compatibility patches") between multiple 'parent' plugins. It does not (in general) reduce the number of active plugins. It resolves conflicts between plugins where they "overlap". Only in the instance where the resolution results in all of the records of a plugin being incorporated into the "merge/bash patch", does it reduce the number of plugins that must remain active.

There is generally only one "merge/bash patch" file placed at the end of the "load order", though there can be one of each. In this latter instance the manually created "merge patch" is placed after the automated "bash patch" in order to override the "bash patch" winning records with your preferences. (The same basic process applies.) However, "Mator Smash" can now be used to create a "merged patch" of record conflicts for just the plugins you intend to merge together into a "merged plugin". This can be done for each "merged plugin" you create. It has the advantage of reducing the number of record conflicts that the final "merged/bashed patch" has to resolve.

Programs and Tools

Details

Terminology

First some terminology clarification. Forgive me if this is covering old ground but it's better to ensure we are on the same page.

Mods have (among other file types) "plugin files" with ESM and ESP extensions. In very broad terms, ESMs contain new "assets" and ESPs place those "assets" in the game world. These files contain records which are alterations to the vanilla game records or add new records to the game. In the normal course of events only the last such plugin to load that touches on a particular vanilla record "wins" when there are conflicting changes by different plugins. The winning plugin wins completely: all the other plugins that "lost" are simply ignored in their entirety, even the non-conflicting records. This is because the default conflict resolution behavior is "file" rather than "record" oriented. This "only one can win" behavior is called "the rule of one".

A "merged plugin" combines more than one plugin into a single file. This is typically done to reduce the number of active ESPs (and occasionally ESMs), but it also has the benefit of allowing non-conflicting records from other plugins included in the merge to be added to the game as well.

A "merged patch" combines more than one plugin's conflicting records to get a particular "winner" record as a result. The "Bashed Patch" (BP) created by Wrye Flash is such a "merged patch" file. (We refer to the WF file as the "bashed patch" and the manually created one as the "merged patch" to distinguish between their origins.) As a result it needs to be last (or near enough that no other plugin affects the same records) in the Load Order (LO) in order to ensure it's "winner" records remain the winners. For this reason the usual advice has been to use only one such "merged patch". If more than one patch file touches on the same record as the other, "the rule of one" comes into play and only one file will be used anyway.

So you had to choose between using either the BP (which uses algorithms and "bash tags" to determine the winner) and a "roll your own" merged patch file. But no longer. Now you can combine the use of both a "merged patch" and a "bashed patch". gromulos has provided a "Beginners guide to Xedit merged patch / WRYE bashed patch used together" which provides a "step by step" manual so the two work together due to some improvements to xEdit. A "Smashed Patch" produced by Mator Smash can used to provide record level conflict resolution for "merged plugin" files, and later as an alternative to the Wrye "bashed patch".

There is another mitigating factor though. I use "merged plugins" to both reduce the plugin count and to manually determine the winners of some conflicts within a subset of conflicts. Say I merge several plugins that affect "Perks". There may be a few conflicting records. I can determine which record wins in that "merged plugin" and then I have one file with only one "winner" record in each instance. "Mator Smash" can be used to automate the conflict resolution among just these plugins as well, by producing a "Merged <purpose> - Smash.esp" file which is then "merged" in with the remainder of the plugins. No internal conflicts, and unless there are other similar plugins not in my merge (which defeats one of the purposes, but perhaps not enough to forgo it), no other conflicting plugins. It's a way of breaking down the problem into more manageable chunks.

The arguments over which to use come down to how much control you want to exert over the result and how much time you are willing to spend to get that result. The BP uses "bash tags" to enable an earlier loading plugin to win conflicts. Otherwise the later loading wins. A manually create "merged patch" is strictly controlled by the creator, who determines the conflict winner. However, my BP affects over 10,000 records. I prefer to spend the time playing instead of working out "winners" to that many record conflicts. The xEdit "TES5Merged.esp" patch file as laid out by gromulos automates that pre-BP process to reduce those conflicts, and then you can edit the result to get exactly the winning record you desire, which gets passed on to the BP as the only record without conflicts. "Mator Smash" also recognizes and uses "bash tags", but not in the exact same manner as the "Bashed Patch" process.

Merging

The latest tool for automated creation of merge files is "Merge Plugins Standalone" (MPS), which was released in December 2015. It is supported here. It uses the API for xEdit (aka TES5Edit, FNVEdit, etc.). It makes the process very easy, but you do have to exercise some judgement and understand the problem areas it identifies for you as it uses a "merging down" approach. Be sure to carefully read the included documentation. Various setup choices may need to be revisited for each merge plugin such as record copying, FormID renumbering, and inclusion of "art assets".


There are two potential problems with this automated "merging down" method. One is that it has the ability to renumber "FormIDs" when they conflict between plugins. This is often necessary but can cause "duplicate" entities with the same "name" but different IDs. In the case of "Actors" if the "bash tag" "names" is used this can result in NPCs (such as "companions") sometimes appearing with "(dupe)" appended to their name, and sometimes not, which means there are two such actors with the same name and they will start to differ in particulars like their experience or carried items. This is not a situation you want to have in your "saved games". Such "merged down" results are effectively a completely new file.

The second potential problem is that this approach can also break any patches for the original plugin before it was merged. "Patches" designed for the original plugin's Form-IDs and will not work on these "merge down" result files, as their Form-IDs are now different. However, if you "patch" before "merging", this then only becomes an issue when later patches are released, forcing you to re-patch the original files and re-build your merged plugin.

The original solution in both potential problem cases is to "merge up" using xEdit and following the PDF Gribbleshnibit8's Merge Up Guide (MUG) instead of using Mator's "Merge Plugins Standalone" (MPS) as described in the remainder of this article. The video Making a Merge Patch by Roy Batty and the FNVEdit Training Manual both use this "merge up" approach. It is more complex and time consuming, but safer as well as more certain to produce your desired results. There is now a section here devoted to that procedure with some supplemental information not covered in either those other guides.

Now that Mator Smash is available (although only in an "Alpha" release state at the moment), it provides a functional alternative way to resolve record level conflicts other than "merge up" or Wrye Bash and it's variants. However as it is still in testing and development, you need to test the effectiveness of it's results and be willing to report bugs to help it reach it's full potential. But it works with "bash tags" and you can edit the results in xEdit to fine tune the results if desired. It can also produce a "final patch" alternative to the Wrye "Bashed Patch".

But regardless of which approach is used, the basic concepts of what to include in different "merged plugins" still apply. Once the "problem cases" have been "merged up" or "Smash Patched" you can use MPS to merge them with other non-conflicting plugins.

Merge Up Procedure

This procedure conforms to the naming convention used in Gribbleshnibit8's Merge Up Guide, and is best used with "patches" as the ModB files that only apply to the parent or ModA file. ("Compatibility patches" to enable more than one plugin to work together should be in a "merged/bash patch" file, which is a different process.)

Step 1. Load ModA related plugins.

  • Load the target ModA 'parent plugin', and all the ModB plugins related to it (even those you do not intend to merge), into xEdit.
  • Go up to and <Right-Click> on the "FormID" column header, select the right pointing arrow after "Files", and make sure "always by load order" is selected. (If this capability is not available, it means you are not using the latest version of xEdit.)
  • <Right-Click> in any open, unused space in the left-hand pane and select "Remove Filter" from the displayed "Context Menu". This ensures nothing is being filtered out of our view.
Each and every plugin that will be merged directly into the parent will all be done at the same time and not in steps.
  • Note that xEdit assigns it's own relative two-character hexadecimal "mod index" to the plugins it loads. All records shown will begin with one of these relative mod indexes (which appear to the left of each plugin name in [square brackets]). When referring to unique records in a plugin, we mean one which has the same "mod index" as that plugin (regardless of background color), and not the index of any other mod.


Step 2. Add 'Masters'.

Add any masters before ModA to ModA if missing.
  • Add all the masters that are loading before each ModB plugin to each ModB plugin, EXCEPT any other ModB plugins themselves (as they will be dropped later once they have been merged). Especially add the ModA plugin as master to all the ModB plugins. This includes adding to related ModB plugins that are NOT going to be merged, so any references will be updated when FormIDs are changed. (If they aren't actually needed such added masters will get removed later when we "clean masters".)
  • Sort the masters in each plugin after adding.


MfMUP Fig. 1-2n3
  • Save all the plugins that had 'masters' added to them (by enabling/selecting the checkbox for each in the 'target' window that pops-up when you attempt to close xEdit).
(See Fig. 1-2n3. Click the link/thumbnail to see the enlarged image. Use your browser "<back page>" control to return to this page.)
  • Reload all plugins from Step 1, and check the added 'masters' are in each plugin's "File Header" before proceeding to Step 3.
The 'Add Masters' selection window should come up empty if all is correct, allowing you to 'cancel' it by clicking on the "x" icon in the upper right-corner of the pop-up window. Otherwise, add the 'masters' indicated, and repeat this step when all have been checked this pass.
Failure to perform this step can result in the loss of hours of work in the last step.


Step 3. Select ModB.

Select the first loading unmerged plugin (other than the 'parent') to be the first/next to be merged, and expand it so you can see the record types.


MfMUP Fig. 1-4
Step 4. Basic Record Injection.
(Record types vary by plugin.)
  • This is the order used to inject unique records for each of the "ModB" plugins (one at a time) to be merged, broken down into 11 sub-steps by record type:
  1. All forms which are not record type: CELL, Worldspace, Dialogue, Script or Quest.
    • All record types to be selected in one plugin can be expanded and done at once, but if there are too many to fit on the screen, you may find it easier to break the step up into batches of record types for selection. Just remember to process them all in the record type sequence given, and inject them all at one time for each plugin to ensure they each receive a unique form number. Refer to the FNVEdit Training Manual section on "Merging a Plugin into another Plugin or Master" or "Gribbleshnibit8's Merge Up Guide (MUG)" if you have any questions.
    • You want to select all of these unique records (use <Shift+Click> and <Ctrl+Click>) before you:
    • review your selections to make sure you didn't accidentally skip/miss some and verify only unique records are selected,
    • renumber by right clicking on one of the selected records and selecting "Change FormID" from the "Context Menu", then
    • selecting the main 'parent plugin' (ModA) as the target file when the list of plugins appears.
    This will "inject" all the selected records into the "ModA" plugin. The consequence is the parent index number of the FormIDs for those unique records will be changed to that of ModA's index. (Any resulting duplicate FormIDs will automatically be adjusted.)
    • If selecting only one record to be renumbered, you will need to note the last "Changing FormID [<number>] in file <filename> to [<new number>]" message in the "Messages" tab of xEdit, add one to the "<new number>" in hexadecimal, and place that value in the "New FormID" prompt manually. ("<Ctrl+I>" Saves the FormID of the highlighted record to the clipboard.) The "starting value" given is not correct otherwise. This is only required when renumbering single records.
    xEdit may now prompt you with a message "There are overrides in later plugins, do you want to update them?" You should choose Yes to this and it will automatically fix them for your merged plugin. This function was added to xEdit for this very purpose. (Thank you Zilav).
    (You may find it necessary to scroll up and down the left-hand pane before all the updated FormIDs are displayed correctly in Bold Italics.)
    • Examine the "Messages" tab in the right-hand pane of xEDit for any errors.
    • Now process the remaining unique (white-background) form record types (following as sub-steps 2-11) in the same manner. It's important to process each record type separately so the FormIDs are updated properly.
  2. CELL Persistent records (Expand "Cell" | < Block # > | < Sub-Block # > | < record # > | "Persistent")
  3. CELL Temporary records (Expand "Cell" | < Block # > | < Sub-Block # > | < record # > | "Temporary")
  4. CELL Parent Records (if unique) (Expand "Cell" | < Block # > | < Sub-Block # > | < record # > on "white-background")
  5. Worldspace Persistent records
  6. Worldspace Temporary records
  7. Worldspace Parent Records (if unique)
  8. Scripts
  9. Quests
  10. Dialogue Responses
  11. Dialogue Top Level topics (if unique)
    • Examine the "Messages" tab in the right-hand pane of xEdit for any errors.
  12. You want to collapse all record types for the plugin by "<Left+Click>" on the "-" sign to the left of the plugin name, and then "<Alt+Left+Click>" on the "+" sign that replaces it to expand them ALL again. (Recall that collapsed record types don't get copied.) When we are finished injecting all unique records, there should not be any records that do not begin with a mod index other than 00-05 or that of our "ModA" plugin. Anything else in an earlier loading "ModB" plugin should have already been injected and updated in this plugin. (If there are any, they will cause Step 6 to fail and you have to cancel the session: uncheck all plugins and "<Left+Click>" the "X" in the upper right corner of the "save changes" window to exit without saving, and restart Step 4 over. Take the time to check now.)


Step 5. Inject All ModB Records.

  • Repeat steps 3 and 4 for all 11 sub-step record types of Step-4 for ALL the other loaded ModB plugins you intend to merge, working your way down from lowest numbered (earliest loading) to highest numbered (latest loading) in load order.
Examine the "Messages" tab in the right-hand pane of xEdit for any errors.


MfMUP Fig. 1-6
Step 6. Copy as Override.
  • Expand any record types in any of the ModB plugins that haven't been (i.e. collapse each plugin file and then <Alt+click> the "+" to the left to expand everything if not sure), and select all sub-records from the first ModB to the bottom of the other merged plugins loading after it (this will ignore/de-select the "File Header" records). This time records of all background colors should be included (white, green, yellow, and red). Place the cursor over any 'unique' sub-record, right-click and select "copy as override" into ModA; or to "<new file>" if you intend this to be a new merged plugin file with ModA as it's master. If you don't merge into ModA, then ModA's records are not included in the result.
    MfMUP Fig. 1-6a
(See Figs. 1-6 & 1-6a. Click the link/thumbnail to see the enlarged image. Use your browser "<back page>" control to return to this page.)
  • Note you have to select "sub-records", under the first "record type" level. This will cause the Cell "Block #" sub-record types to be selected but not their sub-records, but all other sub-records should be selected.
  • (If you only have "deep copy as override" as an option, the problem is you didn't right-click on a 'unique' sub-record to bring up the context menu.)
  • You may be prompted that this will change the FormID of other plugin records loading after. You want this to happen. It is why you loaded those plugins even though they are not being merged.
(The previous records you just injected in Step 5 will now have a green background color once the "copy as override" is done.)


Step 7. Apply 'Merge Overrides' Script.

Be sure to have used the "Copy as Override" command on the selected records in Step 6, before proceeding. It's easy to confuse these two steps as they are similar but have significant differences.

Now that all records in all the loaded, merged ModB plugins are overrides, you can now select them all (white-, green-, yellow-, and red- backgrounds), right-click on one and use the "Apply Script" command with the script "Merge overrides into master.psc" (bundled in TESEdit 5 v3.1.3+) to merge overrides into the parent (ModA) only.
MfMUP Fig. 1-7
MfMUP Fig. 1-7a
(See Figs. 1-7 & 1-7a. Click the link/thumbnail to see the enlarged image. Use your browser "<back page>" control to return to this page.)
  • xEdit will not merge overridden records by default; this script was created for this very purpose. (Thank you Zilav).
  • You need to select the sub-records, not just the "record type" when using the script.
  • The script will not copy 'injected' records even if selected. Don't worry about it.
  • Try not to use "Deep Copy" as it doesn't always work as expected. Try with right-clicking on a different sub-record if that is your only option.
  • Always check to make sure all records are merged (now either green- or yellow-backgrounds) when the script is done.
  • Ignore red-backgrounds, which are "losing conflicts" and being dropped from the result).
  • Orange/olive-text on red-backgrounds are 'Identical To Master'(ITM) "conflict winners" and won't be merged.


Step 8. Clean Masters.

  • Right-click on ModA and select "clean masters" from the context menu.
  • Repeat on any ModB 'dependent' plugins (ones which have ModB as a 'master') as well. Once the ModB master is deactivated, their dependents should be using the new ModA as their master. This may require adding ModA as a new master to the dependent plugin.
Generally this means working down through all the plugins loaded after "ModA" because you added "prior loading plugins" as masters to them in Step 2.
Never manually delete a parent from a plugin unless you are absolutely sure it is not referenced. Always use "clean masters" instead, as it won't remove a "master" if there are any references to it.
In some instances "clean masters" will remove plugins (like with TTW) when it should not; re-add them afterwards and "sort masters" again after saving and reloading.


Step 9. Check All Plugins for Errors.

Note that some plugins will not be in Bold Italics (indicating they were changed) prior to this step. The xEdit "check for errors" process will flag them as changed, so you may want to take a screenshot of this window first or otherwise note which files have changed by this point. You will need to know these file names in Step 11, the "Verify and Cleanup" step for copying to their own project folders.
  1. Check for errors by right-clicking on each plugin, and selecting "check for errors" from the context menu that then appears.
  2. Check in the 'File Header' for unexpected 'master files'.
  3. The other loaded but unmerged plugins may have some of the ModB files we merged into ModA as 'masters'. As those records have now been injected into ModA, we have to make sure each unmerged plugin has ModA as a 'master' instead or it is added as needed, and then that we "clean masters" so the merged ModB plugins get removed. If they do not, it means we missed updating some record FormID, likely at step 6 - "copy as overrides" and have to start over.
  4. If there are any remaining dependencies (evidenced by a master you expected to be dropped still being present in the plugin's "File Header" record) you can use the script "List records referencing specific plugin.pas" on that plugin to list them in the log window so you can fix them.
  5. Once corrected, "sort" masters on that plugin.
  6. "Clean masters" again on the ModA plugin, and re-check all plugins for errors.

If you are unable to remove an unexpected master dependency by "clean masters", then you will need to start over from scratch, by restoring all files modified by this procedure from your backups and beginning from Step 1 of this Stage again. Rethink the master files that you add to the plugin(s) having the problem.


Step 10. Save the Result.

  • Save the resulting ModA as the original ModA's filename (or the "<new file>" name selected in Step 6 as the target of the "Copy as Override" or as a "<new file>" replacement version of ModA). This is now our 'patched parent plugin'.
    MfMUP Fig. 1-10
  • Save any other modified plugins as well. These will usually be those un-merged plugins that had one or more of the ModB plugins as masters that should now be gone and replaced with ModA. Some may have have some records updated with new FormIDs from the ModA plugin as well.
(See Fig. 1-10. Click the link/thumbnail to see the enlarged image. Use your browser "<back page>" control to return to this page.)
  • If you haven't already in Step 9, take a screenshot of this window or otherwise note which files have changed. You will need to know these file names in the next (Verify and Cleanup) step.


Step 11. Verify and Cleanup.

The ModA file now includes the records from the other plugins we merged. Only our new version of ModA will be needed hereafter, so the ModB files that were merged into it can be left out. They will be deactivated/uninstalled from our "load order" once we check (once again) all the records have been successfully merged. However, do load the unmerged "ModB" plugins that were previously loaded.
  1. When we have verified (as in Step 9) that the ModB merges into ModA are correct and complete, we should copy the new ModA 'parent plugin' file to it's new project folder where we have been tracking progress in our 'ReadMe' file.
  2. In addition to ModA, various ModB plugins that were not merged have been changed (such as the replacement of the merged plugins as "masters" with the new ModA, or records updated with new ModA FormIDs). These should also be saved to the ModA project folder and added to the ReadMe file, as they are no longer the same as their original versions but specific this ModA version. They should be installed (overwriting the originals) when this package is installed.
  3. There is still one further consideration: other 'asset files' (meshes, textures, sounds, etc.) that are part of the original plugins, both the 'parent' and any in the merged patches. Here you have a choice: either copy those asset files into the 'project folder' (starting with the parent's files, and then the patch files in the same sequence as they were in the "load order", overwriting any conflicts); or by noting that they exist in the original mod and have to be installed without that plugin. Whichever choice you make, record it in the "ReadMe" file so you don't forget it.


Note: This is just the general procedure and there will be times when you need to do some other records first. You'll get the "Requires master to be before" message and you'll have to determine what needs to be done first by going in smaller blocks. This will help avoid any "deal breaker" issues, and you should be able to handle any issues which come up.

Merge Down Procedure

  • First of all don't try to put everything into a single merge plugin.
  • If you haven't already, ensure you don't have any "missing masters" required by any of your plugins. See the Nexus FNV Wiki article Missing Masters which describes how.
  • Disable your "Bashed Patch" (BP) or manually created "merged patch" before working on merges, and activate any mods deactivated by it.
    • You should be using a Wrye Flash Bashed Patch or a manually created "merge patch" file if you have many plugins. (Resolves record level conflicts.) See this S.T.E.P. Guide to Merging Plugins which covers both approaches. (Use of "Mod Organizer" is not required.)
    • You can manually create a "merge patch" file instead of using Wrye Flash, but if you play a heavily modded game WF is simply easier and "good enough" when thousands records are involved. You can always use FNVEdit to tweak the resulting BP if desired.
    • Use the Wrye Bash Pictorial Guide to quickly get functional with "Wrye Flash" and the "Mods" tab to build a "Bashed Patch", which I think is the easier approach for a novice modder.
  • Sort with LOOT. While it's results may not be 100% (primarily due to indirect masters that are not listed in a given plugin), the sort result can be adjusted and those changes incorporated into LOOT's "metadata" for future use.
  • Follow the "Fallout New Vegas Beginners guide to modding" article by gromulos to get your "Wrye Flash] (WF) "load order" check-boxes "green". The article covers why this is a "good idea".

The mods that can go into a merge need to be "contiguously" adjacent in your load order (LO). No "active" mods not included in the merge between files being merged. This provides your first clue as to how many separate merges you are going to need.

Note that once LOOT has sorted your LO, you can "move" mods up and down in the Wrye Flash "Mods" tab with <Ctrl+Cursor> keys to make them contiguous. You can "group select" (<Shift+Click> or <Ctrl+Click>) files as well before moving them. Likely other mod managers also provide this capability. I always then run LOOT again to see if it doesn't like the result of my manual manipulations. Sometimes it really wants to put a mod elsewhere that doesn't fit adjacently into a "merge group". Personally I play it safe and leave such mods alone unless I feel really confident I know why it's safe to override LOOT's insistence.

As a general "placement" rule, the group needs to be positioned where the "highest numbered" (lowest in the LO list) is positioned. This is where you should place the resulting merge plugin. This is the primary reason you need all the mods you use active, even those that the BP may deactivate.

The S.T.E.P. guide Merging Plugins covers the basic "merge down" process very well in my opinion. The following are some key points I learned to supplement it.

Types of Merge

The simplest merge to create is "non-conflicting" mods. These are usually things that add new items, such as weapons or armor. Let Wrye Flash deal with conflicting mods, unless you don't like the results and want to create a "merged patch" file instead of using the Bashed Patch. "Mator Smash" has now developed to the point where can also be used for this "patch of conflicting plugin records" purpose, but please bear in mind that it is still in development. Use the "Smash Patch" or "Bashed Patch" for the final, overall record conflict resolution, but not both; or a manual "merged patch" file of the Bashed Patched file in order to get a different preferred "winner" in the case of particular records. Note that if you "renumber FormIDs" you will lose any such items you already have in your save game's inventory, so this is best done before you start playing in earnest. If "renumbering FormIDs" seems to be necessary, strongly consider using the "merge up" approach instead.

An example of this type of simple merge are the "Pre-Order Packs" (POPs; referred to by some as the "crap packs"). They just give you free starting equipment and nothing else. You can remove them as long as you don't have any mods that use them because the POPs inject their records into "FalloutNV.esm" so that they can be overwritten without a master dependency. That's how patches like "Yukichigai Unofficial Patch (YUP)" are able to patch them without requiring them.

You can also merge all four of them together to save room, although that requires you to extract the contents of their BSAs and either repackage them to a BSA named for your "merge plugin" file or leave the contents as loose assets in your data folder. They will still work just fine with YUP and the like if you do this. While most plugins shouldn't modify them, note that many that add or modify unique weapons, "weapon mods", or armor in them do require the individual ESM files to be present. Know what mods you are going to install require before merging the POPs.

Be careful with plugins that add their own "configuration menu" by way of entries in the Pipboy tab menus such as "Apparel" or "Misc". Usually these plugins need to be able to see their own ESP file in order to work correctly. They do not make good candidates for a "merged plugin" for this reason. Test in-game carefully if you do. This does not seem to be an issue with plugins that use "Mod Configuration Manager" (MCM) instead.

The next simplest merge is "fixes" that are unrelated to each other (because they aren't conflicting). This might be considered a subset of the first group, but I find they are less likely to be updated over time, and therefor worth having in their own merge. (Actually I have several such "fix" merges because they need to reside in different areas of my large load order.)

Third simplest are mods that all relate to a single topic mod (such as "Perks" or "Advanced Recon Armor"). Optional files for mods fall into this group UNLESS they are "patches" for other mods to work with the subject mod. Those fall into the next group. Here the LO is going to determine which of these merged mods wins any conflicts.

Less simple are files all related to an "overhaul package" such as Project Nevada, WMX, "Weapons of the New Millenia", etc. The primary issue here is dependencies (other files looking for a specific file name in the package as a "master", or that have a BSA file).

Merging quest mods are the most complex to do and not even expert mod creators would try and merge some mods like the SomeguySeries "New Vegas Bounties I-III, The Inheritance, Russell, etc." because renaming all the voice files without an automated solution would take eons of time. Breaking dialogue is extremely easy to do. You'll run into issues where the worldspace and cell records need to be added first, then scripts and quests and there can be complications with references through out. These kinds of things are best merged using GECK in version control mode. However that requires a server/client setup and has it's own set of complications. You can do it in FNVEdit, but only after gaining experience and knowing how to deal with the problems. Compatibility patches are something else to keep in mind and you'll have to have to those loaded along with the mods they are patches for in order to not break them. They are dependencies that won't work with your merged plugin otherwise.

Merges of patch files into their master plugin can be simple (only one master) or complex (multiple masters). In either case, these should use the "merge up" approach identified above; not MPS. "Mator Smash" can be used, but remember that: 1) it is still in development so the results need to carefully be examined and tested, and 2) it creates a completely new file with new Form-IDs.

MPS won't let you merge mods with "errors", but it does have an "ignore errors" option. Be very skeptical about using this. The only time I use it is for "Unknown flags" which have a value. That just suggests FNVEdit doesn't know how to interpret the flag being used, which is not fatal (as far as I know). I do not ignore "NULL references". Those should be examined in FNVEdit or GECK and resolved or the plugin excluded.

Complications

  • The things to always watch out for are:
  1. Unresolved errors, (fix them or exclude from merging)
  2. BSA files, (see the MPS "Advanced Options" tab)
  3. Sound Files, (see the MPS "Advanced Options" tab)
  4. Self-referencing scripts, (exclude them)
  5. Deleted NavMeshes. If these cannot be fixed, do not include the plugin in the merge. For a guide on how to fix these, download the Guide for fixing deleted NavMeshes by AndrealphusVIII. (Written for Skyrim but works for deleted NavMeshes in general). See also:
  6. NavMesh conflicts. These need to be manually resolved in the game Editor, so are best left to the authors. However, for "NAVI" errors, see the NAVI NavMesh Conflicts section. If this doesn't resolve the problem, leave out of a personal merge.
  7. Dependencies, all of which MPS will spot and tell you about with icons.
  8. Patches. The "merge down" process of MPS cannot handle patches correctly as they require preserving the FormIDs of the 'parent' plugin. Use "Mator Smash" or the "merge up" process to deal with them.
  • ESMs can be merged, but generally they are masters for dependencies and I leave them out (block from merging) just to avoid the complications. While it is possible to deal with these issues, unless you want to become an advanced merge maker, it's easier to skip including files (i.e. "Block" them from merging) that have any of these issues. There are usually plenty of other, simpler merges to get you under the plugin cap.
  • Note that when you create a merge you may need to still have some elements of the original mod installed separately (ESMs, menus, etc) while others such as ESPs and meshes & textures can be included in the merge. For this reason I recommend creating a "ReadMe" file for each merge plugin you create that lists which files are included and which still need to be installed separately. I also find it useful to make note of mods I initially thought to include but later rejected and the reason why. I also note any Bash Tags from the original mod, and which mod the merge needs to follow in the LO. You can add the Bash Tags in the "Comments" section of the "Installer" tab of WF (aka BAIN) as "{{BASH:<tag>, <tag>, <etc>}}" (which then get picked up by LOOT), or individually in the "Mods" tab in the "Bash Tags" field in the bottom right part of the screen for the selected merge plugin.

NAVI NavMesh Conflicts

Sometimes after merging plugins, you will see a Pop-up window:

 "There were some NavMesh conflicts. You'll need to rebuild these navmeshes in the CK:
[< Merged Plugin Filename >.esp] (i.e. "[Merged - More Perks.esp]")
 [NAVI:<Form-ID>] (i.e. "[NAVI:00014B92]")

This pop-up does not currently create an entry in the log files, so be sure to note the "NAVI:<Form-ID>" given.

In xEdit: search on the NAVI < Form-ID > (i.e. "00014B92"), which should return:

"< Filename > \ Navigation Mesh Info Map \ < Form-ID >", as in:
"FalloutNV.esm \ Navigation Mesh Info Map \ 00014B92" in the "View" tab.

According to Mator:

NAVI records are used to effectively "connect" a bunch of navmesh and door records into a single "network". When you merge multiple plugins that affect the same NAVI record, merge plugins follows the rule of one and takes the winning NAVI. The issue is that conflicting NAVI edits really need to be merged. If your smashed patch handled NAVI records then this shouldn't be a problem. If it didn't then you will need to load the ESP in the CK and save it (the CK will automatically build correct NAVI information). Loading in the CK requires you to ''ESMify'' the plugin's masters for things to not get clobbered.

(You can ESMify an ESP file by loading it into xEdit, selecting the plugin in the left-hand pane and expanding it (< Left-Click > on the "+" to the left of the plugin name), and select "File Header". In the "View" tab in the right-hand pane, toggle the "ESM" flag on. Upon exiting this will create a new "duplicate" of the ESP file, but with an ESM extension.)

In other words, you may need to ESMify the master plugins of the "Merged Plugin" file (if necessary), load those ESM masters and the "Merged Plugin.esp" into the Construction Kit, and save the (should be automatically corrected) result immediately as the "Merged Plugin" file again or as a new filename.

You don't need to keep the ESM versions of those files afterwards, though. They should be removed when done. (This is primarily to avoid breaking anything that depends upon those being ESP files, such as sound files, etc. Also because the game really doesn't expect ESP files to act as if ESMs.) Leaving both around will cause the game to use the ESM files even though the ESPs still exist. Avoid confusing the game engine. An ESM is not inherently superior to an ESP file.

Errors preventing merges

Most errors in a plugin that MPS cannot resolve are not going to work in the game anyway. The general approach is then to remove them from the plugin using xEdit, so you can merge the result. Some you can attempt to "read the mind of the author" and correct yourself if you can figure out how things were supposed to be. But most can simply be done without.

Not everything can be fixed with xEdit. Sometimes, all you need is to load the original or merged plugin file into the Construction Kit, and immediately re-save it without changing anything. That alone will sometimes resolve the issues, as some "unresolved references" like QSTI or NAVI sub-records that xEdit won't touch or applies the "rule of one" to will automatically be removed or fixed. If that isn't sufficient, reconsider including that plugin in your merge unless you really want to get involved in editing it. If the plugin requires a "Script Extender" like NVSE, be sure to have your Construction Kit load with it. (This is generally a good idea even if not required.) Use the GECK Power-Up v1.4 version (which is NVSE aware) or see Issue: How to get GECK to load with NVSE?.

Mator has a video covering the basics of this removal process intended for users of MPS: TES5Edit - Fixing Errors. For those who prefer written instructions, or clarification of points raised in the video, see the TES5Edit documentation Mod Cleaning and Error Checking. To help with understanding the record types, see the next section.

Plugin File Header and Record information by game

Thanks to Sharlikran on the "Afkmods" site Wrye Bash - All Games thread for the following:

"While these pages still provide the best public documentation of the Oblivion file formats, all mod makers should also be using Tes4View, which currently provides the most complete and correct understanding of the mod file format in a readily accessible manner." - UESP Wiki (on the Oblivion page for the TES4 Mod File Format). [These days, instead of the Oblivion tool TES4View, use the game-specific version of xEdit instead (e.g. TES4Edit, FNVEdit, etc.).]

"One of the things about those Wiki pages is the format of the 24 bytes doesn't change much. In my opinion you really only need to compare Oblivion, with one of the other games. About the only thing that changes is the Flags because they are record specific. The TES4 header is defined in the following Data which is a group of subrecords. A GRUP or Top Group also has 24 bytes and indicates the beginning of what will follow. Bethesda's internal tool sometimes introduces duplicate Top Groups which are combined in xEdit. Each record starts with 24 bytes, and then the subrecords follow. After the 24 bytes is the size of the data that will follow from what I understand. From what I know Oblivion is 20 bytes, and that's about the only difference. Zilav [of the xEdit team] only mentioned the other day when I asked that Oblivion doesn't have a Form Version because it was the first use of the new format."

Fallout Record Types

Sources:

  1. Tes5Mod talk:Mod File Format contains comparative counts of use by type between FO3 and FNV, omitted below.
  2. TES5Edit/fopdoc as of 2 Jun 2014 documents the FO3, FNV, and FO4 plugin file formats known to the xEdit team. Only those pertaining to FO3 and FNV are listed here.
FOPDOC has each record on a separate GitHub Flavored Markdown page with sub-record type information. This should be referenced for further details like flag values. (Markdown format is quite readable by the average user with an ordinary plaintext editor.)
  • ACHR = Placed Character/NPC Reference
  • ACRE = Placed Creature Reference
  • ACTI = Activator (usually something like a light switch, plant, etc.)
  • ADDN = Addon Node (Particle Effect)
  • ALCH = Ingestible Potion/Medicine (Alchemy)
  • ALOC = Media Location Controller (FNV only)
  • AMEF = Ammo Effects (FNV only)
  • AMMO = Ammunition
  • ANIO = Animated Object
  • ARMA = Armor Components (Addon)
  • ARMO = Armor
  • ASPC = Acoustic Space (sound effects)
  • AVIF = Actor Value information
  • BOOK = Books (usually one that's actually usable.)
  • BPTD = Body Part Data
  • CAMS = Cameras
  • CCRD = Cards (Caravan) (FNV only)
  • CDCK = Deck of Cards (Caravan) (FNV only)
  • CELL = Cell (Areas that make up the game world. Each interior is in its own cell.)
  • CHAL = Challenges (FNV only)
  • CHIP = Gambling Chips (FNV only)
  • CLAS = Class
  • CLMT = Climate
  • CMNY = Caravan Money (FNV only)
  • CONT = Container
  • CPTH = Camera Path
  • CREA = Creature
  • CSNO = Casino (FNV only)
  • CSTY = Combat Style
  • DEBR = Debris
  • DEHY = Dehydration level (FNV Hardcore mode only)
  • DIAL = Dialog topic
  • DOBJ = Default Object Manager
  • DOOR = Door (Teleports or removes barrier, often used to teleport the player into cells.)
  • ECZN = Encounter Zone
  • EFSH = Effect Shader
  • ENCH = Enchantment (such as Stimpak's healing ability.)
  • EXPL = Explosion
  • EYES = Eyes
  • FACT = Faction (All NPCs and creatures are assigned to one.)
  • FLST = Form ID List
  • FURN = Furniture
  • GLOB = Global
  • GMST = Game Setting
  • GRAS = Grass
  • GRUP = Group
  • HAIR = Hair
  • HDPT = Headpart (Eyes, hair, beards, etc.)
  • HUNG = Hunger level (FNV Hardcore mode only)
  • IDLE = Idle Animations
  • IDLM = Idle Marker
  • IMAD = Image Space Adapter
  • IMGS = Image Space
  • IMOD = Item Mods
  • INFO = Dialog Response
  • INGR = Ingredient
  • IPCT = Impacts
  • IPDS = Impact Data Set
  • KEYM = Key
  • LAND = Land
  • LGTM = Lighting Template
  • LIGH = Light
  • LSCR = Load Screen
  • LSCT = Load Screen Type (FNV only)
  • LTEX = Land Texture
  • LVLC = Leveled Creature
  • LVLI = Leveled Item
  • LVLN = Leveled NPC
  • MAPM = Mapmarker (These are what tell you where a town, etc. is on the map.)
  • MESG = Message
  • MGEF = Base Magic Effect
  • MICN = Menu Icon
  • MISC = Miscellaneous item (such as random junk.)
  • MSET = Media Set (FNV only)
  • MSTT = Moveable Static
  • MUSC = Music
  • NAVI = Navigation Mesh Info Map
  • NAVM = Navigation Mesh (NavMesh)
  • NOTE = Note (which can be activated to show a text string, or the FormID of a Dialog record.)
  • NPC_ = NPC (Non Playable Characters)
  • PACK = Package (See the FOPDOC for the types of sub-records controlling when it gets triggered.)
  • PERK = Skill Perks
  • PGRE = Placed Grenade
  • PMIS = Placed Missile (FNV only)
  • PROJ = Projectile
  • PWAT = Placeable Water
  • QUST = Quest
  • RACE = Race
  • RADS = Radiation Stage
  • RCCT = Recipe Category (Collection)
  • RCPE = Recipe
  • REFR = Placed Object Reference
  • REGN = Region
  • REPU = Reputation
  • RGDL = Ragdoll
  • SCOL = Static Collection
  • SCPT = Script
  • SLPD = Sleep Deprivation Stage (FNV Hardcore mode only)
  • SOUN = Sound
  • SPEL = Actor Effect (Spell)
  • STAT = Static
  • TACT = Talking Activator
  • TERM = Terminal
  • TES4 = TES4 record format header. (Remnant of TES4:Oblivion.)
  • TREE = Tree
  • TXST = Texture Set
  • VTYP = Voice Type
  • WATR = Water
  • WEAP = Weapon
  • WRLD = Worldspace
  • WTHR = Weather


Finishing the Process

  • Examine the resulting merge in xEdit for any errors or undesired conflict results. IF there are any, resolve them or remove the plugin from the merge.
  • The MPS places the resulting merge file in it's own folder. You will need to copy and create a "package" (7zip or RAR file) to keep your ReadMe with the plugin and any assets. (I recommend you create a naming convention to make it easier to identify your merges: i.e. "Merge - <purpose>".)
  • Install the merged plugin you just made.
  • Run LOOT and adjust in the LO as needed to get it placed just after where the highest numbered plugin that was merged is located.
  • Disable all the mod plugins that have been merged. (I uninstall them, and then re-install only the necessary files not included in the merger according to my ReadMe information because I use WF as my mod manager and then I can "Anneal All" to resolve "Install Order" issues.)
  • Use the "metadata" capability of LOOT to identify the plugin that your merge needs to "Load After" to ensure it is placed correctly in your LO. Use the metadata's: "required" field to identify plugins that are "indirect masters" but not in your "merged plugin" list of "master files". (This "required" field is ONLY for master files not included in a mod's list of masters. Consider adding the missing master to that plugin's list by means of FNVEdit instead.) Note this "metadata" may need updating if you add other mods later or rebuild the merge.
  • Run LOOT again to ensure no issues have been created when you disabled or uninstalled mods that were merged (such as "missing masters"), and that your LOOT metadata information is working as intended.
  • Run FNVLODGen to rebuild your LOD files (because you have greatly changed your LO and the LOD "quads" are LO dependent.)
  • There is a xEdit "Generate Bash Tags" script for which can determine Bash Tags for a mod, including your newly created "merged plugins" and Smash "merged patch" files for those plugins.
  • Note that "Mator Smash" uses "tags" differently: as a set of named logical groups that point to the specific data in records to process by the patcher. "Smash" (at least at the moment) applies tags to all records.
  • Run the script against your newly created Smash "merge patch" for plugins that will go into a "merged plugin file.
  • Then run the script again against the "merged plugin" file.
  • However, when you use the "Bashed Patch" you still have to exercise some judgement as the "tag generator" script can't tell if you really want Wrye Bash to apply those tags to that mod. Only use a Tag for Bash when it is IMPORTANT that mod's tagged records win. Just because a tag can be applied is not a reason to do so. Remember that when more than one mod has the same Bash Tag applied to it's record, the LAST tagged wins.
  • Rebuild your Bashed Patch (or "merged patch" file).
  • Test each merge plugin just like the original mods before moving on to the next merge. The time to find problems is now, before they get into your save game.
  • While you can add a new plugin to an existing merge, you have to completely redo it to remove one. This is another reason to have several rather than one massive merge.

Combining Smashed Patches and Merged Plugins

According to Mator, the author of both:

  • Make a "Smashed Patch" for the plugins you intent to merge.
(It should not be named the same as your final "Smashed Patch" file. You can have multiple "Smash Patches" for sub-sets of plugins with any names, so use one related to the plugins for which it is resolving conflicts.)
  • Include that "Smashed Patch" in the "Merged Plugin" file.
  • Smash everything after all merges have been completed.
This will be your "final" Smashed Patch file.
  • Place the final Smashed Patch.esp at the end of your "load order", similarly to with a "Bashed Patch".
Note that LOOT will recognize the file name Smashed Patch.esp in it's masterlist, tagged with "NoMerge", global priority the same as Bashed Patch, and additionally set to load after TES5Merged.esp.

See the Mator Smash Quick Start guide for help getting "Smash" initially configured for games other than Skyrim, along with additional information and resource links.

Merged Patch, a Bashed Patch, or both?

Which to use

A manually created Merged Patch and Bashed Patch both at the same time would be redundant.

If you have an easy or straightforward "load order" and you want to keep it simple, use the "TES5Merged Patch" from FNVEdit (xEdit). A guide to this is the gromulos article linked above and the wiki article Load order and you.

If you want more control over what will exactly be merged (take the time to study the Bashed Tags and comprehend what they do), then go for the "Bashed Patch" from Wrye Bash/Flash. See the Wrye Bash Pictorial Guide.

With a complicated load order with many merged files, either tailor a "TES5Merged Patch" file to eliminate the conflicts, or create one manual "Override.esp" to correct or add things the Bashed Patch didn't take into account or "get right" for your purposes. This uses the same FNVEdit technique as creating a "Merged Patch".

For example let's say you have three mods that alter weapons (i.e. two ESM's and one ESP), the merged patch strictly follows your load order and the modifications made by the ESP will win as it is loaded after your ESMs. In case of the Bashed Patch you can bypass that load order limitation by assigning Bash Tags to the loaded mods you wish to win and it will import the things you selected, so it is more configurable than the Merged Patch.

But let's say you want the weapons stats from the first ESM, the "on dead" effects from the second (like the beautiful energy weapons criticals from EVE) and the weapon mods from the third, and "bash tags" aren't giving you the result you want. Perhaps a "TES5Merged Patch" file is enough to give the BP an easy solution. But if not, then you have to create a manual "Override patch" placed after the "Bashed Patch" to do just that. (By copying the weapons into a blank ("new") ESP file that is loaded after the "Bashed Patch" and copying the things needed from the three mods in this example.) Same thing for character appearances: Lings Coiffure and Project Mikoto's mods can be made to work together this way for example, by selecting the NPC records you want from both. The possibilities are virtually endless, provided you are willing to put the time and effort into it.

TL;DR Summary

Even the "Bashed Patch" with "bash tags" can only do "so much".

For more in depth on "bash tags" (their role and how to determine which to use), see the wiki article Bash Tags and the Wrye Bash Patch.

Another example that will hopefully help to make it clearer: Suppose you have many mods active that alter NPC appearances. Unfortunately if you "bash tag" all these files with "npcfaces" the last loaded will win and all NPCs it alters will have the appearance they have in that last mod. However, if you prefer the appearance for particular NPCs only that are in a mod that's higher in your load order, do "copy as deep override" with 'xEdit' on all NPCs you want to have a different appearance into your custom override file and load it after the BP to override the BP's changes you don't want. So the custom override file has many masters, but the BP is not one of them.

References

Nexus wiki articles referred to by this article:



Nexus wiki articles that refer to this article: