Difference between revisions of "Setting up Database"

From Nexus Mods Wiki
Jump to: navigation, search
(9 intermediate revisions by 3 users not shown)
Line 9: Line 9:
  
 
If you do not have acces to the pgAdmin you can run this via command line using ''psql.exe'' (comes with PostgreSQL package) with the following commands. 
 
If you do not have acces to the pgAdmin you can run this via command line using ''psql.exe'' (comes with PostgreSQL package) with the following commands. 
<div style="background:#000; border:1px solid #cccccc; padding:5px 10px"><tt>CREATE DATABASE ConfigDB;<br/> CREATE EXTENSION "uuid-ossp";</tt></div>  
+
<div style="background:#000; border:1px solid #cccccc; padding:5px 10px"><tt>CREATE DATABASE ConfigDB;<br/> CREATE EXTENSION "uuid-ossp";</tt></div>  
 
Note that creating your database through the psql tool will result in stripping out the case sensitivity (ConfigDB becomes configdb). The mod tools do not appear to require precise case for the database name,&nbsp; but other SQL tools (such as running psql from the command line in the example below) might.
 
Note that creating your database through the psql tool will result in stripping out the case sensitivity (ConfigDB becomes configdb). The mod tools do not appear to require precise case for the database name,&nbsp; but other SQL tools (such as running psql from the command line in the example below) might.
  
After&nbsp;completing first step we use&nbsp;''psql.exe&nbsp;''&nbsp;to restore the database from provided dumps.
+
After&nbsp;completing first step we use&nbsp;''psql.exe or cmd'' to restore the database from provided dumps.
<div style="background:#000; border:1px solid #cccccc; padding:5px 10px">
+
<div style="background:#000; border:1px solid #cccccc; padding:5px 10px">
 
<tt><PostgreSQL Installation Path>/bin/psql.exe -f <KCD Installation Path>/Data_reference/modding.sql -U&nbsp;postgres –d&nbsp;ConfigDB -q&nbsp;</tt>
 
<tt><PostgreSQL Installation Path>/bin/psql.exe -f <KCD Installation Path>/Data_reference/modding.sql -U&nbsp;postgres –d&nbsp;ConfigDB -q&nbsp;</tt>
  
Line 21: Line 21:
  
 
&nbsp;
 
&nbsp;
 +
 +
&nbsp;
 +
 +
&nbsp;
 +
  
 
== Autoincrement ==
 
== Autoincrement ==
  
Some tables use autoincrement to generate IDs for new entries. If you are going to create new dialogues, you are going to need to adjust the initial value for this autoincrement to a value unique for your mod. If you don’t, your mod will clash with any other mods that also failed to set a unique value. To do this, open file <tt>Data_references.sql/sequences.sql</tt> and change the value 10578 to any value unique to your mod (ideally a value that is far enough from other mod’s values, by at least several hundreds. Since the first ids are used by KCD’s entries,choose a number higher than 100000).Then run following command:&nbsp;
+
<span style="background:#ff9900">WARNING: This process was changed completely in version 2 of the modding tools. Make sure you have the new sequences.sql file</span>
<div style="background:#000;  border:1px solid #cccccc;  padding:5px 10px"><tt><PostgreSQL Installation Path>/bin/psql.exe -f <KCD Installation Path>/Data_reference/sequences.sql -U postgres –d ConfigDB –q&nbsp;</tt></div>  
+
 
&nbsp;
+
Some tables use autoincrement to generate IDs for new entries. If you are going to create new dialogues, you are going to need to adjust the initial value for this autoincrement to a value unique for your mod. If you don’t, your mod will clash with any other mods that also failed to set a unique value. To do this, pick an offset ([//forums.nexusmods.com/index.php?/topic/8510843-register-unique-modding-ids-20/ ''unofficial registry of IDs for modders'']), then run following command (replace X with your selected offset):&nbsp;
 +
<div style="background:#000;  border:1px solid #cccccc;  padding:5px 10px"><tt><PostgreSQL Installation Path>/bin/psql.exe -f <KCD Installation Path>/Data_reference/sequences.sql -U postgres –d ConfigDB -v sequenceoffset=X –q</tt></div>
 +
When using PostgreSQL 12 or newer, you will get an error message about "adsrc" column. To fix this error, open sequences.sql in text editor and replace all "adsrc" with "pg_get_expr(ad.adbin, ad.adrelid)", then save file and re-execute command.
 +
 
 +
== (Optional) Inspecting&nbsp;the Database ==
 +
 
 +
You can check yourself whether the database has been set up properly by looking at the&nbsp;<tt><q>START</q></tt>&nbsp;values of the sequences in the&nbsp;<tt><q>ConfigDB</q></tt>&nbsp;database. The values in the&nbsp;<tt><q>modding</q></tt>&nbsp;schema should either be equal to values in&nbsp;the cmd output of the first&nbsp;''psql.exe&nbsp;''command above or a sum of one of these&nbsp;cmd output values and your offset. Similarly, the value in the&nbsp;<tt><q>public</q></tt>&nbsp;schema should be equal to the&nbsp;value&nbsp;in&nbsp;the cmd output of the second&nbsp;''psql.exe&nbsp;''command above.
 +
 
 +
https://staticdelivery.nexusmods.com/images/2298/47833563-1584964381.png
 +
 
 +
<tt><q>START</q></tt>&nbsp;value = cmd output (20) + offset (69000)
  
 
== Setting up tools&nbsp; ==
 
== Setting up tools&nbsp; ==
Line 42: Line 57:
 
&nbsp;
 
&nbsp;
  
 +
== (Tip) Closing&nbsp;the Database ==
 +
 +
Don't fear the browser warning below, your setup is automatically saved and the browser can safely be closed without having to redo the database setup. You have to remember your password though.
 +
 +
https://staticdelivery.nexusmods.com/images/2298/47833563-1584964929.png
 +
 +
&nbsp;
  
 
== TL;DR ==
 
== TL;DR ==
Line 50: Line 72:
 
#Create&nbsp;''ConfigDB''&nbsp;database.  
 
#Create&nbsp;''ConfigDB''&nbsp;database.  
 
#Add&nbsp;''uuid-ossp&nbsp;''extension to that database.  
 
#Add&nbsp;''uuid-ossp&nbsp;''extension to that database.  
#Edit ''Data_reference/sequences.sql&nbsp;''number 10578&nbsp;to something higher than 100000.  
+
#Edit ''Data_reference/sequences.sql&nbsp;''number 10578&nbsp;to&nbsp;[//forums.nexusmods.com/index.php?/topic/8510843-register-unique-modding-ids-20/ the ID you claimed].  
 
#Migrate files&nbsp;<br/> <br/> <tt>psql.exe -f&nbsp;Data_reference/modding.sql -U&nbsp;postgres –d&nbsp;ConfigDB -q</tt><br/> <tt>psql.exe -f&nbsp;Data_reference/public.sql -U&nbsp;postgres –d&nbsp;ConfigDB -q</tt><br/> <tt>psql.exe -f Data_reference/sequences.sql -U postgres –d ConfigDB –q&nbsp;</tt><br/> &nbsp;  
 
#Migrate files&nbsp;<br/> <br/> <tt>psql.exe -f&nbsp;Data_reference/modding.sql -U&nbsp;postgres –d&nbsp;ConfigDB -q</tt><br/> <tt>psql.exe -f&nbsp;Data_reference/public.sql -U&nbsp;postgres –d&nbsp;ConfigDB -q</tt><br/> <tt>psql.exe -f Data_reference/sequences.sql -U postgres –d ConfigDB –q&nbsp;</tt><br/> &nbsp;  
 
#Configure your&nbsp;''user.cfg''&nbsp;in root directory<br/> <br/> wh_sys_SQLServer = "127.0.0.1"<br/> wh_sys_SQLDatabase = "ConfigDB"<br/> wh_sys_SQLUser = "postgres"<br/> wh_sys_SQLPassword = "'''YOUR DATABASE PASSWORD'''"<br/> wh_sys_SQLBranch = "modding"<br/> wh_sys_PreferredDB = SQL, TBL, XML<br/> wh_sys_SQLWritable = 1  
 
#Configure your&nbsp;''user.cfg''&nbsp;in root directory<br/> <br/> wh_sys_SQLServer = "127.0.0.1"<br/> wh_sys_SQLDatabase = "ConfigDB"<br/> wh_sys_SQLUser = "postgres"<br/> wh_sys_SQLPassword = "'''YOUR DATABASE PASSWORD'''"<br/> wh_sys_SQLBranch = "modding"<br/> wh_sys_PreferredDB = SQL, TBL, XML<br/> wh_sys_SQLWritable = 1  

Revision as of 15:37, 19 February 2021

Database

The Game database contains all the game data, such as items, dialogues, perks, etc. The modding pack contains a dump of the game database in SQL format. In this tutorial we are going to use PostreSQL hosted locally on your computer, which the modding tools will connect to. 

First download and install PostreSQL from their website. (Do not install version 12, as this might cause a missing column "adsrc" error when you later run the sequences.sql script) You will also want to install the 64-bit PosgreSQL ODBC driver, which you can get from https://odbc.postgresql.org/. After install the database server should be up and running. Then we open pgAdmin (comes with the PostgreSQL package) tool and connect to the server. After connecting to the server we have to create a new database called ConfigDB (the name is unfortunatley hardcoded into some of the tools). When created add new uuid-ossp extention to this database.

Setting_Up_Database_image1.jpg

If you do not have acces to the pgAdmin you can run this via command line using psql.exe (comes with PostgreSQL package) with the following commands. 

CREATE DATABASE ConfigDB;
CREATE EXTENSION "uuid-ossp";

Note that creating your database through the psql tool will result in stripping out the case sensitivity (ConfigDB becomes configdb). The mod tools do not appear to require precise case for the database name,  but other SQL tools (such as running psql from the command line in the example below) might.

After completing first step we use psql.exe or cmd to restore the database from provided dumps.

<PostgreSQL Installation Path>/bin/psql.exe -f <KCD Installation Path>/Data_reference/modding.sql -U postgres –d ConfigDB -q 

<PostgreSQL Installation Path>/bin/psql.exe -f <KCD Installation Path>/Data_reference/public.sql -U postgres –d ConfigDB -q 

You will be asked for your database password again. Ideally this command should run with no output of any kind (the -q switch hides all non-errors, and there should be no errors). In your database, there should be now a schema called modding with 468 tables.  

 

 

 


Autoincrement

WARNING: This process was changed completely in version 2 of the modding tools. Make sure you have the new sequences.sql file

Some tables use autoincrement to generate IDs for new entries. If you are going to create new dialogues, you are going to need to adjust the initial value for this autoincrement to a value unique for your mod. If you don’t, your mod will clash with any other mods that also failed to set a unique value. To do this, pick an offset (unofficial registry of IDs for modders), then run following command (replace X with your selected offset): 

<PostgreSQL Installation Path>/bin/psql.exe -f <KCD Installation Path>/Data_reference/sequences.sql -U postgres –d ConfigDB -v sequenceoffset=X –q

When using PostgreSQL 12 or newer, you will get an error message about "adsrc" column. To fix this error, open sequences.sql in text editor and replace all "adsrc" with "pg_get_expr(ad.adbin, ad.adrelid)", then save file and re-execute command.

(Optional) Inspecting the Database

You can check yourself whether the database has been set up properly by looking at the START values of the sequences in the ConfigDB database. The values in the modding schema should either be equal to values in the cmd output of the first psql.exe command above or a sum of one of these cmd output values and your offset. Similarly, the value in the public schema should be equal to the value in the cmd output of the second psql.exe command above.

47833563-1584964381.png

START value = cmd output (20) + offset (69000)

Setting up tools 

There are several tools that work with the SQL database. Most of them require you to set up correct registry keys which they read connection information from. To do that, modify file Tools/!registry/db.reg  (you only need to modify the password, postgres is the default user) and then run it. All of the tools, except the Sandbox editor, use these registry keys. To get the Sandbox to connect, fill in correct connection information in user.cfg. Sandbox editor also requires an ODBC Driver to use. Install one for postgreSQL from their site (use 64-bit version). 

 

Working with Database

The database can be edited with any SQL client you prefer, or you can use our custom tool.  It is a plugin for MS Excel, which simplifies editing tables by unpacking foreign keys. To use it, you need to install in from Tools/ExcelDbAddin2, which installs it directly into MS Excel (it can be removed at any time in “Add or Remove programs”). It will appear as a new menu item called WHS DB

Any changes made to the database will be visible in the Sandbox editor (with the exception of Localization). The game, however, reads the database from XML files inside Tables.pak or mod PAK files. To create such PAKs, use the editor’s modding window .

 

(Tip) Closing the Database

Don't fear the browser warning below, your setup is automatically saved and the browser can safely be closed without having to redo the database setup. You have to remember your password though.

47833563-1584964929.png

 

TL;DR

  1. Install PostgreSQL server.
  2. Install ODBC driver (64bit) for PostgreSQL.
  3. Open pgAdmin, it's installed with PostgreSQL package.
  4. Create ConfigDB database.
  5. Add uuid-ossp extension to that database.
  6. Edit Data_reference/sequences.sql number 10578 to the ID you claimed.
  7. Migrate files 

    psql.exe -f Data_reference/modding.sql -U postgres –d ConfigDB -q
    psql.exe -f Data_reference/public.sql -U postgres –d ConfigDB -q
    psql.exe -f Data_reference/sequences.sql -U postgres –d ConfigDB –q 
     
  8. Configure your user.cfg in root directory

    wh_sys_SQLServer = "127.0.0.1"
    wh_sys_SQLDatabase = "ConfigDB"
    wh_sys_SQLUser = "postgres"
    wh_sys_SQLPassword = "YOUR DATABASE PASSWORD"
    wh_sys_SQLBranch = "modding"
    wh_sys_PreferredDB = SQL, TBL, XML
    wh_sys_SQLWritable = 1
  9.  
  10. Done, start editor.