Changes

== Overall process ==

== Overall process ==

When setting up the environment for the first time or ForgeGradle determines the environment is out of data (changed access transformers, mappings, or forge version), a gradle refresh triggers three things:

Whenever a gradle refresh is triggered, a few things occur:

# ForgeGradle downloads the jar and MCPConfig zip and triggers the extractSrg/createSrgToMcp task.

# ForgeGradle downloads the jar and MCPConfig zip and triggers the extractSrg/createSrgToMcp task.

# After that, it processes the jar - applies access transformers from forge/userdev, SAS from forge, and decompiles.

# After that, it processes the jar - applies access transformers from forge/userdev, SAS from forge, and decompiles.

# Finally, it patches and finalizes the code, ready for modder consumption.

# Finally, it patches and finalizes the code, ready for modder consumption.

== Needed knowledge ==

== Needed knowledge ==

=== Obfuscation ===

=== Obfuscation ===

'''Obfuscation''' is the process of renaming all of the fields, methods, and classes of the compiled code into unreadable, machine-generated names (such as <code>aaa</code>, <code>bC</code>), and removing package structures (this makes everything package-local and makes the obfuscated code smaller somewhat. This is commonly used by companies to prevent external entities from easily decompiling their released binaries/executables and retrieving their source code/intellectual property, though it does have size advantages.

'''Obfuscation''' is the process of renaming all of the fields, methods, and classes of the compiled code into unreadable, machine-generated names (such as <code>aaa</code>, <code>bC</code>), and removing package structures (this makes everything package-local and makes the obfuscated code smaller somewhat. This is commonly used by companies to prevent external entities from easily decompiling their released binaries/executables and retrieving their source code/intellectual property.

Additionally, due to the way the Local Variable Table (LVT) of Java bytecode is stored, every function-local variable name is turned to <code>☃</code> (that's right, a snowman) in the compiled files. This makes immediate recompilation of the game literally and physically impossible, as every Java compiler currently available requires that local variables have unique names.

Additionally, due to the way the Local Variable Table (LVT) of Java bytecode is stored, every function-local variable name is turned to <code>☃</code> (that's right, a snowman) in the compiled files. This makes immediate recompilation of the game literally and physically impossible, as every Java compiler currently available requires that local variables have unique names.

The SRG name of the member is then derived from its SRG ID and its type (method {given the prefix <code>m_</code>}, field {given the prefix <code>f_</code>}, or parameter {given the prefix <code>p_</code>}). <ref>The SRG name for a given member is only created once, when it first appears in the code.</ref>. This inclusion of the SRG ID into the name guarantees that the SRG name for all members are unique, and is the reason the ID is generated.

The SRG name of the member is then derived from its SRG ID and its type (method {given the prefix <code>m_</code>}, field {given the prefix <code>f_</code>}, or parameter {given the prefix <code>p_</code>}). <ref>The SRG name for a given member is only created once, when it first appears in the code.</ref>. This inclusion of the SRG ID into the name guarantees that the SRG name for all members are unique, and is the reason the ID is generated.

The actual conversion of obf names to SRG names is done by a tool called [[Toolchain:Vignette|Vignette]]. More information on how it works can be found on that page.

The actual conversion of obf names to SRG names is done by a tool called [[Toolchain#Vignette|Vignette]]. More information on how it works can be found on that page.

== The Setup ==

== The Setup ==

The process can be broken up into 3 steps; MCPConfig, patch and provide.

The process can be broken up into 3 steps; MCPConfig, patch and provide.

The MCPConfig step is, understandably, the biggest and most prone to failure.

The MCPConfig step is, understandably, the biggest and most prone to failure.

An explanation of MCPConfig itself, how it works, what it's for (but NOT how to use it) can be found [[Toolchain:MCPConfig|here]]. For the purpose of this guide, you need only know that its' goal is to get the game decompiled, and into a state where it can immediately be recompiled. Due to certain flaws in the rest of the toolchain, this means it needs to fix and patch the source code before passing it onto Forge.

An explanation of MCPConfig itself, how it works, what it's for (but NOT how to use it) can be found [[Toolchain:MCPConfig|here]]. For the purpose of this guide, you need only know that its' goal is to get the game decompiled, and into a state where it can immediately be recompiled. This means it needs to fix and patch the source code before passing it onto Forge.

In this way, MCPConfig can be thought of the vanilla side of the setup. It does not modify the game.

In this way, MCPConfig can be thought of the vanilla side of the setup. It does not modify the game.

** java -jar &lt;version&gt; &lt;args&gt; &lt;jvmArgs&gt;

** java -jar &lt;version&gt; &lt;args&gt; &lt;jvmArgs&gt;

An example config.json can be found [https://github.com/MinecraftForge/MCPConfig/blob/master/versions/release/1.17.1/config.json here].

{{Tip|content=A config.json can be found [https://github.com/MinecraftForge/MCPConfig/blob/master/versions/release/1.17.1/config.json here] which defines the following steps:

 

It defines the steps:

* [[Toolchain:ForgeFlower|fernflower]]

* [[Toolchain:ForgeFlower|fernflower]]

** version: net.minecraftforge:forgeflower:1.5.498.12

** version: net.minecraftforge:forgeflower:1.5.498.12

** args: --jar-in {input} --jar-out {output} --mapping-format tsrg2 --mappings {mappings} --fernflower-meta --cfg {libraries} --create-inits --fix-param-annotations

** args: --jar-in {input} --jar-out {output} --mapping-format tsrg2 --mappings {mappings} --fernflower-meta --cfg {libraries} --create-inits --fix-param-annotations

More information about each of these tools can be found at the link provided, as well as what each of these arguments do. A brief description is provided.

More information about each of these tools can be found at the link provided, as well as what each of these arguments do. A brief description is provided.}}

=== ForgeFlower ===  

=== ForgeFlower ===  

The decompiler used by ForgeGradle is a custom fork of [https://github.com/JetBrains/intellij-community/tree/master/plugins/java-decompiler/engine Jetbrains' FernFlower], called [https://github.com/MinecraftForge/ForgeFlower ForgeFlower].

The decompiler used by ForgeGradle is a custom fork of [https://github.com/JetBrains/intellij-community/tree/master/plugins/java-decompiler/engine Jetbrains' FernFlower], called [https://github.com/MinecraftForge/ForgeFlower ForgeFlower] which searches the jar for files, cleans up the bytecode, and then converts it into a reasonable best-guess interpretation.

 

It simply searches the jar for files, cleans up the bytecode, and then converts it into a reasonable best-guess interpretation.

It:

It also produces the following side effects:

* Removes synthetic parameters from constructors

* Removes synthetic parameters from constructors

** In bytecode, inner classes have the outer class as their first constructor parameter, but Java source code does not.

** In bytecode, inner classes have the outer class as their first constructor parameter, but Java source code does not.

=== Mergetool ===

=== Mergetool ===

The game is split into two distributions; server and client.

The game is split into two distributions; server and client. Since the server is just a subset of the client, the client contains the server-only classes as well.

 

Because the server contains no rendering code, and the client contains none of the server-specific code (like the UI), this means there are differences between what can run on one side or the other. Since the server is just a subset of the client, the client contains the server-only classes as well.

To get around this, we have a tool called Mergetool, which can search for the differences between two files (down to the function level) and merge them into one large (referred to as joined) jar file. It also can also annotate those files as necessary.

To get around this, we have a tool called Mergetool, which can search for the differences between two files (down to the function level) and merge them into one large (referred to as joined) jar file. It also can also annotate those files as necessary.

Vignette is where SRG starts to come into play. It serves the role of our deobfuscator, performing deobfuscation.

Vignette is where SRG starts to come into play. It serves the role of our deobfuscator, performing deobfuscation.

This process is done with the help of a '''deobfusation map''', a file generated by the original obfuscator (in this case, ProGuard) that contains a map of the obfuscated names to original, non-obfuscated names. This is commonly used on debofuscating stack traces outputted by an obfuscated program, for debugging purposes.<ref name="retrace"/>

This process is done with the help of a '''deobfusation map''', a file generated by the original obfuscator (in this case, ProGuard) that contains a map of the obfuscated names to original, non-obfuscated names. This is commonly used on deobfuscating stack traces outputted by an obfuscated program, for debugging purposes.<ref name="retrace"/>

We have three sets of deobfuscation maps available to us; the obf->SRG mappings distributed with the MCPConfig system, the Yarn intermediary system, or the official mappings.<ref name="mojmappings"/>.

We have three sets of deobfuscation maps available to us; the obf->SRG mappings distributed with the MCPConfig system, the Yarn intermediary system, or the official mappings.<ref name="mojmappings"/>.

=== Patches ===

=== Patches ===

Once we have the source code ready to go, the final step in the setup is to apply patches.

Once we have the source code ready to go, the final step in the setup is to apply patches.

These are done trivially using [https://github.com/MinecraftForge/DiffPatch DiffPatch].

These are done trivially using [https://github.com/MinecraftForge/DiffPatch DiffPatch].

== The Forge Side ==

== The Forge Side ==

After the MCPConfig/SetupMCP tasks are finished, ForgeGradle will print '''MCP Environment Setup is complete''' and wait for the user input.

When Forge is first loaded by gradle, it prepares the files for setting up the workspace.

The next step is to run the <code>gradlew setup</code> task, which does what it implies: Applying the Forge system to the processed vanilla code.

The initial setup is done through the <code>gradlew setup</code> task. It will run the SetupMCP tasks and then start applying the Forge system to the processed vanilla code. After the MCPConfig/SetupMCP tasks are finished, ForgeGradle will print '''MCP Environment Setup is complete''' and continue. The text '''pausing after requested step''' is also printed twice during this task such that some necessary data can be extracted.

First, it applies ATs. After that, patches. Finally, the official mappings are applied.

First, it applies ATs. After that, patches. Finally, the official mappings, or some other mappings from a provider like Parchment, are applied.

All in all, compared to the MCPConfig setup, this is a string of extremely basic tasks - mostly just one-line commands.

All in all, compared to the MCPConfig setup, this is a string of extremely basic tasks - mostly just one-line commands.

* <code>AccessTransformers.jar --inJar {input} --outJar {output} --logFile accesstransform.log --atFile {at.cfg}</code>

* <code>AccessTransformers.jar --inJar {input} --outJar {output} --logFile accesstransform.log --atFile {at.cfg}</code>

[https://github.com/MinecraftForge/MinecraftForge/blob/1.17.x/src/main/resources/META-INF/accesstransformer.cfg The current AT cfg can be found here.]

[https://github.com/MinecraftForge/MinecraftForge/blob/1.17.x/src/main/resources/META-INF/accesstransformer.cfg The AT cfg for Forge can be found here.]

=== Applying Patches ===

=== Applying Patches ===

The patches used for Forge itself at install time are different from those used by MCPConfig, which means there are two separate patching stages performed.

The patches used for Forge itself are different from those used by MCPConfig, which means there are two separate patching stages performed.

As opposed to the minimal MCPConfig patching, with the goal to make the code recompilable, the Forge patching is done to apply the API and mod loader to the code.

As opposed to the minimal MCPConfig patching, with the goal to make the code recompilable, the Forge patching is done to apply the API and mod loader to the code.

It does this in a very similar way, with the [https://github.com/MinecraftForge/BinaryPatcher BinaryPatcher] utility.

It does this in a very similar way, with the [https://github.com/MinecraftForge/BinaryPatcher BinaryPatcher] utility used at install time/during a CI build or [https://github.com/MinecraftForge/DiffPatch DiffPatch] in all other situations.