Forge Community Wiki - User contributions [en-gb]

Codecs

2023-11-23T17:22:38Z

Commoble: /* Registry Dispatch */

Codecs are a serialization tool from mojang's DataFixerUpper library. Codecs are used alongside [[DynamicOps|DynamicOps]] to allow objects to be serialized to different formats and back, such as JSON or [[Using_NBT|NBT]]. While the DynamicOps describes the format the object is to be serialized to, the Codec describes the manner in which the object is to be serialized; a single Codec can be used to serialize an object to any format for which a DynamicOps exists. Codecs and DynamicOps jointly form an abstraction layer over data serialization, simplifying the effort needed to serialize or deserialize data.

= Using Codecs =

== Serialization and Deserialization ==

The primary use for Codecs is to serialize java objects to some serialized type, such as a JsonElement or a Tag, and to deserialize an serialized object back to its proper java type. This is accomplished with <code>Codec#encodeStart</code> and <code>Codec#parse</code>, respectively. Given a Codec<SomeJavaType> and a DynamicOps<SomeSerializedType>, we can convert instances of SomeJavaType to instances of SomeSerializedType and back.

Each of these methods take a [[DynamicOps]] instance and an instance of the object we are serializing or deserializing, and returns a DataResult:

<syntaxhighlight lang="java">
// let someCodec be a Codec<SomeJavaType>
// let someJavaObject be an instance of SomeJavaType
// let someTag and someJsonElement be instances of Tag and JsonElement, respectively

// serialize some java object to Tag
DataResult<Tag> result = someCodec.encodeStart(NBTOps.INSTANCE, someJavaObject);

// deserialize some Tag instance back to a proper java object
DataResult<SomeJavaType> result = someCodec.parse(NBTOps.INSTANCE, someTag );

// serialize some java object to a JsonElement
DataResult<JsonElement> result = someCodec.encodeStart(JsonOps.INSTANCE, someJavaObject);

// deserialize a JsonElement back to a proper java object
DataResult<SomeJavaType> result = someCodec.parse(JsonOps.INSTANCE, someJsonElement);
</syntaxhighlight>

A DataResult either holds the converted instance, or it holds some error data, depending on whether the conversion was successful or not, respectively. There are several things we can do with this DataResult; <code>DataResult#result</code> simply returns an Optional containing the converted object if the conversion was successful, while <code>DataResult#resultOrPartial</code> also runs a given function if the conversion was unsuccessful (in addition to returning the Optional). #resultOrPartial is particularly useful for logging errors during datapack deserialization:

<syntaxhighlight lang="java">
// deserialize something from json
someCodec.parse(JsonOps.INSTANCE, someJsonElement)
.resultOrPartial(errorMessage -> doSomethingIfBadData(errorMessage))
.ifPresent(someJavaObject -> doSomethingIfGoodData(someJavaObject))
</syntaxhighlight>

== Builtin Codecs ==
=== Primitive Codecs ===
The Codec class itself contains static instances of codecs for all supported primitive types, e.g. <code>Codec.STRING</code> is the canonical <code>Codec<String></code> implementation. Primitive codecs include:
* BOOL, which serializes to a boolean value
* BYTE, SHORT, INT, LONG, FLOAT, and DOUBLE, which serialize to numeric values
* STRING, which serializes to a string
* BYTE_BUFFER, INT_STREAM, and LONG_STREAM, which serialize to lists of numbers
* EMPTY, which represents null objects

=== Other Builtin Codecs ===
Vanilla minecraft has many builtin codecs for objects that it frequently serializes. These codecs are typically static instances in the class the codec is serializing; e.g. <code>ResourceLocation.CODEC</code> is the canonical <code>Codec<ResourceLocation></code>, while <code>BlockPos.CODEC</code> is the codec used for serializing a BlockPos.

Each vanilla <code>Registry</code> acts as the Codec for the type of object the registry contains; e.g. <code>Registry.BLOCK</code> is itself a <code>Codec<Block></code>. Forge Registries, however, do not currently implement Codec and cannot yet be used in this way; custom codecs must be created for forge-specific registries that are not tied to specific vanilla registries.

Of particular note here is the CompoundTag.CODEC, which can be used to e.g. serialize a CompoundTag into a json file. This has a notable limitation in that CompoundTag.CODEC *cannot* safely deserialize lists of numbers from json, due to the strong typing of ListTag and the way that the NBTOps deserializer reads numeric values.

= Creating Codecs =

Suppose we have the following class, and we want to deserialize json files to instances of this class:
<syntaxhighlight lang="java">
public class ExampleCodecClass {

private final int someInt;
private final Item item;
private final List<BlockPos> blockPositions;

public ExampleCodecClass(int someInt, Item item, List<BlockPos> blockPositions) {...}

public int getSomeInt() { return this.someInt; }
public Item getItem() { return this.item; }
public List<BlockPos> getBlockPositions() { return this.blockPositions; }
}
</syntaxhighlight>

Where a json file for an instance of this class might look like:

<syntaxhighlight lang="json">
{
"some_int": 42,
"item": "minecraft:gold_ingot",
"block_positions":
[
[0,0,0],
[10,20,-100]
]
}
</syntaxhighlight>

We can assemble a codec for this class by building a new codec out of smaller codecs. We'll need a codec for each of these fields:
* a <code>Codec<Integer></code>
* a <code>Codec<Item></code>
* a <code>Codec<List<BlockPos>></code>
And then we'll need to assemble these into a <code>Codec<ExampleCodecClass></code>.

As previously mentioned, we can use <code>Codec.INT</code> for the integer codec, and <code>Registry.ITEM</code> for the Item codec. We don't have a builtin codec for list-of-blockpos, but we can use BlockPos.CODEC to create one.

== Lists ==
The <code>Codec#listOf</code> instance method can be used to generate a codec for a List from an existing codec:

<syntaxhighlight lang="java">
// BlockPos.CODEC is a Codec<BlockPos>
Codec<List<BlockPos>> = BlockPos.CODEC.listOf();
</syntaxhighlight>

Codecs created via listOf() serialize things to listlike objects, such as [] json arrays or ListTags.

Deserializing a list in this manner produces an ''immutable'' list. If a mutable list is needed, [[Codecs#Equivalent_Types_and_xmap|xmap]] can be used to convert the list after deserializing.

== Records ==
RecordCodecBuilder is used to generate codecs that serialize instances of classes with explicitly named fields, like our example above. Codecs created via RecordCodecBuilder serialize things to maplike objects, such as {} json objects or CompoundTags.

RecordCodecBuilder can be used in several ways, but the simplest form is as follows:

<syntaxhighlight lang="java">
public static final Codec<SomeJavaClass> = RecordCodecBuilder.create(instance -> instance.group(
someFieldCodecA.fieldOf("field_name_a").forGetter(SomeJavaClass::getFieldA),
someFieldCodecB.fieldOf("field_name_b").forGetter(SomeJavaClass::getFieldB),
someFieldCodecC.fieldOf("field_name_c").forGetter(SomeJavaClass::getFieldC),
// up to 16 fields can be declared here
).apply(instance, SomeJavaClass::new));
</syntaxhighlight>

Where each line in the group specifies a codec instance for the type of that field, the field name in the serialized object, and the corresponding getter function in the java class. The builder is concluded by specifying a constructor or factory for the java class whose arguments are the previously defined fields in the same order.

For example, using RecordCodecBuilder to create a codec for our example class above:
<syntaxhighlight lang="java">
public static final Codec<ExampleCodecClass> = RecordCodecBuilder.create(instance -> instance.group(
Codec.INT.fieldOf("some_int").forGetter(ExampleCodecClass::getSomeInt),
Registry.ITEM.fieldOf("item").forGetter(ExampleCodecClass::getItem),
BlockPos.CODEC.listOf().fieldOf("block_positions").forGetter(ExampleCodecClass::getBlockPositions)
).apply(instance, ExampleCodecClass::new));
</syntaxhighlight>

===Optional and Default Values in Record Fields===
When RecordCodecBuilder is used as shown above, all of the fields are *required* to be in the serialized object (the JsonObject/CompoundTag/etc), or the entire thing will fail to parse when the codec tries to deserialize it. If we wish to have optional or default values, we have several alternatives of fieldOf() we can use.

* <code>someCodec.optionalFieldOf("field_name")</code> creates a field for an Optional. If the field in the json/nbt is not present or invalid, it will deserialize as an empty optional. Empty optionals will not be serialized; the field will be omitted from the json or nbt.
* <code>someCodec.optionalFieldOf("field_name", someDefaultValue)</code> creates an optional field that deserializes as the given default value if the field is not present in the json/nbt. When serializing, if the field in the java object equals the default value, the value will not be serialized and the field will be omitted from the json or nbt.

When using optional fields, be wary that if the field contains bad data or otherwise fails to serialize, the error will be silently caught, and the field will serialize as the default value instead!

===Boxing values as objects===
In some situations, we may need to serialize a single value as a single-field object. We can use fieldOf to box a single value in this way without needing the entire RecordCodecBuilder process:

<syntaxhighlight lang="java">
public static final Codec<Integer> BOXED_INT_CODEC = Codec.INT.fieldOf("value").codec();

JsonElement value = BOXED_INT_CODEC.encodeStart(JsonOps.INSTANCE, 5).result().get();
</syntaxhighlight>

Which serializes the following output:

<syntaxhighlight lang="json">
{"value":5}
</syntaxhighlight>

==Unit==
The <code>Codec.unit(defaultValue)</code> codec creates a Codec that always deserializes a specified default value, regardless of input. When serializing, it serializes nothing.

==Pair==
The <code>Codec.pair(codecA, codecB)</code> static method takes two codecs and generates a Codec<Pair<A,B>> from them.

The only valid arguments for this method are codecs that serialize to objects with explicit fields, such as codecs created using [[Codecs#Records|RecordCodecBuilder]] or [[Codecs#Boxing_values_as_objects|fieldOf]]. Codecs that serialize nothing (such as [[Codecs#Unit|unit codecs]]) are also valid as they act as objects-with-no-fields.

The resulting Pair codec will serialize a single object that has all of the fields of the two original codecs. For example:
<syntaxhighlight lang="java">
public static final Codec<Pair<Integer,String>> PAIR_CODEC = Codec.pair(
Codec.INT.fieldOf("value").codec(),
Codec.STRING.fieldOf("name").codec());

JsonElement encodedPair = PAIR_CODEC.encodeStart(JsonOps.INSTANCE, Pair.of(5, "cheese").result().get();
</syntaxhighlight>
This codec serializes the above value to:
<syntaxhighlight lang="json">
{
"value": 5,
"name": "cheese"
}
</syntaxhighlight>

Codecs that serialize to objects with undefined fields such as [[Codecs#Maps|unboundedMap]] may cause strange and unpredictable behaviour when used here; these objects should be boxed via fieldOf when used in a pair codec.

==Either==
The <code>Codec.either(codecA, codecB)</code> static method takes two codecs and generates a Codec<Either<A,B>> from them.

When this codec is used to de/serialize an object, it first attempts to use the first codec; if and only if that conversion fails, it then attempts to use the second codec. If that conversion also fails, then the returned DataResult will contain the error data from the *second* codec's conversion attempt.

==Numeric Ranges==
The <code>Codec.intRange(min,max)</code>, <code>Codec.floatRange(min,max)</code>, and <code>Codec.doubleRange(min,max)</code> static methods generate Codecs for Integers, Floats, or Doubles, respectively, for which only a specified inclusive range is valid, and values outside that range will fail to de/serialize.

==Maps==
Suppose we want to serialize a HashMap or other Map type, where we could have indefinitely many key-value pairs and we don't know what the keys are ahead of time.

We can create a <code>Codec<Map<KEY,VALUE>></code> using the <code>Codec.unboundedMap</code> static method, which takes a key codec and a value codec and creates a codec for a map type:

<syntaxhighlight lang="java">
public static final Codec<Map<String, BlockPos>> = Codec.unboundedMap(Codec.STRING, BlockPos.CODEC);
</syntaxhighlight>

The serialized form of maps serialized by this codec will be a JsonObject or CompoundTag, whose fields are the key-value pairs in the map; the map's keys will be used as the field names, and the map's values will be the values of those fields

A limitation of using unboundedMap is that it only supports key codecs that serialize to Strings (including codecs for things like ResourceLocation that aren't Strings themselves but still serialize to strings). To create a codec for a Map whose keys are not fundamentally strings, the Map must be serialized as a list of key-value pairs instead of using unboundedMap.

==Equivalent Types and xmap==
Suppose we have two java classes, Amalgam and Box; any Amalgam instance can be converted to a Box, and vice-versa. Now suppose we have a Codec<Amalgam>, but we'd also like to have a Codec<Box>. Rather than creating an entirely new codec for Box from scratch, we can simply xmap our Amalgam codec instead.

The <code>Codec#xmap</code> instance method is used to generate a second codec for a fundamentally equivalent type to the first codec's type. The method takes two function objects as arguments, which are used to convert the first type to the second when deserializing, and converting the second type to the first when serializing:

<syntaxhighlight lang="java">
public static final Codec<Box> = Amalgam.CODEC.xmap(Amalgam::toBox, Box::toAmalgam);
</syntaxhighlight>

Codecs created in this manner will serialize objects in the same format as the starting codec.

==Partially Equivalent Types, flatComapMap, comapFlatMap, and flatXMap==
Consider the ResourceLocation: Any ResourceLocation can be converted to a String, but not all Strings can be converted to a ResourceLocation; ResourceLocations have strict limits on their format and allowed characters.

While we *could* use xmap to convert the Codec.STRING to a codec for ResourceLocations, this would cause attempts to parse an invalid string like <code>SHOUTY:MOD:Invalid$Characters</code> to throw a runtime exception, when we really should be returning a failed DataResult to the parser instead -- which indeed is what the vanilla ResourceLocation codec does.

Codecs have three additional instance methods for creating equivalent codecs for when we can *potentially* convert one type to another, but are not guaranteed to be able to do so. These take conversion function arguments that return DataResults, allowing validation to be performed during serialization and deserialization.

{| class="wikitable"
|+ Codec Conversion Methods
|-
! Can A always be converted to B? !! Can B always be converted to A? !! Which method of codecA should be used to create codecB?
|-
| yes || yes || codecA.xmap
|-
| yes || no || codecA.flatComapMap
|-
| no || yes || codecA.comapFlatMap
|-
| no || no || codecA.flatXmap
|}

== Registry Dispatch ==
Registry Dispatch Codecs allow us to define a registry of codecs and delegate to a specific codec to deserialize a particular json based on a type field in that json. Dispatch codecs are used extensively when deserializing worldgen data.

To create a dispatch codec for a Thing class, the following steps can be performed:
# Create a Thing abstract class class and ThingType interface. The ThingType interface should have a method that supplies a Codec<Thing>, while Thing subclasses must define a method that supplies a ThingType.
# Create a map or registry of ThingTypes, and register a ThingType for each sub-codec we want to have.
# Create a Codec<ThingType>, or have the ThingType registry implement Codec.
# Create our Codec<Thing> master codec by invoking <code>Codec#dispatch</code> on our ThingType codec. This method's arguments are:
## A field name for the ID of the sub-codec (the example json below is using "type")
## A function to retrieve a ThingType from a Thing
## A function to retrieve a Codec<Thing> from a ThingType

We can then use our Codec<Thing> to create Thing fields in other codecs whose serialized format depends on the specific sub-codec used by a Thing instance.

As an example of this, consider the ExampleCodecClass earlier. Suppose we make this class extend Thing and register our codec for it to a codec dispatch registry with the id "ourmod:exampleclass". If we were to define an instance of this class in a Thing field in some json, it would look like

<syntaxhighlight lang="json">
"some_thing":
{
"type": "ourmod:exampleclass",
"some_int": 42,
"item": "minecraft:gold_ingot",
"block_positions":
[
[0,0,0],
[10,20,-100]
]
}
</syntaxhighlight>

Other ThingTypes we register would have different fields in this json object, but would still be valid for the "some_thing" field.

Several examples of vanilla classes that use dispatch codecs:
* RuleTest and RuleTestType
* BlockPlacer and BlockPlacerType
* ConfiguredDecorator and FeatureDecorator

=== Registering MapCodecCodecs for Dispatch Subcodecs ===

When registering a subcodec to any dispatch codec registry, the registered subcodec should be an instance of MapCodecCodec, or the subcodec will be nested in its own object when serialized.

For example, suppose we register a single-field subcodec, where we use fieldOf-and-xmap to convert an int to our int-holding Thing:

<syntaxhighlight lang="java">
public record Thing(int n){}
public static Codec<Thing> CODEC = Codec.INT.fieldOf("n").codec().xmap(Thing::new, Thing::n);
</syntaxhighlight>

This results in this json when serialized:

<syntaxhighlight lang="json">
"some_thing":
{
"type": "ourmod:thing",
"value": {
"n": 5
}
}
</syntaxhighlight>

This occurs because xmap does not produce a MapCodecCodec, and if this nested object is not desired, then our registered subcodec must be a MapCodecCodec.

However, fieldOf() produces a MapCodec, which has a codec() method, which does produce a MapCodecCodec. We can rearrange our codec builder, which then produces a cleaner json:

<syntaxhighlight lang="java">
public static Codec<Thing> CODEC = Codec.INT // Primitive codec
.fieldOf("n") // MapCodec
.xmap(Thing::new, Thing::n) // MapCodec
.codec(); // MapCodecCodec! That's what we want.
</syntaxhighlight>

<syntaxhighlight lang="json">
"some_thing":
{
"type": "ourmod:thing",
"n": 5
}
</syntaxhighlight>

RecordCodecBuilder also produces MapCodecCodecs.

=External Links=
* [https://github.com/Mojang/DataFixerUpper/blob/master/src/main/java/com/mojang/serialization/Codec.java Codecs in Mojang's official public DataFixerUpper repository]
* [https://kvverti.github.io/Documented-DataFixerUpper/snapshot/com/mojang/serialization/Codec.html#flatXmap-java.util.function.Function-java.util.function.Function- Unofficial Codec Javadocs]

Biome Modifiers

2022-09-04T17:43:12Z

Commoble: Add warning about adding feature with multiple biome modifiers to best practices

Biome Modifiers are a data-driven system for modifying biomes. They have the ability to modify biomes in several ways:

* Adding or removing worldgen features and carvers
* Adding or removing mob spawns and mob spawn costs
* Modifying the climate of a biome
* Modifying a biome's client effects, such as water color

<big>'''NOTE:'''</big> Forge provides several [[#Builtin_Biome_Modifier_Types|builtin biome modifier types]], so some basic use cases such as adding features or mob spawns to biomes can be done without registering additional serializers. If you are looking to add a mob or feature to a biome, scroll down to the [[#Builtin_Biome_Modifier_Types|Builtin Biome Modifier Types]] section and use one of those pre-existing Biome Modifiers from Forge instead. No custom Biome Modifier implementation needed.

Creating and using biome modifiers involves up to three steps:

# Creating a [[#Biome_Modifier_Type|biome modifier type]], which defines how to modify a biome.
# Registering a [[#Biome_Modifier_Serializers|biome modifier codec]], which defines how to parse a json into your biome modifier type.
# Creating [[#Biome_Modifier_JSONs|biome modifier JSONs]] to define individual biome modifier instances; each json file provides a glob of data to your serializer to produce an instance of a biome modifier type. These can be [[#Datageneration|datagenerated]] if desired.

=Biome Modifier Types=
To define a new type of biome modifier, begin by implementing a new class that extends <code>BiomeModifier</code>. BiomeModifiers can usually be implemented as records, to reduce boilerplate.

<syntaxhighlight lang="java">
public record ExampleBiomeModifier() implements BiomeModifier
{
public void modify(Holder<Biome> biome, Phase phase, Builder builder)
{
// This allows modifications to the given biome via the provided Builder.
}

public Codec<? extends BiomeModifier> codec()
{
// This must return a registered Codec, see Biome Modifier Serializers below.
}
}
</syntaxhighlight>

Beware! The modify method is called once per each phase per biome per biome modifier, so it's important to check which phase you're in and only make your changes in one phase.

<syntaxhighlight lang="java">
public void modify(Holder<Biome> biome, Phase phase, Builder builder)
{
if (phase == Phase.ADD)
{
// add things to biomes
}
}
</syntaxhighlight>

Typically we also want to restrict biome modifiers to only apply to certain biomes. We can do that by accepting a HolderSet<biome> in our constructor:

<syntaxhighlight lang="java">
public record ExampleBiomeModifier(HolderSet<Biome> biomes) implements BiomeModifier
{
public void modify(Holder<Biome> biome, Phase phase, Builder builder)
{
if (phase == Phase.ADD && biomes.contains(biome))
{
// add things to biomes
}
}
</syntaxhighlight>

We can also accept [[Holders]] or [[HolderSets|HolderSets]] for other [[Registration#Data_Driven_Entries|datapack registry elements]], such as PlacedFeatures, which allows our BiomeModifier to refer to those elements, which can then be defined in their own JSON files.

<syntaxhighlight lang="java">
public record ExampleBiomeModifier(HolderSet<Biome> biomes, Holder<PlacedFeature> feature) implements BiomeModifier
{
public void modify(Holder<Biome> biome, Phase phase, Builder builder)
{
// add a feature to all specified biomes
if (phase == Phase.ADD && biomes.contains(biome))
{
builder.getGenerationSettings().addFeature(Decoration.UNDERGROUND_ORES, feature);
}
}
</syntaxhighlight></biome>

= Biome Modifier Serializers =

Each type of biome modifier must have a [[Codecs|Codec]] registered for it; [[Registration#DeferredRegister|Deferred Registers]] are the recommended way to register biome modifier codecs.

<syntaxhighlight lang="java">
public class YourMod
{
static DeferredRegister<Codec<? extends BiomeModifier>> BIOME_MODIFIER_SERIALIZERS =
DeferredRegister.create(ForgeRegistries.Keys.BIOME_MODIFIER_SERIALIZERS, "yourmodid");

static RegistryObject<Codec<ExampleBiomeModifier>> EXAMPLE_CODEC = BIOME_MODIFIER_SERIALIZERS.register("example", () ->
RecordCodecBuilder.create(builder -> builder.group(
// declare fields
Biome.LIST_CODEC.fieldOf("biomes").forGetter(ExampleBiomeModifier::biomes),
PlacedFeature.CODEC.fieldOf("feature").forGetter(ExampleBiomeModifier::feature)
// declare constructor
).apply(builder, ExampleBiomeModifier::new)));
}
</syntaxhighlight>

We can now implement the codec() method in our biome modifier class:

<syntaxhighlight lang="java">
public record ExampleBiomeModifier(HolderSet<Biome> biomes, Holder<PlacedFeature> feature) implements BiomeModifier
{
public void modify(Holder<Biome> biome, Phase phase, Builder builder)
{
// add a feature to all specified biomes
if (phase == Phase.ADD && biomes.contains(biome))
{
builder.getGenerationSettings().addFeature(Decoration.UNDERGROUND_ORES, feature);
}
}

public Codec<? extends BiomeModifier> codec()
{
return YourMod.EXAMPLE_CODEC.get();
}
}
</syntaxhighlight>

= Biome Modifier JSONs =

Once we've registered a codec for our biome modifier type, we can define jsons for it. Biome modifier jsons must be defined in the directory <code>data/modid/forge/biome_modifier/</code>; a biome modifier at <code>data/modid/forge/biome_modifier/your_biome_modifier.json</code> has the namespaced id <code>modid:your_biome_modifier</code>.

Our codec above defines the json format:

* "biomes" is a special HolderSet field that accepts a single biome id, [list of biome ids], or #biome_tag.
* "feature" accepts a single placed feature id. [https://minecraft.fandom.com/wiki/Placed_feature These can be defined in jsons as well].
* We must also specify "type": "yourmodid:example" to tell the data loader to use our registered codec to read our json.

A json instance of our biome modifier that adds some feature to badlands biomes might look like this:

<syntaxhighlight lang="json">
{
"type": "yourmodid:example",
"biomes": "#minecraft:is_badlands",
"feature": "yourmod:some_feature"
}
</syntaxhighlight>

= Datageneration =

Biome Modifier jsons can be [[Datageneration|datagenerated via GatherDataEvent]]. As biome modifiers are datapack registry objects, this can be done by using JsonCodecProvider#forDatapackRegistry as the data provider. Refer to [[Datageneration/Datapack_Registries]] for additional information on datagenerating datapack registry elements.

= Builtin Biome Modifier Types =

Forge provides the following builtin biome modifier types:

== None ==

A no-op biome modifier type, whose jsons have the following format:

<syntaxhighlight lang="json">
{
"type": "forge:none"
}
</syntaxhighlight>

This allows pack devs or server operators to disable mods' biome modifiers by overriding their biome modifier jsons with the above.

== Add Features ==

This biome modifier type adds placed features to biomes.

<syntaxhighlight lang="json">
{
"type": "forge:add_features", // required
"biomes": "#namespace:your_biome_tag", // accepts a biome id, [list of biome ids], or #namespace:biome_tag
"features": "namespace:your_feature", // accepts a placed feature id, [list of placed feature ids], or #namespace:feature_tag
"step": "underground_ores" // accepts a Decoration enum name
}
</syntaxhighlight>

Decoration steps in order of generation are:
* raw_generation
* lakes
* local_modifications
* underground_structures
* surface_structures
* underground_ores
* underground_decoration
* fluid_springs
* vegetal_decoration
* top_layer_modification

== Remove Features ==

This biome modifier type removes features from biomes.

<syntaxhighlight lang="json">
{
"type": "forge:remove_features", // required
"biomes": "#namespace:your_biome_tag", // accepts a biome id, [list of biome ids], or #namespace:biome_tag
"features": "namespace:your_feature", // accepts a placed feature id, [list of placed feature ids], or #namespace:feature_tag
"steps": "underground_ores" // optional field specifying a Decoration or list of Decorations to remove features from, defaults to all if not specified
}
</syntaxhighlight>

== Add Spawns ==

This biome modifier type adds mob spawns to biomes.

<syntaxhighlight lang="json">
{
"type": "forge:add_spawns", // Required
"biomes": "#namespace:biome_tag", // Accepts a biome id, [list of biome ids], or #namespace:biome_tag
"spawners":
{
"type": "namespace:entity_type", // Type of mob to spawn
"weight": 100, // int, spawn weighting
"minCount": 1, // int, minimum pack size
"maxCount": 4 // int, maximum pack size
}
}
</syntaxhighlight>

== Remove Spawns ==

This biome modifier type removes mob spawns from biomes.

<syntaxhighlight lang="json">
{
"type": "forge:remove_spawns", // Required
"biomes": "#namespace:biome_tag", // Accepts a biome id, [list of biome ids], or #namespace:biome_tag
"entity_types": "#namespace:entitytype_tag" // Accepts an entity type, list, or tag of entitytypes whose spawns are to be removed from the biomes
}
</syntaxhighlight>

= Best Practices =

* Avoid using biome modifiers to add vanilla placed features to biomes, as this may cause a feature cycle violation (the game will crash if two biomes have the same two features in their feature lists but in different orders). Placed features can be referenced in biome jsons or added via biome modifiers, but should not be used in both.
* Avoid adding the same placed feature with more than one biome modifier, as this can cause feature cycle violations.

Holders

2022-08-13T16:44:31Z

Commoble: fix italics markup

Holders are a component of vanilla registries representing a registrable value and/or the identifier of that value; they vaguely resemble a vanilla implementation of RegistryObjects, but with substantial semantic and implementation differences. They are used extensively by [[Tags|tags]] and [[Registration#Data_Driven_Entries|datapack registries]]; modders that use these systems should be aware of their functions and caveats.

Datapack registry elements can have reference to HolderSets, which can use tag references to define sets of registry elements.

== Holders ==

=== Properties of Holders ===

Holders in general have the following properties:
* A holder may or may not have a key (a ResourceLocation/ResourceKey identifying the holder/value by name)
* A holder may or may not have a value (e.g. a block or a biome)
* A holder may or may not belong to a specific registry instance
* A holder may or may not be a mutable object
* A holder is associated with zero or more tags

=== Types of Holder ===

There are three categories of holders: direct holders, standalone reference holders, and intrusive reference holders.

==== Direct Holders ====

Direct holders always hold an unregistered value, and never hold a key. Direct holders are generally created when a datapack registry json inline-defines a value in a holder field instead of declaring a reference to another json. Direct holders are immutable records, and never have tags bound to them.

A vanilla example of direct holders being used is in density functions; density functions are often composed of holders of other density functions, but the child functions themselves don't need to be registered, and so many vanilla density functions are created with inline/direct child functions.

Direct holders should not be created for values that must always exist in the registry, or for values whose registry names must be determined at some point.

==== Reference Holders ====

Reference holders refer to registered values; they are created with either a key or a value, with the other property being bound to them later in the registration process. Reference holders belong to a specific registry instance and are aware of which registry they belong to (if multiple copies of a registry exist, as is the case for datapack registries, a reference holder is only valid for a single instance of that registry).

Reference holders are mutable and may have tags bound to them.

===== Standalone Reference Holders =====

Most holders encountered will be standalone reference holders; they are created with a key, and have a value bound to them later. Each registry has at most one standalone reference holder per key; parsing a reference holder in a datapack registry json computes a holder in the relevant registry if it doesn't already exist.

Standalone reference holders are the means by which datapack registry jsons can refer to each other. They have the following lifecycle:

* A datapack registry json is loaded that refers to another registrable by id, e.g. "minecraft:desert"
* A standalone reference holder is retrieved from the relevant registry via a get-or-create operation
* When a datapack registry json is loaded and fully parsed, a holder for that json is get-or-created ''and'' the parsed value is bound to it
* When registries freeze, the get-or-create operation can no longer create new holders. If any reference holders in the registry are not fully bound at this time (with both a key and a value), an error is raised. This will occur if a datapack registry json refers by id to another datapack registry json that does not exist, and is how these references-by-id are validated.
* Whenever tags load, each tag file's TagKey is bound to all reference holders referred to in that tag (this modifiers the holders in-place).

When datagenerating datapack registry jsons with reference holders, any reference holders ''must'' be created by the specific registry instances used by the RegistryOps/RegistryAccess used to datagen the jsons, or they will fail to serialize.

===== Intrusive Reference Holders =====

Intrusive reference holders are created with a value and have a key bound to them later. Several static registrable types (blocks, items, fluids, entitytypes, and gameevents) create intrusive holders of themselves when constructed, which have a value bound to them once the value is registered.

Intrusive holders and the means of creating them are deprecated by mojang; this seems to be a hack to allow blocks and friends to quickly look up their registry name. Intrusive holders may be removed in future minecraft releases, and should not be used by mods (value->key lookups can be done via a method in the relevant registry, or by creating standalone holders or RegistryObjects to create name-value pairs).

== HolderSets ==

A HolderSet is an indexable set of holders, and is the preferred means by which datapack registrables can refer to other datapack registrables. Vanilla frequently uses holdersets to wire worldgen jsons together, e.g. a structure json has a holderset field for which biomes the structure can spawn in.

Vanilla has two types of holdersets and three json formats; forge expands the holderset serializer to allow additional types of holdersets. A holderset field in a json can accept a single element, a list of elements, a tag ID, or one of forge's special formats, making datapack creation very flexible when holderset fields are used.

<syntaxhighlight lang="json">
{
"a_biome": "desert", // single element holderset
"some_biomes": ["plains", "forest"], // list holderset
"biome_tag": "#is_plains", // tag holderset
"expanded_forge_format": // more on these later
{
"type": "modid:custom_holderset_type",
// additional fields as specified by the type
}
}
</syntaxhighlight>

[[Codecs]] can be created for holdersets to allow holdersets to be read from jsons; however, a holderset codec must be created for each registry holdersets can be made for, e.g. Biome.LIST_OF_LISTS_CODEC is the codec for HolderSet<Biome>.

HolderSets should generally only be serialized when used as components of unsynced (server-only) datapack registry jsons, such as in worldgen features. HolderSets that are serialized without RegistryOps for registry context (including when synced to clients in synced datapack registries, which do not serialize with registry context) will be deserialized as lists of unregistered objects.

=== Types of HolderSets ===

==== Direct HolderSets ====

Direct HolderSets represent an immutable list of holders. They can be defined in json as single elements or lists of elements (the single element format simply parses as a list holderset with one element in it).

<syntaxhighlight lang="json">
{
"a_biome": "desert", // single element holderset
"some_biomes": ["plains", "forest"], // list holderset
}
</syntaxhighlight>

==== Named HolderSets ====

Named HolderSets represent a tag. Named HolderSets are mutable; they cache a list and a set of holders, and this cache is reset each time tags reload.

<syntaxhighlight lang="json">
{
"biome_tag": "#is_plains", // tag holderset
}
</syntaxhighlight>

==== Custom HolderSet Types ====

Forge expands the holderset codecs to allow additional json formats. Custom holderset serializers can be registered by creating a deferred register for ForgeRegistries.Keys.HOLDER_SET_TYPES, though the builtin types provided by forge should be sufficient for most use cases.

====Builtin Custom HolderSet Types====
Forge provides four builtin holderset types, allowing for additional set operations and representations in datapack registry jsons. The <code>and</code>, <code>or</code>,
and <code>not</code> types are composed of other holdersets, and are therefore potentially mutable as they may be composed of mutable tag holdersets whose values are recalculated after tags reload.
=====Any=====
The <code>forge:any</code> holderset type represents the set of all elements of the relevant registry.

<syntaxhighlight lang="json">
{
"biomes":
{
"type": "forge:any"
}
}
</syntaxhighlight>

=====And=====
The <code>forge:and</code> type represents an intersection of other holdersets.

<syntaxhighlight lang="json">
{
"biomes":
{
"type": "forge:and",
"values":
[
// list of holdersets of any format, different formats (string, list, object) can be mixed here
"#is_plains",
"#is_overworld"
]
}
}
</syntaxhighlight>
=====Or=====
The <code>forge:or</code> type represents a union of other holdersets.

<syntaxhighlight lang="json">
{
"biomes":
{
"type": "forge:or",
"values":
[
// list of holdersets of any format, different formats (string, list, object) can be mixed here
"#is_plains",
["swamp", "mangrove_swamp"]
]
}
}
</syntaxhighlight>
=====Not=====
The <code>forge:not</code> type represents the set of all elements in the registry that do not belong to the specified holderset.

<syntaxhighlight lang="json">
{
"biomes":
{
"type": "forge:not",
"value": "#is_plains" // holderset of any format (string, list, or object)
}
}
</syntaxhighlight>

Main Page

2022-08-13T15:41:03Z

Commoble: Add link to holders

__NOTOC__
<templatestyles src=":Main_Page/styles.css" />

<div class="center title">Forge Community Wiki</div>

The Forge Community Wiki exists so that the community can:

* collectively keep track of major changes and updates to Forge and Vanilla code;
* create and edit articles with in-depth explanations about a variety of Forge-related subjects; and
* contribute example code and tutorials for both simple and difficult concepts.

This wiki is editable by any registered user with an account. This is to allow tracking of harmful edits, but isn't imposing any annoying limits. We welcome any edit, however small. Join the [https://discord.gg/Nn42eAh Discord] to discuss changes and edits to the wiki with others and the staff.

{{Supported versions}}

<div class="center round_warning">
Please read the [[FCWMeta:Wiki Policy|wiki policy]] before editing!
</div>

<div class="center" style="margin-bottom: 2em">
{{Supported versions|text=1}}
</div>

<div class="box_container center">

<div class="box">
<h3>Beginner Topics</h3>
* [[Getting Started]]
* [[Proper Mod Structuring]]
* [[Mods.toml file]]
* [[Debug Profiler|The Debug Profiler]]
* [[Version Checker]]
</div>

<div class="box">
<h3>Advanced Topics</h3>
* [[Jar-in-jar]]
</div>

<div class="box">
<h3>Common Concepts</h3>
* [[Sides|Understanding Sides]]
* [[Events|Understanding Events]]
* [[Registration]]
* [[Internationalization]]
* [[Configs]]
</div>

<div class="box">
<h3>Forge Conventions</h3>
* [[Semantic Versioning]]
* [[Stages of Modloading]]
</div>

<div class="box">
<h3>Resources and Data</h3>
* [[Using Resources|Introduction]]
* [[Recipes]]
* [[Tags]]
* [[Holders]]
</div>

<div class="box">
<h3>Blocks</h3>
* [[Making Blocks|Creating Blocks]]
* [[Understanding Blockstates]]
* [[Interacting With Blocks|Block Interactions]]
</div>

<div class="box">
<h3>Items</h3>
* [[Making Items]]
* [[Making Tools]]
* [[BlockEntityWithoutLevelRenderer|<tt>BlockEntityWithoutLevelRenderer</tt>]]
</div>

<div class="box">
<h3>Data Generation</h3>
* [[Datageneration|Introduction]]
* [[Datageneration/Recipes|Recipes]]
* [[Datageneration/Tags|Tags]]
* [[Datageneration/Loot_Tables|Loot Tables]]
* [[Datageneration/I18n|Localization]]
* [[Datageneration/States and Models|<tt>BlockState</tt>s and Models]]
* [[Datageneration/Datapack_Registries|Datapack Registries]]
</div>

<div class="box">
<h3>Block Entities</h3>
* [[Block Entities|Introduction]]
* [[Block Entity Renderer|<tt>BlockEntityRenderer</tt>]]
</div>

<div class="box">
<h3>Models</h3>
* [[Introduction to Models|Introduction]]
* [[Model JSONs]]
* [[BlockState JSONs]]
* [[Coloring textures dynamically|Dynamically Colored Textures]]
* [[Item Properties]]
</div>

<div class="box">
<h3>Handling Information</h3>
* [[Understanding Networking|Introduction]]
* [[Using NBT|Named Binary Tag (NBT)]]
* [[Using FriendlyByteBuf|Using <tt>FriendlyByteBuf</tt>]]
* [[Sending Packets|Sending and Receiving Packets]]
* [[Using SimpleChannel|Using <tt>SimpleChannel</tt>]]
* [[Networking with Entities]]
* [[DynamicOps|Using DynamicOps]]
* [[Codecs|Using Codecs]]
</div>

<div class="box">
<h3>Data Storage</h3>
* [[Capabilities|Understanding Capabilities]]
* [[Capabilities/Attaching|Attaching Capabilities]]
* [[Saved Data]]
</div>

<div class="box">
<h3>Game Effects</h3>
* [[Mob Effects]]
* [[Potions]]
* [[Particles]]
* [[Sounds]]
</div>

<div class="box">
<h3>Events</h3>
* [[Events|Understanding Events]]
* [[Entity Events]]
</div>

<div class="box">
<h3>Others</h3>
* [[Dependencies]]
* [[Dynamic Loot Modification]]
* [[Components|Components and Translation Keys]]
* [[Key Mappings]]
* [[Access Transformers]]
* [[Toolchain]]
* [[Game_Tests|Game Tests]]
* [[Biome_Modifiers|Biome Modifiers]]
</div>

<div class="box">
<h3>Recipes</h3>
* [[Recipes]]
* [[Custom Recipes|Custom Recipe Types]]
* [[Datageneration/Recipes|Datageneration]]
</div>

</div>

Holders

2022-08-13T15:40:36Z

Commoble: Created page with "Holders are a component of vanilla registries representing a registrable value and/or the identifier of that value; they vaguely resemble a vanilla implementation of RegistryO..."

Holders are a component of vanilla registries representing a registrable value and/or the identifier of that value; they vaguely resemble a vanilla implementation of RegistryObjects, but with substantial semantic and implementation differences. They are used extensively by [[Tags|tags]] and [[Registration#Data_Driven_Entries|datapack registries]]; modders that use these systems should be aware of their functions and caveats.

Datapack registry elements can have reference to HolderSets, which can use tag references to define sets of registry elements.

== Holders ==

=== Properties of Holders ===

Holders in general have the following properties:
* A holder may or may not have a key (a ResourceLocation/ResourceKey identifying the holder/value by name)
* A holder may or may not have a value (e.g. a block or a biome)
* A holder may or may not belong to a specific registry instance
* A holder may or may not be a mutable object
* A holder is associated with zero or more tags

=== Types of Holder ===

There are three categories of holders: direct holders, standalone reference holders, and intrusive reference holders.

==== Direct Holders ====

Direct holders always hold an unregistered value, and never hold a key. Direct holders are generally created when a datapack registry json inline-defines a value in a holder field instead of declaring a reference to another json. Direct holders are immutable records, and never have tags bound to them.

A vanilla example of direct holders being used is in density functions; density functions are often composed of holders of other density functions, but the child functions themselves don't need to be registered, and so many vanilla density functions are created with inline/direct child functions.

Direct holders should not be created for values that must always exist in the registry, or for values whose registry names must be determined at some point.

==== Reference Holders ====

Reference holders refer to registered values; they are created with either a key or a value, with the other property being bound to them later in the registration process. Reference holders belong to a specific registry instance and are aware of which registry they belong to (if multiple copies of a registry exist, as is the case for datapack registries, a reference holder is only valid for a single instance of that registry).

Reference holders are mutable and may have tags bound to them.

===== Standalone Reference Holders =====

Most holders encountered will be standalone reference holders; they are created with a key, and have a value bound to them later. Each registry has at most one standalone reference holder per key; parsing a reference holder in a datapack registry json computes a holder in the relevant registry if it doesn't already exist.

Standalone reference holders are the means by which datapack registry jsons can refer to each other. They have the following lifecycle:

* A datapack registry json is loaded that refers to another registrable by id, e.g. "minecraft:desert"
* A standalone reference holder is retrieved from the relevant registry via a get-or-create operation
* When a datapack registry json is loaded and fully parsed, a holder for that json is get-or-created *and* the parsed value is bound to it
* When registries freeze, the get-or-create operation can no longer create new holders. If any reference holders in the registry are not fully bound at this time (with both a key and a value), an error is raised. This will occur if a datapack registry json refers by id to another datapack registry json that does not exist, and is how these references-by-id are validated.
* Whenever tags load, each tag file's TagKey is bound to all reference holders referred to in that tag (this modifiers the holders in-place).

When datagenerating datapack registry jsons with reference holders, any reference holders *must* be created by the specific registry instances used by the RegistryOps/RegistryAccess used to datagen the jsons, or they will fail to serialize.

===== Intrusive Reference Holders =====

Intrusive reference holders are created with a value and have a key bound to them later. Several static registrable types (blocks, items, fluids, entitytypes, and gameevents) create intrusive holders of themselves when constructed, which have a value bound to them once the value is registered.

Intrusive holders and the means of creating them are deprecated by mojang; this seems to be a hack to allow blocks and friends to quickly look up their registry name. Intrusive holders may be removed in future minecraft releases, and should not be used by mods (value->key lookups can be done via a method in the relevant registry, or by creating standalone holders or RegistryObjects to create name-value pairs).

== HolderSets ==

A HolderSet is an indexable set of holders, and is the preferred means by which datapack registrables can refer to other datapack registrables. Vanilla frequently uses holdersets to wire worldgen jsons together, e.g. a structure json has a holderset field for which biomes the structure can spawn in.

Vanilla has two types of holdersets and three json formats; forge expands the holderset serializer to allow additional types of holdersets. A holderset field in a json can accept a single element, a list of elements, a tag ID, or one of forge's special formats, making datapack creation very flexible when holderset fields are used.

<syntaxhighlight lang="json">
{
"a_biome": "desert", // single element holderset
"some_biomes": ["plains", "forest"], // list holderset
"biome_tag": "#is_plains", // tag holderset
"expanded_forge_format": // more on these later
{
"type": "modid:custom_holderset_type",
// additional fields as specified by the type
}
}
</syntaxhighlight>

[[Codecs]] can be created for holdersets to allow holdersets to be read from jsons; however, a holderset codec must be created for each registry holdersets can be made for, e.g. Biome.LIST_OF_LISTS_CODEC is the codec for HolderSet<Biome>.

HolderSets should generally only be serialized when used as components of unsynced (server-only) datapack registry jsons, such as in worldgen features. HolderSets that are serialized without RegistryOps for registry context (including when synced to clients in synced datapack registries, which do not serialize with registry context) will be deserialized as lists of unregistered objects.

=== Types of HolderSets ===

==== Direct HolderSets ====

Direct HolderSets represent an immutable list of holders. They can be defined in json as single elements or lists of elements (the single element format simply parses as a list holderset with one element in it).

<syntaxhighlight lang="json">
{
"a_biome": "desert", // single element holderset
"some_biomes": ["plains", "forest"], // list holderset
}
</syntaxhighlight>

==== Named HolderSets ====

Named HolderSets represent a tag. Named HolderSets are mutable; they cache a list and a set of holders, and this cache is reset each time tags reload.

<syntaxhighlight lang="json">
{
"biome_tag": "#is_plains", // tag holderset
}
</syntaxhighlight>

==== Custom HolderSet Types ====

Forge expands the holderset codecs to allow additional json formats. Custom holderset serializers can be registered by creating a deferred register for ForgeRegistries.Keys.HOLDER_SET_TYPES, though the builtin types provided by forge should be sufficient for most use cases.

==== Builtin Custom HolderSet Types ====

Forge provides four builtin holderset types, allowing for additional set operations and representations in datapack registry jsons. The and, or, and not types are composed of other holdersets, and are therefore potentially mutable as they may be composed of mutable tag holdersets whose values are recalculated after tags reload.

===== Any =====

The <code>forge:any</code> holderset type represents the set of all elements of the relevant registry.

<syntaxhighlight lang="json">
{
"biomes":
{
"type": "forge:any"
}
}
</syntaxhighlight>

===== And =====

The <code>forge:and</code> type represents an intersection of other holdersets.

<syntaxhighlight lang="json">
{
"biomes":
{
"type": "forge:and",
"values":
[
// list of holdersets of any format, different formats (string, list, object) can be mixed here
"#is_plains",
"#is_overworld"
]
}
}
</syntaxhighlight>

===== Or =====

The <code>forge:or</code> type represents a union of other holdersets.

<syntaxhighlight lang="json">
{
"biomes":
{
"type": "forge:or",
"values":
[
// list of holdersets of any format, different formats (string, list, object) can be mixed here
"#is_plains",
["swamp", "mangrove_swamp"]
]
}
}
</syntaxhighlight>

===== Not =====

The <code>forge:not</code> type represents the set of all elements in the registry that do not belong to the specified holderset.

<syntaxhighlight lang="json">
{
"biomes":
{
"type": "forge:not",
"value": "#is_plains" // holderset of any format (string, list, or object)
}
}
</syntaxhighlight>

Biome Modifiers

2022-06-15T23:53:30Z

Commoble: Add summary

Biome Modifiers are a data-driven system for modifying biomes. They have the ability to modify biomes in several ways:

* Adding or removing worldgen features and carvers
* Adding or removing mob spawns and mob spawn costs
* Modifying the climate of a biome
* Modifying a biome's client effects, such as water color

Creating and using biome modifiers involves up to three steps:

# Creating a [[#Biome_Modifier_Type|biome modifier type]], which defines how to modify a biome.
# Registering a [[#Biome_Modifier_Serializers|biome modifier codec]], which defines how to parse a json into your biome modifier type.
# Creating [[#Biome_Modifier_JSONs|biome modifier JSONs]] to define individual biome modifier instances; each json file provides a glob of data to your serializer to produce an instance of a biome modifier type. These can be [[#Datageneration|datagenerated]] if desired.

Forge provides several [[#Builtin_Biome_Modifier_Types|builtin biome modifier types]], so some basic use cases such as adding features or mob spawns to biomes can be done without registering additional serializers.

= Biome Modifier Types =

To define a new type of biome modifier, begin by implementing a new class that extends <code>BiomeModifier</code>. BiomeModifiers can usually be implemented as records, to reduce boilerplate.

<syntaxhighlight lang="java">
public record ExampleBiomeModifier() implements BiomeModifier
{
public void modify(Holder<Biome> biome, Phase phase, Builder builder)
{
// This allows modifications to the given biome via the provided Builder.
}

public Codec<? extends BiomeModifier> codec()
{
// This must return a registered Codec, see Biome Modifier Serializers below.
}
}
</syntaxhighlight>

Beware! The modify method is called once per each phase per biome per biome modifier, so it's important to check which phase you're in and only make your changes in one phase.

<syntaxhighlight lang="java">
public void modify(Holder<Biome> biome, Phase phase, Builder builder)
{
if (phase == Phase.ADD)
{
// add things to biomes
}
}
</syntaxhighlight>

Typically we also want to restrict biome modifiers to only apply to certain biomes. We can do that by accepting a HolderSet<Biome> in our constructor:

<syntaxhighlight lang="java">
public record ExampleBiomeModifier(HolderSet<Biome> biomes) implements BiomeModifier
{
public void modify(Holder<Biome> biome, Phase phase, Builder builder)
{
if (phase == Phase.ADD && biomes.contains(biome))
{
// add things to biomes
}
}
</syntaxhighlight>

We can also accept Holders or HolderSets for other [[Registration#Data_Driven_Entries|datapack registry elements]], such as PlacedFeatures, which allows our BiomeModifier to refer to those elements, which can then be defined in their own JSON files.

<syntaxhighlight lang="java">
public record ExampleBiomeModifier(HolderSet<Biome> biomes, Holder<PlacedFeature> feature) implements BiomeModifier
{
public void modify(Holder<Biome> biome, Phase phase, Builder builder)
{
// add a feature to all specified biomes
if (phase == Phase.ADD && biomes.contains(biome))
{
builder.getGenerationSettings().addFeature(Decoration.UNDERGROUND_ORES, feature);
}
}
</syntaxhighlight>

= Biome Modifier Serializers =

Each type of biome modifier must have a [[Codecs|Codec]] registered for it; [[Registration#DeferredRegister|Deferred Registers]] are the recommended way to register biome modifier codecs.

<syntaxhighlight lang="java">
public class YourMod
{
static DeferredRegister<Codec<? extends BiomeModifier>> BIOME_MODIFIER_SERIALIZERS =
DeferredRegister.create(ForgeRegistries.Keys.BIOME_MODIFIER_SERIALIZERS, "yourmodid");

static RegistryObject<Codec<ExampleBiomeModifier>> EXAMPLE_CODEC = BIOME_MODIFIER_SERIALIZERS.register("example", () ->
RecordCodecBuilder.create(builder -> builder.group(
// declare fields
Biome.LIST_CODEC.fieldOf("biomes").forGetter(ExampleBiomeModifier::biomes),
PlacedFeature.CODEC.fieldOf("feature").forGetter(ExampleBiomeModifier::feature)
// declare constructor
).apply(builder, ExampleBiomeModifier::new)));
}
</syntaxhighlight>

We can now implement the codec() method in our biome modifier class:

<syntaxhighlight lang="java">
public record ExampleBiomeModifier(HolderSet<Biome> biomes, Holder<PlacedFeature> feature) implements BiomeModifier
{
public void modify(Holder<Biome> biome, Phase phase, Builder builder)
{
// add a feature to all specified biomes
if (phase == Phase.ADD && biomes.contains(biome))
{
builder.getGenerationSettings().addFeature(Decoration.UNDERGROUND_ORES, feature);
}
}

public Codec<? extends BiomeModifier> codec()
{
return YourMod.EXAMPLE_CODEC.get();
}
}
</syntaxhighlight>

= Biome Modifier JSONs =

Once we've registered a codec for our biome modifier type, we can define jsons for it. Biome modifier jsons must be defined in the directory <code>data/modid/forge/biome_modifier/</code>; a biome modifier at <code>data/modid/forge/biome_modifier/your_biome_modifier.json</code> has the namespaced id <code>modid:your_biome_modifier</code>.

Our codec above defines the json format:

* "biomes" is a special HolderSet field that accepts a single biome id, [list of biome ids], or #biome_tag.
* "feature" accepts a single placed feature id. [https://minecraft.fandom.com/wiki/Placed_feature These can be defined in jsons as well].
* We must also specify "type": "yourmodid:example" to tell the data loader to use our registered codec to read our json.

A json instance of our biome modifier that adds some feature to badlands biomes might look like this:

<syntaxhighlight lang="json">
{
"type": "yourmodid:example",
"biomes": "#minecraft:is_badlands",
"feature": "yourmod:some_feature"
}
</syntaxhighlight>

== Side Note on HolderSets ==

HolderSets are a vanilla feature that, when defined in JSON, can be specified as a single id, list of ids, or tag. Our example above uses a holderset of biomes, so all three of these are valid biome fields:

<syntaxhighlight lang="json">
{
"biomes": "forest",
"biomes": ["forest", "birch_forest"],
"biomes": "#is_forest"
}
</syntaxhighlight>

= Datageneration =

Biome Modifier jsons can be [[Datageneration|datagenerated via GatherDataEvent]]. As biome modifiers are datapack registry objects, this can be done by using JsonCodecProvider#forDatapackRegistry as the data provider. Refer to [[Datageneration/Datapack_Registries]] for additional information on datagenerating datapack registry elements.

= Builtin Biome Modifier Types =

Forge provides the following builtin biome modifier types:

== None ==

A no-op biome modifier type, whose jsons have the following format:

<syntaxhighlight lang="json">
{
"type": "forge:none"
}
</syntaxhighlight>

This allows pack devs or server operators to disable mods' biome modifiers by overriding their biome modifier jsons with the above.

== Add Features ==

This biome modifier type adds placed features to biomes.

<syntaxhighlight lang="json">
{
"type": "forge:add_features", // required
"biomes": "#namespace:your_biome_tag" // accepts a biome id, [list of biome ids], or #namespace:biome_tag
"features": "namespace:your_feature", // accepts a placed feature id, [list of placed feature ids], or #namespace:feature_tag
"step": "underground_ores" // accepts a Decoration enum name
}
</syntaxhighlight>

Decoration steps in order of generation are:
* raw_generation
* lakes
* local_modifications
* underground_structures
* surface_structures
* underground_ores
* underground_decoration
* fluid_springs
* vegetal_decoration
* top_layer_modification

== Remove Features ==

This biome modifier type removes features from biomes.

<syntaxhighlight lang="json">
{
"type": "forge:remove_features", // required
"biomes": "#namespace:your_biome_tag" // accepts a biome id, [list of biome ids], or #namespace:biome_tag
"features": "namespace:your_feature", // accepts a placed feature id, [list of placed feature ids], or #namespace:feature_tag
"steps": "underground_ores" // optional field specifying a Decoration or list of Decorations to remove features from, defaults to all if not specified
}
</syntaxhighlight>

== Add Spawns ==

This biome modifier type adds mob spawns to biomes.

<syntaxhighlight lang="json">
{
"type": "forge:add_spawns", // Required
"biomes": "#namespace:biome_tag", // Accepts a biome id, [list of biome ids], or #namespace:biome_tag
"spawners":
{
"type": "namespace:entity_type", // Type of mob to spawn
"weight": 100, // int, spawn weighting
"minCount": 1, // int, minimum pack size
"maxCount": 4, // int, maximum pack size
}
}
</syntaxhighlight>

== Remove Spawns ==

This biome modifier type removes mob spawns from biomes.

<syntaxhighlight lang="json">
{
"type": "forge:remove_spawns", // Required
"biomes": "#namespace:biome_tag", // Accepts a biome id, [list of biome ids], or #namespace:biome_tag
"entity_types": "#namespace:entitytype_tag" // Accepts an entity type, list, or tag of entitytypes whose spawns are to be removed from the biomes
}
</syntaxhighlight>

= Best Practices =

* Avoid using biome modifiers to add vanilla placed features to biomes, as this may cause a feature cycle violation (the game will crash if two biomes have the same two features in their feature lists but in different orders). Placed features can be referenced in biome jsons or added via biome modifiers, but should not be used in both.

Main Page

2022-06-15T01:08:53Z

Commoble: Add link to Datageneration/Datapack_Registries

__NOTOC__
<templatestyles src=":Main_Page/styles.css" />

<div class="center title">Forge Community Wiki</div>

The Forge Community Wiki exists so that the community can:

* collectively keep track of major changes and updates to Forge and Vanilla code;
* create and edit articles with in-depth explanations about a variety of Forge-related subjects; and
* contribute example code and tutorials for both simple and difficult concepts.

This wiki is editable by any registered user with an account. This is to allow tracking of harmful edits, but isn't imposing any annoying limits. We welcome any edit, however small. Join the [https://discord.gg/Nn42eAh Discord] to discuss changes and edits to the wiki with others and the staff.

{{Supported versions}}

<div class="center round_warning">
Please read the [[FCWMeta:Wiki Policy|wiki policy]] before editing!
</div>

<div class="center" style="margin-bottom: 2em">
{{Supported versions|text=1}}
</div>

<div class="box_container center">

<div class="box">
<h3>Beginner Topics</h3>
* [[Getting Started]]
* [[Proper Mod Structuring]]
* [[Mods.toml file]]
* [[Debug Profiler|The Debug Profiler]]
* [[Version Checker]]
</div>

<div class="box">
<h3>Common Concepts</h3>
* [[Sides|Understanding Sides]]
* [[Events|Understanding Events]]
* [[Registration]]
* [[Internationalization]]
* [[Configs]]
</div>

<div class="box">
<h3>Forge Conventions</h3>
* [[Semantic Versioning]]
* [[Stages of Modloading]]
</div>

<div class="box">
<h3>Resources and Data</h3>
* [[Using Resources|Introduction]]
* [[Recipes]]
* [[Tags]]
</div>

<div class="box">
<h3>Blocks</h3>
* [[Making Blocks|Creating Blocks]]
* [[Understanding Blockstates]]
* [[Interacting With Blocks|Block Interactions]]
</div>

<div class="box">
<h3>Items</h3>
* [[Making Items]]
* [[Making Tools]]
* [[BlockEntityWithoutLevelRenderer|<tt>BlockEntityWithoutLevelRenderer</tt>]]
</div>

<div class="box">
<h3>Data Generation</h3>
* [[Datageneration|Introduction]]
* [[Datageneration/Recipes|Recipes]]
* [[Datageneration/Tags|Tags]]
* [[Datageneration/Loot_Tables|Loot Tables]]
* [[Datageneration/I18n|Localization]]
* [[Datageneration/States and Models|<tt>BlockState</tt>s and Models]]
* [[Datageneration/Datapack_Registries|Datapack Registries]]
</div>

<div class="box">
<h3>Block Entities</h3>
* [[Block Entities|Introduction]]
* [[Block Entity Renderer|<tt>BlockEntityRenderer</tt>]]
</div>

<div class="box">
<h3>Models</h3>
* [[Introduction to Models|Introduction]]
* [[Model JSONs]]
* [[BlockState JSONs]]
* [[Coloring textures dynamically|Dynamically Colored Textures]]
* [[Item Properties]]
</div>

<div class="box">
<h3>Handling Information</h3>
* [[Understanding Networking|Introduction]]
* [[Using NBT|Named Binary Tag (NBT)]]
* [[Using FriendlyByteBuf|Using <tt>FriendlyByteBuf</tt>]]
* [[Sending Packets|Sending and Receiving Packets]]
* [[Using SimpleChannel|Using <tt>SimpleChannel</tt>]]
* [[Networking with Entities]]
* [[DynamicOps|Using DynamicOps]]
* [[Codecs|Using Codecs]]
</div>

<div class="box">
<h3>Data Storage</h3>
* [[Capabilities|Understanding Capabilities]]
* [[Capabilities/Attaching|Attaching Capabilities]]
* [[Saved Data]]
</div>

<div class="box">
<h3>Game Effects</h3>
* [[Mob Effects]]
* [[Potions]]
* [[Particles]]
* [[Sounds]]
</div>

<div class="box">
<h3>Events</h3>
* [[Events|Understanding Events]]
* [[Entity Events]]
</div>

<div class="box">
<h3>Others</h3>
* [[Dependencies]]
* [[Dynamic Loot Modification]]
* [[Components|Components and Translation Keys]]
* [[Key Mappings]]
* [[Access Transformers]]
* [[Toolchain]]
* [[Game_Tests|Game Tests]]
* [[Biome_Modifiers|Biome Modifiers]]
</div>

<div class="box">
<h3>Recipes</h3>
* [[Recipes]]
* [[Custom Recipes|Custom Recipe Types]]
* [[Datageneration/Recipes|Datageneration]]
</div>

</div>

Biome Modifiers

2022-06-15T00:59:28Z

Commoble: Add the four new stock biome modifiers and a datagen link

Biome Modifiers are a data-driven system for modifying biomes. They have the ability to modify biomes in several ways:

* Adding or removing worldgen features and carvers
* Adding or removing mob spawns and mob spawn costs
* Modifying the climate of a biome
* Modifying a biome's client effects, such as water color

= Biome Modifier Types =

To define a new type of biome modifier, begin by implementing a new class that extends <code>BiomeModifier</code>. BiomeModifiers can usually be implemented as records, to reduce boilerplate.

<syntaxhighlight lang="java">
public record ExampleBiomeModifier() implements BiomeModifier
{
public void modify(Holder<Biome> biome, Phase phase, Builder builder)
{
// This allows modifications to the given biome via the provided Builder.
}

public Codec<? extends BiomeModifier> codec()
{
// This must return a registered Codec, see Biome Modifier Serializers below.
}
}
</syntaxhighlight>

Beware! The modify method is called once per each phase per biome per biome modifier, so it's important to check which phase you're in and only make your changes in one phase.

<syntaxhighlight lang="java">
public void modify(Holder<Biome> biome, Phase phase, Builder builder)
{
if (phase == Phase.ADD)
{
// add things to biomes
}
}
</syntaxhighlight>

Typically we also want to restrict biome modifiers to only apply to certain biomes. We can do that by accepting a HolderSet<Biome> in our constructor:

<syntaxhighlight lang="java">
public record ExampleBiomeModifier(HolderSet<Biome> biomes) implements BiomeModifier
{
public void modify(Holder<Biome> biome, Phase phase, Builder builder)
{
if (phase == Phase.ADD && biomes.contains(biome))
{
// add things to biomes
}
}
</syntaxhighlight>

We can also accept Holders or HolderSets for other [[Registration#Data_Driven_Entries|datapack registry elements]], such as PlacedFeatures, which allows our BiomeModifier to refer to those elements, which can then be defined in their own JSON files.

<syntaxhighlight lang="java">
public record ExampleBiomeModifier(HolderSet<Biome> biomes, Holder<PlacedFeature> feature) implements BiomeModifier
{
public void modify(Holder<Biome> biome, Phase phase, Builder builder)
{
// add a feature to all specified biomes
if (phase == Phase.ADD && biomes.contains(biome))
{
builder.getGenerationSettings().addFeature(Decoration.UNDERGROUND_ORES, feature);
}
}
</syntaxhighlight>

= Biome Modifier Serializers =

Each type of biome modifier must have a [[Codecs|Codec]] registered for it; [[Registration#DeferredRegister|Deferred Registers]] are the recommended way to register biome modifier codecs.

<syntaxhighlight lang="java">
public class YourMod
{
static DeferredRegister<Codec<? extends BiomeModifier>> BIOME_MODIFIER_SERIALIZERS =
DeferredRegister.create(ForgeRegistries.Keys.BIOME_MODIFIER_SERIALIZERS, "yourmodid");

static RegistryObject<Codec<ExampleBiomeModifier>> EXAMPLE_CODEC = BIOME_MODIFIER_SERIALIZERS.register("example", () ->
RecordCodecBuilder.create(builder -> builder.group(
// declare fields
Biome.LIST_CODEC.fieldOf("biomes").forGetter(ExampleBiomeModifier::biomes),
PlacedFeature.CODEC.fieldOf("feature").forGetter(ExampleBiomeModifier::feature)
// declare constructor
).apply(builder, ExampleBiomeModifier::new)));
}
</syntaxhighlight>

We can now implement the codec() method in our biome modifier class:

<syntaxhighlight lang="java">
public record ExampleBiomeModifier(HolderSet<Biome> biomes, Holder<PlacedFeature> feature) implements BiomeModifier
{
public void modify(Holder<Biome> biome, Phase phase, Builder builder)
{
// add a feature to all specified biomes
if (phase == Phase.ADD && biomes.contains(biome))
{
builder.getGenerationSettings().addFeature(Decoration.UNDERGROUND_ORES, feature);
}
}

public Codec<? extends BiomeModifier> codec()
{
return YourMod.EXAMPLE_CODEC.get();
}
}
</syntaxhighlight>

= Biome Modifier JSONs =

Once we've registered a codec for our biome modifier type, we can define jsons for it. Biome modifier jsons must be defined in the directory <code>data/modid/forge/biome_modifier/</code>; a biome modifier at <code>data/modid/forge/biome_modifier/your_biome_modifier.json</code> has the namespaced id <code>modid:your_biome_modifier</code>.

Our codec above defines the json format:

* "biomes" is a special HolderSet field that accepts a single biome id, [list of biome ids], or #biome_tag.
* "feature" accepts a single placed feature id. [https://minecraft.fandom.com/wiki/Placed_feature These can be defined in jsons as well].
* We must also specify "type": "yourmodid:example" to tell the data loader to use our registered codec to read our json.

A json instance of our biome modifier that adds some feature to badlands biomes might look like this:

<syntaxhighlight lang="json">
{
"type": "yourmodid:example",
"biomes": "#minecraft:is_badlands",
"feature": "yourmod:some_feature"
}
</syntaxhighlight>

== Side Note on HolderSets ==

HolderSets are a vanilla feature that, when defined in JSON, can be specified as a single id, list of ids, or tag. Our example above uses a holderset of biomes, so all three of these are valid biome fields:

<syntaxhighlight lang="json">
{
"biomes": "forest",
"biomes": ["forest", "birch_forest"],
"biomes": "#is_forest"
}
</syntaxhighlight>

= Datageneration =

Biome Modifier jsons can be [[Datageneration|datagenerated via GatherDataEvent]]. As biome modifiers are datapack registry objects, this can be done by using JsonCodecProvider#forDatapackRegistry as the data provider. Refer to [[Datageneration/Datapack_Registries]] for additional information on datagenerating datapack registry elements.

= Builtin Biome Modifier Types =

Forge provides the following builtin biome modifier types:

== None ==

A no-op biome modifier type, whose jsons have the following format:

<syntaxhighlight lang="json">
{
"type": "forge:none"
}
</syntaxhighlight>

This allows pack devs or server operators to disable mods' biome modifiers by overriding their biome modifier jsons with the above.

== Add Features ==

This biome modifier type adds placed features to biomes.

<syntaxhighlight lang="json">
{
"type": "forge:add_features", // required
"biomes": "#namespace:your_biome_tag" // accepts a biome id, [list of biome ids], or #namespace:biome_tag
"features": "namespace:your_feature", // accepts a placed feature id, [list of placed feature ids], or #namespace:feature_tag
"step": "underground_ores" // accepts a Decoration enum name
}
</syntaxhighlight>

Decoration steps in order of generation are:
* raw_generation
* lakes
* local_modifications
* underground_structures
* surface_structures
* underground_ores
* underground_decoration
* fluid_springs
* vegetal_decoration
* top_layer_modification

== Remove Features ==

This biome modifier type removes features from biomes.

<syntaxhighlight lang="json">
{
"type": "forge:remove_features", // required
"biomes": "#namespace:your_biome_tag" // accepts a biome id, [list of biome ids], or #namespace:biome_tag
"features": "namespace:your_feature", // accepts a placed feature id, [list of placed feature ids], or #namespace:feature_tag
"steps": "underground_ores" // optional field specifying a Decoration or list of Decorations to remove features from, defaults to all if not specified
}
</syntaxhighlight>

== Add Spawns ==

This biome modifier type adds mob spawns to biomes.

<syntaxhighlight lang="json">
{
"type": "forge:add_spawn", // Required
"biomes": "#namespace:biome_tag", // Accepts a biome id, [list of biome ids], or #namespace:biome_tag
"spawner":
{
"type": "namespace:entity_type", // Type of mob to spawn
"weight": 100, // int, spawn weighting
"minCount": 1, // int, minimum pack size
"maxCount": 4, // int, maximum pack size
}
}
</syntaxhighlight>

== Remove Spawns ==

This biome modifier type removes mob spawns from biomes.

<syntaxhighlight lang="json">
{
"type": "forge:add_spawn", // Required
"biomes": "#namespace:biome_tag", // Accepts a biome id, [list of biome ids], or #namespace:biome_tag
"entity_types": "#namespace:entitytype_tag" // Accepts an entity type, list, or tag of entitytypes whose spawns are to be removed from the biomes
}
</syntaxhighlight>

= Best Practices =

* Avoid using biome modifiers to add vanilla placed features to biomes, as this may cause a feature cycle violation (the game will crash if two biomes have the same two features in their feature lists but in different orders). Placed features can be referenced in biome jsons or added via biome modifiers, but should not be used in both.

Datapack Registries

2022-06-15T00:49:41Z

Commoble: Created page with "Datapack Registries (sometimes called dynamic registries or worldgen registries) are a set of registries that are able to load data from JSONS when a server starts. These incl..."

Datapack Registries (sometimes called dynamic registries or worldgen registries) are a set of registries that are able to load data from JSONS when a server starts. These include all vanilla registries enumerated in RegistryAccess.REGISTRIES (such as biomes, placed features, and dimensiontypes) as well as any [[Registration#Creating_Custom_Registries|custom forge registry]] marked with <code>datapackRegistry()</code> in its registry builder (such as forge's [[Biome Modifiers]] and Structure Modifiers).

These registries have the ability to have their objects registered in Java ''or'' defined in JSON.

If a mod ships a JSON for a datapack registry element in its builtin datapack, then it is not necessary for that mod to register the object in java; however, it is also possible to create the object during GatherDataEvent and datagenerate the JSON.

All datapack registries are automatically datagenerable; each registry has a [[Codecs|Codec]] registered to it, which defines the serialization, and each registry's registry id determines its json directory. Elements of vanilla registries are loaded from <code>data/{element-namespace}/{registry-path}/{element-path}.json</code>, while elements of custom registries are loaded from <code>data/{element-namespace}/{registry-namespace}/{registry-path}/{element-path}.json</code>, using the namespace and path of the registry and the element.

= RegistryOps and RegistryAccess =

RegistryOps is a special [[DynamicOps]] made specifically for de/serializing datapack registries; it provides additional registry context and enables the use of special codecs that can only be used with RegistryOps. Datageneration of datapack registry elements must always be done using RegistryOps to convert elements to JsonElements.

RegistryOps can be created via <code>RegistryOps.create(JsonElement.INSTANCE, RegistryAccess.builtinCopy())</code>. RegistryAccess.builtinCopy() creates a set of writable datapack registries, which is necessary for datagenerating unregistered objects. All datageneration done in a GatherDataEvent handler must use the same RegistryAccess/RegistryOps instances (trying to encode an object in one set of registries that refers to an object in another set of registries will fail with strange holder errors).

== Holders ==

A Holder vaguely resembles a Pair<Key, Value> that either starts with a key and has a value bound later, or starts with a value and may have a key bound later.

Many datapack registry elements must be constructed with Holders that refer to elements of other registries; for example, PlacedFeatures are constructed with a Holder of a ConfiguredFeature. When datagenerating objects, we can use <code>RegistryOps#registry</code> to get a registry and <code>Registry#getOrCreateHolderOrThrow</code> to produce key-only reference holders (the key being the only part of the holder we need, as holder codecs encode only the key of the holder when using a RegistryOps).

Holders referring to datapack registry elements must be retrieved from the RegistryAccess/RegistryOps that will be used for datageneration; holders referring to static registry elements (such as blocks) can come directly from a ForgeRegistry as e.g. there is only ever one registry for blocks.

= JsonCodecProvider =

Forge provides a dataprovider for datapack registry elements that, given a registry key and a map of objects to datagenerate, datagenerates all elements in the map to the locations determined by their keys.

<syntaxhighlight lang="java">
void onGatherData(GatherDataEvent event)
{
DataGenerator generator = event.getDataGenerator();
ExistingFileHelper existingFileHelper = event.getExistingFileHelper();
RegistryAccess registryAccess = RegistryAccess.builtinCopy();
RegistryOps<JsonElement> registryOps = RegistryOps.create(JsonOps.INSTANCE, registryAccess);

ResourceLocation placedFeatureRL = new ResourceLocation("modid", "sponge_everywhere");
PlacedFeature placedFeature = new PlacedFeature(//etc;)
// All placed features to be datagenerated can be in the map
Map<ResourceLocation, PlacedFeature> map = Map.of(placedFeatureRL, placedFeature);

JsonCodecProvider provider = JsonCodecProvider.forDatapackRegistry(
dataGenerator, existingFileHelper, "modid", registryOps, Registry.PLACED_FEATURE_REGISTRY, map);

event.addProvider(event.includeServer(), provider);
}
</syntaxhighlight>

Biome Modifiers

2022-06-14T23:27:18Z

Commoble: /* Biome Modifier JSONs */

Main Page

2022-06-10T00:14:45Z

Commoble: Link to biome modifiers

__NOTOC__
<templatestyles src=":Main_Page/styles.css" />

<div class="center title">Forge Community Wiki</div>

The Forge Community Wiki exists so that the community can:

* collectively keep track of major changes and updates to Forge and Vanilla code;
* create and edit articles with in-depth explanations about a variety of Forge-related subjects; and
* contribute example code and tutorials for both simple and difficult concepts.

This wiki is editable by any registered user with an account. This is to allow tracking of harmful edits, but isn't imposing any annoying limits. We welcome any edit, however small. Join the [https://discord.gg/Nn42eAh Discord] to discuss changes and edits to the wiki with others and the staff.

{{Supported versions}}

<div class="center round_warning">
Please read the [[FCWMeta:Wiki Policy|wiki policy]] before editing!
</div>

<div class="center" style="margin-bottom: 2em">
{{Supported versions|text=1}}
</div>

<div class="box_container center">

<div class="box">
<h3>Beginner Topics</h3>
* [[Getting Started]]
* [[Proper Mod Structuring]]
* [[Mods.toml file]]
* [[Debug Profiler|The Debug Profiler]]
* [[Version Checker]]
</div>

<div class="box">
<h3>Common Concepts</h3>
* [[Sides|Understanding Sides]]
* [[Events|Understanding Events]]
* [[Registration]]
* [[Internationalization]]
* [[Configs]]
</div>

<div class="box">
<h3>Forge Conventions</h3>
* [[Semantic Versioning]]
* [[Stages of Modloading]]
</div>

<div class="box">
<h3>Resources and Data</h3>
* [[Using Resources|Introduction]]
* [[Recipes]]
* [[Tags]]
</div>

<div class="box">
<h3>Blocks</h3>
* [[Making Blocks|Creating Blocks]]
* [[Understanding Blockstates]]
* [[Interacting With Blocks|Block Interactions]]
</div>

<div class="box">
<h3>Items</h3>
* [[Making Items]]
* [[Making Tools]]
* [[BlockEntityWithoutLevelRenderer|<tt>BlockEntityWithoutLevelRenderer</tt>]]
</div>

<div class="box">
<h3>Data Generation</h3>
* [[Datageneration|Introduction]]
* [[Datageneration/Recipes|Recipes]]
* [[Datageneration/Tags|Tags]]
* [[Datageneration/Loot_Tables|Loot Tables]]
* [[Datageneration/I18n|Localization]]
* [[Datageneration/States and Models|<tt>BlockState</tt>s and Models]]
</div>

<div class="box">
<h3>Block Entities</h3>
* [[Block Entities|Introduction]]
* [[Block Entity Renderer|<tt>BlockEntityRenderer</tt>]]
</div>

<div class="box">
<h3>Models</h3>
* [[Introduction to Models|Introduction]]
* [[Model JSONs]]
* [[BlockState JSONs]]
* [[Coloring textures dynamically|Dynamically Colored Textures]]
* [[Item Properties]]
</div>

<div class="box">
<h3>Handling Information</h3>
* [[Understanding Networking|Introduction]]
* [[Using NBT|Named Binary Tag (NBT)]]
* [[Using FriendlyByteBuf|Using <tt>FriendlyByteBuf</tt>]]
* [[Sending Packets|Sending and Receiving Packets]]
* [[Using SimpleChannel|Using <tt>SimpleChannel</tt>]]
* [[Networking with Entities]]
* [[DynamicOps|Using DynamicOps]]
* [[Codecs|Using Codecs]]
</div>

<div class="box">
<h3>Data Storage</h3>
* [[Capabilities|Understanding Capabilities]]
* [[Capabilities/Attaching|Attaching Capabilities]]
* [[Saved Data]]
</div>

<div class="box">
<h3>Game Effects</h3>
* [[Mob Effects]]
* [[Potions]]
* [[Particles]]
* [[Sounds]]
</div>

<div class="box">
<h3>Events</h3>
* [[Events|Understanding Events]]
* [[Entity Events]]
</div>

<div class="box">
<h3>Others</h3>
* [[Dependencies]]
* [[Dynamic Loot Modification]]
* [[Components|Components and Translation Keys]]
* [[Key Mappings]]
* [[Access Transformers]]
* [[Toolchain]]
* [[Game_Tests|Game Tests]]
* [[Biome_Modifiers|Biome Modifiers]]
</div>

<div class="box">
<h3>Recipes</h3>
* [[Recipes]]
* [[Custom Recipes|Custom Recipe Types]]
* [[Datageneration/Recipes|Datageneration]]
</div>

</div>

Biome Modifiers

2022-06-10T00:12:07Z

Commoble: Biome modifiers documentation

Codecs

2021-08-06T20:41:34Z

Commoble: Fix incorrect name in pair codec example

Codecs are a serialization tool from mojang's DataFixerUpper library. Codecs are used alongside [[DynamicOps|DynamicOps]] to allow objects to be serialized to different formats and back, such as JSON or [[Using_NBT|NBT]]. While the DynamicOps describes the format the object is to be serialized to, the Codec describes the manner in which the object is to be serialized; a single Codec can be used to serialize an object to any format for which a DynamicOps exists. Codecs and DynamicOps jointly form an abstraction layer over data serialization, simplifying the effort needed to serialize or deserialize data.

= Using Codecs =

== Serialization and Deserialization ==

The primary use for Codecs is to serialize java objects to some serialized type, such as a JsonElement or a Tag, and to deserialize an serialized object back to its proper java type. This is accomplished with <code>Codec#encodeStart</code> and <code>Codec#parse</code>, respectively. Given a Codec<SomeJavaType> and a DynamicOps<SomeSerializedType>, we can convert instances of SomeJavaType to instances of SomeSerializedType and back.

Each of these methods take a [[DynamicOps]] instance and an instance of the object we are serializing or deserializing, and returns a DataResult:

<syntaxhighlight lang="java">
// let someCodec be a Codec<SomeJavaType>
// let someJavaObject be an instance of SomeJavaType
// let someTag and someJsonElement be instances of Tag and JsonElement, respectively

// serialize some java object to Tag
DataResult<Tag> result = someCodec.encodeStart(NBTOps.INSTANCE, someJavaObject);

// deserialize some Tag instance back to a proper java object
DataResult<SomeJavaType> result = someCodec.parse(NBTOps.INSTANCE, someTag );

// serialize some java object to a JsonElement
DataResult<JsonElement> result = someCodec.encodeStart(JsonOps.INSTANCE, someJavaObject);

// deserialize a JsonElement back to a proper java object
DataResult<SomeJavaType> result = someCodec.parse(JsonOps.INSTANCE, someJsonElement);
</syntaxhighlight>

A DataResult either holds the converted instance, or it holds some error data, depending on whether the conversion was successful or not, respectively. There are several things we can do with this DataResult; <code>DataResult#result</code> simply returns an Optional containing the converted object if the conversion was successful, while <code>DataResult#resultOrPartial</code> also runs a given function if the conversion was unsuccessful (in addition to returning the Optional). #resultOrPartial is particularly useful for logging errors during datapack deserialization:

<syntaxhighlight lang="java">
// deserialize something from json
someCodec.parse(JsonOps.INSTANCE, someJsonElement)
.resultOrPartial(errorMessage -> doSomethingIfBadData(errorMessage))
.ifPresent(someJavaObject -> doSomethingIfGoodData(someJavaObject))
</syntaxhighlight>

== Builtin Codecs ==
=== Primitive Codecs ===
The Codec class itself contains static instances of codecs for all supported primitive types, e.g. <code>Codec.STRING</code> is the canonical <code>Codec<String></code> implementation. Primitive codecs include:
* BOOL, which serializes to a boolean value
* BYTE, SHORT, INT, LONG, FLOAT, and DOUBLE, which serialize to numeric values
* STRING, which serializes to a string
* BYTE_BUFFER, INT_STREAM, and LONG_STREAM, which serialize to lists of numbers
* EMPTY, which represents null objects

=== Other Builtin Codecs ===
Vanilla minecraft has many builtin codecs for objects that it frequently serializes. These codecs are typically static instances in the class the codec is serializing; e.g. <code>ResourceLocation.CODEC</code> is the canonical <code>Codec<ResourceLocation></code>, while <code>BlockPos.CODEC</code> is the codec used for serializing a BlockPos.

Each vanilla <code>Registry</code> acts as the Codec for the type of object the registry contains; e.g. <code>Registry.BLOCK</code> is itself a <code>Codec<Block></code>. Forge Registries, however, do not currently implement Codec and cannot yet be used in this way; custom codecs must be created for forge-specific registries that are not tied to specific vanilla registries.

Of particular note here is the CompoundTag.CODEC, which can be used to e.g. serialize a CompoundTag into a json file. This has a notable limitation in that CompoundTag.CODEC *cannot* safely deserialize lists of numbers from json, due to the strong typing of ListTag and the way that the NBTOps deserializer reads numeric values.

= Creating Codecs =

Suppose we have the following class, and we want to deserialize json files to instances of this class:
<syntaxhighlight lang="java">
public class ExampleCodecClass {

private final int someInt;
private final Item item;
private final List<BlockPos> blockPositions;

public ExampleCodecClass(int someInt, Item item, List<BlockPos> blockPositions) {...}

public int getSomeInt() { return this.someInt; }
public Item getItem() { return this.item; }
public List<BlockPos> getBlockPositions() { return this.blockPositions; }
}
</syntaxhighlight>

Where a json file for an instance of this class might look like:

<syntaxhighlight lang="json">
{
"some_int": 42,
"item": "minecraft:gold_ingot",
"block_positions":
[
[0,0,0],
[10,20,-100]
]
}
</syntaxhighlight>

We can assemble a codec for this class by building a new codec out of smaller codecs. We'll need a codec for each of these fields:
* a <code>Codec<Integer></code>
* a <code>Codec<Item></code>
* a <code>Codec<List<BlockPos>></code>
And then we'll need to assemble these into a <code>Codec<ExampleCodecClass></code>.

As previously mentioned, we can use <code>Codec.INT</code> for the integer codec, and <code>Registry.ITEM</code> for the Item codec. We don't have a builtin codec for list-of-blockpos, but we can use BlockPos.CODEC to create one.

== Lists ==
The <code>Codec#listOf</code> instance method can be used to generate a codec for a List from an existing codec:

<syntaxhighlight lang="java">
// BlockPos.CODEC is a Codec<BlockPos>
Codec<List<BlockPos>> = BlockPos.CODEC.listOf();
</syntaxhighlight>

Codecs created via listOf() serialize things to listlike objects, such as [] json arrays or ListTags.

Deserializing a list in this manner produces an ''immutable'' list. If a mutable list is needed, [[Codecs#Equivalent_Types_and_xmap|xmap]] can be used to convert the list after deserializing.

== Records ==
RecordCodecBuilder is used to generate codecs that serialize instances of classes with explicitly named fields, like our example above. Codecs created via RecordCodecBuilder serialize things to maplike objects, such as {} json objects or CompoundTags.

RecordCodecBuilder can be used in several ways, but the simplest form is as follows:

<syntaxhighlight lang="java">
public static final Codec<SomeJavaClass> = RecordCodecBuilder.create(instance -> instance.group(
someFieldCodecA.fieldOf("field_name_a").forGetter(SomeJavaClass::getFieldA),
someFieldCodecB.fieldOf("field_name_b").forGetter(SomeJavaClass::getFieldB),
someFieldCodecC.fieldOf("field_name_c").forGetter(SomeJavaClass::getFieldC),
// up to 16 fields can be declared here
).apply(instance, SomeJavaClass::new));
</syntaxhighlight>

Where each line in the group specifies a codec instance for the type of that field, the field name in the serialized object, and the corresponding getter function in the java class. The builder is concluded by specifying a constructor or factory for the java class whose arguments are the previously defined fields in the same order.

For example, using RecordCodecBuilder to create a codec for our example class above:
<syntaxhighlight lang="java">
public static final Codec<ExampleCodecClass> = RecordCodecBuilder.create(instance -> instance.group(
Codec.INT.fieldOf("some_int").forGetter(ExampleCodecClass::getSomeInt),
Registry.ITEM.fieldOf("item").forGetter(ExampleCodecClass::getItem),
BlockPos.CODEC.listOf().fieldOf("block_positions").forGetter(ExampleCodecClass::getBlockPositions)
).apply(instance, ExampleCodecClass::new));
</syntaxhighlight>

===Optional and Default Values in Record Fields===
When RecordCodecBuilder is used as shown above, all of the fields are *required* to be in the serialized object (the JsonObject/CompoundTag/etc), or the entire thing will fail to parse when the codec tries to deserialize it. If we wish to have optional or default values, we have several alternatives of fieldOf() we can use.

* <code>someCodec.optionalFieldOf("field_name")</code> creates a field for an Optional. If the field in the json/nbt is not present or invalid, it will deserialize as an empty optional. Empty optionals will not be serialized; the field will be omitted from the json or nbt.
* <code>someCodec.optionalFieldOf("field_name", someDefaultValue)</code> creates an optional field that deserializes as the given default value if the field is not present in the json/nbt. When serializing, if the field in the java object equals the default value, the value will not be serialized and the field will be omitted from the json or nbt.

When using optional fields, be wary that if the field contains bad data or otherwise fails to serialize, the error will be silently caught, and the field will serialize as the default value instead!

===Boxing values as objects===
In some situations, we may need to serialize a single value as a single-field object. We can use fieldOf to box a single value in this way without needing the entire RecordCodecBuilder process:

<syntaxhighlight lang="java">
public static final Codec<Integer> BOXED_INT_CODEC = Codec.INT.fieldOf("value").codec();

JsonElement value = BOXED_INT_CODEC.encodeStart(JsonOps.INSTANCE, 5).result().get();
</syntaxhighlight>

Which serializes the following output:

<syntaxhighlight lang="json">
{"value":5}
</syntaxhighlight>

==Unit==
The <code>Codec.unit(defaultValue)</code> codec creates a Codec that always deserializes a specified default value, regardless of input. When serializing, it serializes nothing.

==Pair==
The <code>Codec.pair(codecA, codecB)</code> static method takes two codecs and generates a Codec<Pair<A,B>> from them.

The only valid arguments for this method are codecs that serialize to objects with explicit fields, such as codecs created using [[Codecs#Records|RecordCodecBuilder]] or [[Codecs#Boxing_values_as_objects|fieldOf]]. Codecs that serialize nothing (such as [[Codecs#Unit|unit codecs]]) are also valid as they act as objects-with-no-fields.

The resulting Pair codec will serialize a single object that has all of the fields of the two original codecs. For example:
<syntaxhighlight lang="java">
public static final Codec<Pair<Integer,String>> PAIR_CODEC = Codec.pair(
Codec.INT.fieldOf("value").codec(),
Codec.STRING.fieldOf("name").codec());

JsonElement encodedPair = PAIR_CODEC.encodeStart(JsonOps.INSTANCE, Pair.of(5, "cheese").result().get();
</syntaxhighlight>
This codec serializes the above value to:
<syntaxhighlight lang="json">
{
"value": 5,
"name": "cheese"
}
</syntaxhighlight>

Codecs that serialize to objects with undefined fields such as [[Codecs#Maps|unboundedMap]] may cause strange and unpredictable behaviour when used here; these objects should be boxed via fieldOf when used in a pair codec.

==Either==
The <code>Codec.either(codecA, codecB)</code> static method takes two codecs and generates a Codec<Either<A,B>> from them.

When this codec is used to de/serialize an object, it first attempts to use the first codec; if and only if that conversion fails, it then attempts to use the second codec. If that conversion also fails, then the returned DataResult will contain the error data from the *second* codec's conversion attempt.

==Numeric Ranges==
The <code>Codec.intRange(min,max)</code>, <code>Codec.floatRange(min,max)</code>, and <code>Codec.doubleRange(min,max)</code> static methods generate Codecs for Integers, Floats, or Doubles, respectively, for which only a specified inclusive range is valid, and values outside that range will fail to de/serialize.

==Maps==
Suppose we want to serialize a HashMap or other Map type, where we could have indefinitely many key-value pairs and we don't know what the keys are ahead of time.

We can create a <code>Codec<Map<KEY,VALUE>></code> using the <code>Codec.unboundedMap</code> static method, which takes a key codec and a value codec and creates a codec for a map type:

<syntaxhighlight lang="java">
public static final Codec<Map<String, BlockPos>> = Codec.unboundedMap(Codec.STRING, BlockPos.CODEC);
</syntaxhighlight>

The serialized form of maps serialized by this codec will be a JsonObject or CompoundTag, whose fields are the key-value pairs in the map; the map's keys will be used as the field names, and the map's values will be the values of those fields

A limitation of using unboundedMap is that it only supports key codecs that serialize to Strings (including codecs for things like ResourceLocation that aren't Strings themselves but still serialize to strings). To create a codec for a Map whose keys are not fundamentally strings, the Map must be serialized as a list of key-value pairs instead of using unboundedMap.

==Equivalent Types and xmap==
Suppose we have two java classes, Amalgam and Box; any Amalgam instance can be converted to a Box, and vice-versa. Now suppose we have a Codec<Amalgam>, but we'd also like to have a Codec<Box>. Rather than creating an entirely new codec for Box from scratch, we can simply xmap our Amalgam codec instead.

The <code>Codec#xmap</code> instance method is used to generate a second codec for a fundamentally equivalent type to the first codec's type. The method takes two function objects as arguments, which are used to convert the first type to the second when deserializing, and converting the second type to the first when serializing:

<syntaxhighlight lang="java">
public static final Codec<Box> = Amalgam.CODEC.xmap(Amalgam::toBox, Box::toAmalgam);
</syntaxhighlight>

Codecs created in this manner will serialize objects in the same format as the starting codec.

==Partially Equivalent Types, flatComapMap, comapFlatMap, and flatXMap==
Consider the ResourceLocation: Any ResourceLocation can be converted to a String, but not all Strings can be converted to a ResourceLocation; ResourceLocations have strict limits on their format and allowed characters.

While we *could* use xmap to convert the Codec.STRING to a codec for ResourceLocations, this would cause attempts to parse an invalid string like <code>SHOUTY:MOD:Invalid$Characters</code> to throw a runtime exception, when we really should be returning a failed DataResult to the parser instead -- which indeed is what the vanilla ResourceLocation codec does.

Codecs have three additional instance methods for creating equivalent codecs for when we can *potentially* convert one type to another, but are not guaranteed to be able to do so. These take conversion function arguments that return DataResults, allowing validation to be performed during serialization and deserialization.

{| class="wikitable"
|+ Codec Conversion Methods
|-
! Can A always be converted to B? !! Can B always be converted to A? !! Which method of codecA should be used to create codecB?
|-
| yes || yes || codecA.xmap
|-
| yes || no || codecA.flatComapMap
|-
| no || yes || codecA.comapFlatMap
|-
| no || no || codecA.flatXmap
|}

== Registry Dispatch ==
Registry Dispatch Codecs allow us to define a registry of codecs and delegate to a specific codec to deserialize a particular json based on a type field in that json. Dispatch codecs are used extensively when deserializing worldgen data.

To create a dispatch codec for a Thing class, the following steps can be performed:
# Create a Thing abstract class class and ThingType interface. The ThingType interface should have a method that supplies a Codec<Thing>, while Thing subclasses must define a method that supplies a ThingType.
# Create a map or registry of ThingTypes, and register a ThingType for each sub-codec we want to have.
# Create a Codec<ThingType>, or have the ThingType registry implement Codec.
# Create our Codec<Thing> master codec by invoking <code>Codec#dispatch</code> on our ThingType codec. This method's arguments are:
## A field name for the ID of the sub-codec (the example json below is using "type")
## A function to retrieve a ThingType from a Thing
## A function to retrieve a Codec<Thing> from a ThingType

We can then use our Codec<Thing> to create Thing fields in other codecs whose serialized format depends on the specific sub-codec used by a Thing instance.

As an example of this, consider the ExampleCodecClass earlier. Suppose we make this class extend Thing and register our codec for it to a codec dispatch registry with the id "ourmod:exampleclass". If we were to define an instance of this class in a Thing field in some json, it would look like

<syntaxhighlight lang="json">
"some_thing":
{
"type": "ourmod:exampleclass",
"some_int": 42,
"item": "minecraft:gold_ingot",
"block_positions":
[
[0,0,0],
[10,20,-100]
]
}
</syntaxhighlight>

Other ThingTypes we register would have different fields in this json object, but would still be valid for the "some_thing" field.

Several examples of vanilla classes that use dispatch codecs:
* RuleTest and RuleTestType
* BlockPlacer and BlockPlacerType
* ConfiguredDecorator and FeatureDecorator

=External Links=
* [https://github.com/Mojang/DataFixerUpper/blob/master/src/main/java/com/mojang/serialization/Codec.java Codecs in Mojang's official public DataFixerUpper repository]
* [https://kvverti.github.io/Documented-DataFixerUpper/snapshot/com/mojang/serialization/Codec.html#flatXmap-java.util.function.Function-java.util.function.Function- Unofficial Codec Javadocs]

Codecs

2021-01-09T21:19:51Z

Commoble: Add note that listOf codecs produce immutable Lists

Codecs are a serialization tool from mojang's DataFixerUpper library. Codecs are used alongside [[DynamicOps|DynamicOps]] to allow objects to be serialized to different formats and back, such as JSON or [[Using_NBT|NBT]]. While the DynamicOps describes the format the object is to be serialized to, the Codec describes the manner in which the object is to be serialized; a single Codec can be used to serialize an object to any format for which a DynamicOps exists. Codecs and DynamicOps jointly form an abstraction layer over data serialization, simplifying the effort needed to serialize or deserialize data.

= Using Codecs =

== Serialization and Deserialization ==

The primary use for Codecs is to serialize java objects to some serialized type, such as a JsonElement or an INBT, and to deserialize an serialized object back to its proper java type. This is accomplished with <code>Codec#encodeStart</code> and <code>Codec#parse</code>, respectively. Given a Codec<SomeJavaType> and a DynamicOps<SomeSerializedType>, we can convert instances of SomeJavaType to instances of SomeSerializedType and back.

Each of these methods take a [[DynamicOps]] instance and an instance of the object we are serializing or deserializing, and returns a DataResult:

<syntaxhighlight lang="java">
// let someCodec be a Codec<SomeJavaType>
// let someJavaObject be an instance of SomeJavaType
// let someNBT and someJsonElement be instances of INBT and JsonElement, respectively

// serialize some java object to INBT
DataResult<INBT> result = someCodec.encodeStart(NBTDynamicOps.INSTANCE, someJavaObject);

// deserialize some INBT instance back to a proper java object
DataResult<SomeJavaType> result = someCodec.parse(NBTDynamicOps.INSTANCE, someNBT);

// serialize some java object to a JsonElement
DataResult<JsonElement> result = someCodec.encodeStart(JsonOps.INSTANCE, someJavaObject);

// deserialize a JsonElement back to a proper java object
DataResult<SomeJavaType> result = someCodec.parse(JsonOps.INSTANCE, someJsonElement);
</syntaxhighlight>

A DataResult either holds the converted instance, or it holds some error data, depending on whether the conversion was successful or not, respectively. There are several things we can do with this DataResult; <code>DataResult#result</code> simply returns an Optional containing the converted object if the conversion was successful, while <code>DataResult#resultOrPartial</code> also runs a given function if the conversion was unsuccessful (in addition to returning the Optional). #resultOrPartial is particularly useful for logging errors during datapack deserialization:

<syntaxhighlight lang="java">
// deserialize something from json
someCodec.parse(JsonOps.INSTANCE, someJsonElement)
.resultOrPartial(errorMessage -> doSomethingIfBadData(errorMessage))
.ifPresent(someJavaObject -> doSomethingIfGoodData(someJavaObject))
</syntaxhighlight>

== Builtin Codecs ==
=== Primitive Codecs ===
The Codec class itself contains static instances of codecs for all supported primitive types, e.g. <code>Codec.STRING</code> is the canonical <code>Codec<String></code> implementation. Primitive codecs include:
* BOOL, which serializes to a boolean value
* BYTE, SHORT, INT, LONG, FLOAT, and DOUBLE, which serialize to numeric values
* STRING, which serializes to a string
* BYTE_BUFFER, INT_STREAM, and LONG_STREAM, which serialize to lists of numbers
* EMPTY, which represents null objects

=== Other Builtin Codecs ===
Vanilla minecraft has many builtin codecs for objects that it frequently serializes. These codecs are typically static instances in the class the codec is serializing; e.g. <code>ResourceLocation.CODEC</code> is the canonical <code>Codec<ResourceLocation></code>, while <code>BlockPos.CODEC</code> is the codec used for serializing a BlockPos.

Each vanilla <code>Registry</code> acts as the Codec for the type of object the registry contains; e.g. <code>Registry.BLOCK</code> is itself a <code>Codec<Block></code>. Forge Registries, however, do not currently implement Codec and cannot yet be used in this way; custom codecs must be created for forge-specific registries that are not tied to specific vanilla registries.

Of particular note here is the CompoundNBT.CODEC, which can be used to e.g. serialize a CompoundNBT into a json file. This has a notable limitation in that CompoundNBT.CODEC *cannot* safely deserialize lists of numbers from json, due to the strong typing of ListNBT and the way that the NBTDynamicOps deserializer reads numeric values.

= Creating Codecs =

Suppose we have the following class, and we want to deserialize json files to instances of this class:
<syntaxhighlight lang="java">
public class ExampleCodecClass {

private final int someInt;
private final Item item;
private final List<BlockPos> blockPositions;

public ExampleCodecClass(int someInt, Item item, List<BlockPos> blockPositions) {...}

public int getSomeInt() { return this.someInt; }
public Item getItem() { return this.someItem; }
public List<BlockPos> getBlockPositions() { return this.blockPositions; }
}
</syntaxhighlight>

Where a json file for an instance of this class might look like:

<syntaxhighlight lang="json">
{
"some_int": 42,
"item": "minecraft:gold_ingot",
"block_positions":
[
[0,0,0],
[10,20,-100]
]
}
</syntaxhighlight>

We can assemble a codec for this class by building a new codec out of smaller codecs. We'll need a codec for each of these fields:
* a <code>Codec<Integer></code>
* a <code>Codec<Item></code>
* a <code>Codec<List<BlockPos>></code>
And then we'll need to assemble these into a <code>Codec<ExampleCodecClass</code>.

As previously mentioned, we can use <code>Codec.INT</code> for the integer codec, and <code>Registry.ITEM</code> for the Item codec. We don't have a builtin codec for list-of-blockpos, but we can use BlockPos.CODEC to create one.

== Lists ==
The <code>Codec#listOf</code> instance method can be used to generate a codec for a List from an existing codec:

<syntaxhighlight lang="java">
// BlockPos.CODEC is a Codec<BlockPos>
Codec<List<BlockPos>> = BlockPos.CODEC.listOf();
</syntaxhighlight>

Codecs created via listOf() serialize things to listlike objects, such as [] json arrays or ListNBTs.

Deserializing a list in this manner produces an ''immutable'' list. If a mutable list is needed, [[Codecs#Equivalent_Types_and_xmap|xmap]] can be used to convert the list after deserializing.

== Records ==
RecordCodecBuilder is used to generate codecs that serialize instances of classes with explicitly named fields, like our example above. Codecs created via RecordCodecBuilder serialize things to maplike objects, such as {} json objects or CompoundNBTs.

RecordCodecBuilder can be used in several ways, but the simplest form is as follows:

<syntaxhighlight lang="java">
public static final Codec<SomeJavaClass> = RecordCodecBuilder.create(instance -> instance.group(
someFieldCodecA.fieldOf("field_name_a").forGetter(SomeJavaClass::getFieldA),
someFieldCodecB.fieldOf("field_name_b").forGetter(SomeJavaClass::getFieldB),
someFieldCodecC.fieldOf("field_name_c").forGetter(SomeJavaClass::getFieldC),
// up to 16 fields can be declared here
).apply(instance, SomeJavaClass::new));
</syntaxhighlight>

Where each line in the group specifies a codec instance for the type of that field, the field name in the serialized object, and the corresponding getter function in the java class. The builder is concluded by specifying a constructor or factory for the java class whose arguments are the previously defined fields in the same order.

For example, using RecordCodecBuilder to create a codec for our example class above:
<syntaxhighlight lang="java">
public static final Codec<ExampleCodecClass> = RecordCodecBuilder.create(instance -> instance.group(
Codec.INT.fieldOf("some_int").forGetter(ExampleCodecClass::getSomeInt),
Registry.ITEM.fieldOf("item").forGetter(ExampleCodecClass::getItem),
BlockPos.CODEC.listOf().fieldOf("block_positions").forGetter(ExampleCodecClass::getBlockPositions)
).apply(instance, ExampleCodecClass::new));
</syntaxhighlight>

===Optional and Default Values in Record Fields===
When RecordCodecBuilder is used as shown above, all of the fields are *required* to be in the serialized object (the JsonObject/CompoundNBT/etc), or the entire thing will fail to parse when the codec tries to deserialize it. If we wish to have optional or default values, we have several alternatives of fieldOf() we can use.

* <code>someCodec.optionalFieldOf("field_name")</code> creates a field for an Optional. If the field in the json/nbt is not present or invalid, it will deserialize as an empty optional. Empty optionals will not be serialized; the field will be omitted from the json or nbt.
* <code>someCodec.optionalFieldOf("field_name", someDefaultValue)</code> creates an optional field that deserializes as the given default value if the field is not present in the json/nbt. When serializing, if the field in the java object equals the default value, the value will not be serialized and the field will be omitted from the json or nbt.

When using optional fields, be wary that if the field contains bad data or otherwise fails to serialize, the error will be silently caught, and the field will serialize as the default value instead!

===Boxing values as objects===
In some situations, we may need to serialize a single value as a single-field object. We can use fieldOf to box a single value in this way without needing the entire RecordCodecBuilder process:

<syntaxhighlight lang="java">
public static final Codec<Integer> BOXED_INT_CODEC = Codec.INT.fieldOf("value").codec();

JsonElement value = BOXED_INT_CODEC.encodeStart(JsonOps.INSTANCE, 5).result().get();
</syntaxhighlight>

Which serializes the following output:

<syntaxhighlight lang="json">
{"value":5}
</syntaxhighlight>

==Unit==
The <code>Codec.unit(defaultValue)</code> codec creates a Codec that always deserializes a specified default value, regardless of input. When serializing, it serializes nothing.

==Pair==
The <code>Codec.pair(codecA, codecB)</code> static method takes two codecs and generates a Codec<Pair<A,B>> from them.

The only valid arguments for this method are codecs that serialize to objects with explicit fields, such as codecs created using [[Codecs#Records|RecordCodecBuilder]] or [[Codecs#Boxing_values_as_objects|fieldOf]]. Codecs that serialize nothing (such as [[Codecs#Unit|unit codecs]]) are also valid as they act as objects-with-no-fields.

The resulting Pair codec will serialize a single object that has all of the fields of the two original codecs. For example:
<syntaxhighlight lang="java">
public static final Codec<Pair<Integer,String>> PAIR_CODEC = Codec.pair(
Codec.INT.fieldOf("value").codec(),
Codec.STRING.fieldOf("name").codec());

JsonElement encodedPair = BOXED_INT_CODEC.encodeStart(JsonOps.INSTANCE, Pair.of(5, "cheese").result().get();
</syntaxhighlight>
This codec serializes the above value to:
<syntaxhighlight lang="json">
{
"value": 5,
"name": "cheese"
}
</syntaxhighlight>

Codecs that serialize to objects with undefined fields such as [[Codecs#Maps|unboundedMap]] may cause strange and unpredictable behaviour when used here; these objects should be boxed via fieldOf when used in a pair codec.

==Either==
The <code>Codec.either(codecA, codecB)</code> static method takes two codecs and generates a Codec<Either<A,B>> from them.

When this codec is used to de/serialize an object, it first attempts to use the first codec; if and only if that conversion fails, it then attempts to use the second codec. If that conversion also fails, then the returned DataResult will contain the error data from the *second* codec's conversion attempt.

==Numeric Ranges==
The <code>Codec.intRange(min,max)</code>, <code>Codec.floatRange(min,max)</code>, and <code>Codec.doubleRange(min,max)</code> static methods generate Codecs for Integers, Floats, or Doubles, respectively, for which only a specified inclusive range is valid, and values outside that range will fail to de/serialize.

==Maps==
Suppose we want to serialize a HashMap or other Map type, where we could have indefinitely many key-value pairs and we don't know what the keys are ahead of time.

We can create a <code>Codec<Map<KEY,VALUE>></code> using the <code>Codec.unboundedMap</code> static method, which takes a key codec and a value codec and creates a codec for a map type:

<syntaxhighlight lang="java">
public static final Codec<Map<String, BlockPos>> = Codec.unboundedMap(Codec.STRING, BlockPos.CODEC);
</syntaxhighlight>

The serialized form of maps serialized by this codec will be a JsonObject or CompoundNBT, whose fields are the key-value pairs in the map; the map's keys will be used as the field names, and the map's values will be the values of those fields

A limitation of using unboundedMap is that it only supports key codecs that serialize to Strings (including codecs for things like ResourceLocation that aren't Strings themselves but still serialize to strings). To create a codec for a Map whose keys are not fundamentally strings, the Map must be serialized as a list of key-value pairs instead of using unboundedMap.

==Equivalent Types and xmap==
Suppose we have two java classes, Amalgam and Box; any Amalgam instance can be converted to a Box, and vice-versa. Now suppose we have a Codec<Amalgam>, but we'd also like to have a Codec<Box>. Rather than creating an entirely new codec for Box from scratch, we can simply xmap our Amalgam codec instead.

The <code>Codec#xmap</code> instance method is used to generate a second codec for a fundamentally equivalent type to the first codec's type. The method takes two function objects as arguments, which are used to convert the first type to the second when deserializing, and converting the second type to the first when serializing:

<syntaxhighlight lang="java">
public static final Codec<Box> = Amalgam.CODEC.xmap(Amalgam::toBox, Box::toAmalgam);
</syntaxhighlight>

Codecs created in this manner will serialize objects in the same format as the starting codec.

==Partially Equivalent Types, flatComapMap, comapFlatMap, and flatXMap==
Consider the ResourceLocation: Any ResourceLocation can be converted to a String, but not all Strings can be converted to a ResourceLocation; ResourceLocations have strict limits on their format and allowed characters.

While we *could* use xmap to convert the Codec.STRING to a codec for ResourceLocations, this would cause attempts to parse an invalid string like <code>SHOUTY:MOD:Invalid$Characters</code> to throw a runtime exception, when we really should be returning a failed DataResult to the parser instead -- which indeed is what the vanilla ResourceLocation codec does.

Codecs have three additional instance methods for creating equivalent codecs for when we can *potentially* convert one type to another, but are not guaranteed to be able to do so. These take conversion function arguments that return DataResults, allowing validation to be performed during serialization and deserialization.

{| class="wikitable"
|+ Codec Conversion Methods
|-
! Can A always be converted to B? !! Can B always be converted to A? !! Which method of codecA should be used to create codecB?
|-
| yes || yes || codecA.xmap
|-
| yes || no || codecA.flatComapMap
|-
| no || yes || codecA.comapFlatMap
|-
| no || no || codecA.flatXmap
|}

== Registry Dispatch ==
Registry Dispatch Codecs allow us to define a registry of codecs and delegate to a specific codec to deserialize a particular json based on a type field in that json. Dispatch codecs are used extensively when deserializing worldgen data.

To create a dispatch codec for a Thing class, the following steps can be performed:
# Create a Thing abstract class class and ThingType interface. The ThingType interface should have a method that supplies a Codec<Thing>, while Thing subclasses must define a method that supplies a ThingType.
# Create a map or registry of ThingTypes, and register a ThingType for each sub-codec we want to have.
# Create a Codec<ThingType>, or have the ThingType registry implement Codec.
# Create our Codec<Thing> master codec by invoking <code>Codec#dispatch</code> on our ThingType codec. This method's arguments are:
## A field name for the ID of the sub-codec (the example json below is using "type")
## A function to retrieve a ThingType from a Thing
## A function to retrieve a Codec<Thing> from a ThingType

We can then use our Codec<Thing> to create Thing fields in other codecs whose serialized format depends on the specific sub-codec used by a Thing instance.

As an example of this, consider the ExampleCodecClass earlier. Suppose we make this class extend Thing and register our codec for it to a codec dispatch registry with the id "ourmod:exampleclass". If we were to define an instance of this class in a Thing field in some json, it would look like

<syntaxhighlight lang="json">
"some_thing":
{
"type": "ourmod:exampleclass",
"some_int": 42,
"item": "minecraft:gold_ingot",
"block_positions":
[
[0,0,0],
[10,20,-100]
]
}
</syntaxhighlight>

Other ThingTypes we register would have different fields in this json object, but would still be valid for the "some_thing" field.

Several examples of vanilla classes that use dispatch codecs:
* RuleTest and IRuleTestType
* IParticleData and ParticleType
* ConfiguredPlacement and Placement

=External Links=
* [https://github.com/Mojang/DataFixerUpper/blob/master/src/main/java/com/mojang/serialization/Codec.java Codecs in Mojang's official public DataFixerUpper repository]
* [https://kvverti.github.io/Documented-DataFixerUpper/snapshot/com/mojang/serialization/Codec.html#flatXmap-java.util.function.Function-java.util.function.Function- Unofficial Codec Javadocs]

Codecs

2021-01-06T00:41:46Z

Commoble: Remove incorrect statement re: pairs and strings from Maps section, clarify relationship between map keys and object fields

Codecs are a serialization tool from mojang's DataFixerUpper library. Codecs are used alongside [[DynamicOps|DynamicOps]] to allow objects to be serialized to different formats and back, such as JSON or [[Using_NBT|NBT]]. While the DynamicOps describes the format the object is to be serialized to, the Codec describes the manner in which the object is to be serialized; a single Codec can be used to serialize an object to any format for which a DynamicOps exists. Codecs and DynamicOps jointly form an abstraction layer over data serialization, simplifying the effort needed to serialize or deserialize data.

= Using Codecs =

== Serialization and Deserialization ==

The primary use for Codecs is to serialize java objects to some serialized type, such as a JsonElement or an INBT, and to deserialize an serialized object back to its proper java type. This is accomplished with <code>Codec#encodeStart</code> and <code>Codec#parse</code>, respectively. Given a Codec<SomeJavaType> and a DynamicOps<SomeSerializedType>, we can convert instances of SomeJavaType to instances of SomeSerializedType and back.

Each of these methods take a [[DynamicOps]] instance and an instance of the object we are serializing or deserializing, and returns a DataResult:

<syntaxhighlight lang="java">
// let someCodec be a Codec<SomeJavaType>
// let someJavaObject be an instance of SomeJavaType
// let someNBT and someJsonElement be instances of INBT and JsonElement, respectively

// serialize some java object to INBT
DataResult<INBT> result = someCodec.encodeStart(NBTDynamicOps.INSTANCE, someJavaObject);

// deserialize some INBT instance back to a proper java object
DataResult<SomeJavaType> result = someCodec.parse(NBTDynamicOps.INSTANCE, someNBT);

// serialize some java object to a JsonElement
DataResult<JsonElement> result = someCodec.encodeStart(JsonOps.INSTANCE, someJavaObject);

// deserialize a JsonElement back to a proper java object
DataResult<SomeJavaType> result = someCodec.parse(JsonOps.INSTANCE, someJsonElement);
</syntaxhighlight>

A DataResult either holds the converted instance, or it holds some error data, depending on whether the conversion was successful or not, respectively. There are several things we can do with this DataResult; <code>DataResult#result</code> simply returns an Optional containing the converted object if the conversion was successful, while <code>DataResult#resultOrPartial</code> also runs a given function if the conversion was unsuccessful (in addition to returning the Optional). #resultOrPartial is particularly useful for logging errors during datapack deserialization:

<syntaxhighlight lang="java">
// deserialize something from json
someCodec.parse(JsonOps.INSTANCE, someJsonElement)
.resultOrPartial(errorMessage -> doSomethingIfBadData(errorMessage))
.ifPresent(someJavaObject -> doSomethingIfGoodData(someJavaObject))
</syntaxhighlight>

== Builtin Codecs ==
=== Primitive Codecs ===
The Codec class itself contains static instances of codecs for all supported primitive types, e.g. <code>Codec.STRING</code> is the canonical <code>Codec<String></code> implementation. Primitive codecs include:
* BOOL, which serializes to a boolean value
* BYTE, SHORT, INT, LONG, FLOAT, and DOUBLE, which serialize to numeric values
* STRING, which serializes to a string
* BYTE_BUFFER, INT_STREAM, and LONG_STREAM, which serialize to lists of numbers
* EMPTY, which represents null objects

=== Other Builtin Codecs ===
Vanilla minecraft has many builtin codecs for objects that it frequently serializes. These codecs are typically static instances in the class the codec is serializing; e.g. <code>ResourceLocation.CODEC</code> is the canonical <code>Codec<ResourceLocation></code>, while <code>BlockPos.CODEC</code> is the codec used for serializing a BlockPos.

Each vanilla <code>Registry</code> acts as the Codec for the type of object the registry contains; e.g. <code>Registry.BLOCK</code> is itself a <code>Codec<Block></code>. Forge Registries, however, do not currently implement Codec and cannot yet be used in this way; custom codecs must be created for forge-specific registries that are not tied to specific vanilla registries.

Of particular note here is the CompoundNBT.CODEC, which can be used to e.g. serialize a CompoundNBT into a json file. This has a notable limitation in that CompoundNBT.CODEC *cannot* safely deserialize lists of numbers from json, due to the strong typing of ListNBT and the way that the NBTDynamicOps deserializer reads numeric values.

= Creating Codecs =

Suppose we have the following class, and we want to deserialize json files to instances of this class:
<syntaxhighlight lang="java">
public class ExampleCodecClass {

private final int someInt;
private final Item item;
private final List<BlockPos> blockPositions;

public ExampleCodecClass(int someInt, Item item, List<BlockPos> blockPositions) {...}

public int getSomeInt() { return this.someInt; }
public Item getItem() { return this.someItem; }
public List<BlockPos> getBlockPositions() { return this.blockPositions; }
}
</syntaxhighlight>

Where a json file for an instance of this class might look like:

<syntaxhighlight lang="json">
{
"some_int": 42,
"item": "minecraft:gold_ingot",
"block_positions":
[
[0,0,0],
[10,20,-100]
]
}
</syntaxhighlight>

We can assemble a codec for this class by building a new codec out of smaller codecs. We'll need a codec for each of these fields:
* a <code>Codec<Integer></code>
* a <code>Codec<Item></code>
* a <code>Codec<List<BlockPos>></code>
And then we'll need to assemble these into a <code>Codec<ExampleCodecClass</code>.

As previously mentioned, we can use <code>Codec.INT</code> for the integer codec, and <code>Registry.ITEM</code> for the Item codec. We don't have a builtin codec for list-of-blockpos, but we can use BlockPos.CODEC to create one.

== Lists ==
The <code>Codec#listOf</code> instance method can be used to generate a codec for a List from an existing codec:

<syntaxhighlight lang="java">
// BlockPos.CODEC is a Codec<BlockPos>
Codec<List<BlockPos>> = BlockPos.CODEC.listOf();
</syntaxhighlight>

Codecs created via listOf() serialize things to listlike objects, such as [] json arrays or ListNBTs.

== Records ==
RecordCodecBuilder is used to generate codecs that serialize instances of classes with explicitly named fields, like our example above. Codecs created via RecordCodecBuilder serialize things to maplike objects, such as {} json objects or CompoundNBTs.

RecordCodecBuilder can be used in several ways, but the simplest form is as follows:

<syntaxhighlight lang="java">
public static final Codec<SomeJavaClass> = RecordCodecBuilder.create(instance -> instance.group(
someFieldCodecA.fieldOf("field_name_a").forGetter(SomeJavaClass::getFieldA),
someFieldCodecB.fieldOf("field_name_b").forGetter(SomeJavaClass::getFieldB),
someFieldCodecC.fieldOf("field_name_c").forGetter(SomeJavaClass::getFieldC),
// up to 16 fields can be declared here
).apply(instance, SomeJavaClass::new));
</syntaxhighlight>

Where each line in the group specifies a codec instance for the type of that field, the field name in the serialized object, and the corresponding getter function in the java class. The builder is concluded by specifying a constructor or factory for the java class whose arguments are the previously defined fields in the same order.

For example, using RecordCodecBuilder to create a codec for our example class above:
<syntaxhighlight lang="java">
public static final Codec<ExampleCodecClass> = RecordCodecBuilder.create(instance -> instance.group(
Codec.INT.fieldOf("some_int").forGetter(ExampleCodecClass::getSomeInt),
Registry.ITEM.fieldOf("item").forGetter(ExampleCodecClass::getItem),
BlockPos.CODEC.listOf().fieldOf("block_positions").forGetter(ExampleCodecClass::getBlockPositions)
).apply(instance, ExampleCodecClass::new));
</syntaxhighlight>

===Optional and Default Values in Record Fields===
When RecordCodecBuilder is used as shown above, all of the fields are *required* to be in the serialized object (the JsonObject/CompoundNBT/etc), or the entire thing will fail to parse when the codec tries to deserialize it. If we wish to have optional or default values, we have several alternatives of fieldOf() we can use.

* <code>someCodec.optionalFieldOf("field_name")</code> creates a field for an Optional. If the field in the json/nbt is not present or invalid, it will deserialize as an empty optional. Empty optionals will not be serialized; the field will be omitted from the json or nbt.
* <code>someCodec.optionalFieldOf("field_name", someDefaultValue)</code> creates an optional field that deserializes as the given default value if the field is not present in the json/nbt. When serializing, if the field in the java object equals the default value, the value will not be serialized and the field will be omitted from the json or nbt.

When using optional fields, be wary that if the field contains bad data or otherwise fails to serialize, the error will be silently caught, and the field will serialize as the default value instead!

===Boxing values as objects===
In some situations, we may need to serialize a single value as a single-field object. We can use fieldOf to box a single value in this way without needing the entire RecordCodecBuilder process:

<syntaxhighlight lang="java">
public static final Codec<Integer> BOXED_INT_CODEC = Codec.INT.fieldOf("value").codec();

JsonElement value = BOXED_INT_CODEC.encodeStart(JsonOps.INSTANCE, 5).result().get();
</syntaxhighlight>

Which serializes the following output:

<syntaxhighlight lang="json">
{"value":5}
</syntaxhighlight>

==Unit==
The <code>Codec.unit(defaultValue)</code> codec creates a Codec that always deserializes a specified default value, regardless of input. When serializing, it serializes nothing.

==Pair==
The <code>Codec.pair(codecA, codecB)</code> static method takes two codecs and generates a Codec<Pair<A,B>> from them.

The only valid arguments for this method are codecs that serialize to objects with explicit fields, such as codecs created using [[Codecs#Records|RecordCodecBuilder]] or [[Codecs#Boxing_values_as_objects|fieldOf]]. Codecs that serialize nothing (such as [[Codecs#Unit|unit codecs]]) are also valid as they act as objects-with-no-fields.

The resulting Pair codec will serialize a single object that has all of the fields of the two original codecs. For example:
<syntaxhighlight lang="java">
public static final Codec<Pair<Integer,String>> PAIR_CODEC = Codec.pair(
Codec.INT.fieldOf("value").codec(),
Codec.STRING.fieldOf("name").codec());

JsonElement encodedPair = BOXED_INT_CODEC.encodeStart(JsonOps.INSTANCE, Pair.of(5, "cheese").result().get();
</syntaxhighlight>
This codec serializes the above value to:
<syntaxhighlight lang="json">
{
"value": 5,
"name": "cheese"
}
</syntaxhighlight>

Codecs that serialize to objects with undefined fields such as [[Codecs#Maps|unboundedMap]] may cause strange and unpredictable behaviour when used here; these objects should be boxed via fieldOf when used in a pair codec.

==Either==
The <code>Codec.either(codecA, codecB)</code> static method takes two codecs and generates a Codec<Either<A,B>> from them.

When this codec is used to de/serialize an object, it first attempts to use the first codec; if and only if that conversion fails, it then attempts to use the second codec. If that conversion also fails, then the returned DataResult will contain the error data from the *second* codec's conversion attempt.

==Numeric Ranges==
The <code>Codec.intRange(min,max)</code>, <code>Codec.floatRange(min,max)</code>, and <code>Codec.doubleRange(min,max)</code> static methods generate Codecs for Integers, Floats, or Doubles, respectively, for which only a specified inclusive range is valid, and values outside that range will fail to de/serialize.

==Maps==
Suppose we want to serialize a HashMap or other Map type, where we could have indefinitely many key-value pairs and we don't know what the keys are ahead of time.

We can create a <code>Codec<Map<KEY,VALUE>></code> using the <code>Codec.unboundedMap</code> static method, which takes a key codec and a value codec and creates a codec for a map type:

<syntaxhighlight lang="java">
public static final Codec<Map<String, BlockPos>> = Codec.unboundedMap(Codec.STRING, BlockPos.CODEC);
</syntaxhighlight>

The serialized form of maps serialized by this codec will be a JsonObject or CompoundNBT, whose fields are the key-value pairs in the map; the map's keys will be used as the field names, and the map's values will be the values of those fields

A limitation of using unboundedMap is that it only supports key codecs that serialize to Strings (including codecs for things like ResourceLocation that aren't Strings themselves but still serialize to strings). To create a codec for a Map whose keys are not fundamentally strings, the Map must be serialized as a list of key-value pairs instead of using unboundedMap.

==Equivalent Types and xmap==
Suppose we have two java classes, Amalgam and Box; any Amalgam instance can be converted to a Box, and vice-versa. Now suppose we have a Codec<Amalgam>, but we'd also like to have a Codec<Box>. Rather than creating an entirely new codec for Box from scratch, we can simply xmap our Amalgam codec instead.

The <code>Codec#xmap</code> instance method is used to generate a second codec for a fundamentally equivalent type to the first codec's type. The method takes two function objects as arguments, which are used to convert the first type to the second when deserializing, and converting the second type to the first when serializing:

<syntaxhighlight lang="java">
public static final Codec<Box> = Amalgam.CODEC.xmap(Amalgam::toBox, Box::toAmalgam);
</syntaxhighlight>

Codecs created in this manner will serialize objects in the same format as the starting codec.

==Partially Equivalent Types, flatComapMap, comapFlatMap, and flatXMap==
Consider the ResourceLocation: Any ResourceLocation can be converted to a String, but not all Strings can be converted to a ResourceLocation; ResourceLocations have strict limits on their format and allowed characters.

While we *could* use xmap to convert the Codec.STRING to a codec for ResourceLocations, this would cause attempts to parse an invalid string like <code>SHOUTY:MOD:Invalid$Characters</code> to throw a runtime exception, when we really should be returning a failed DataResult to the parser instead -- which indeed is what the vanilla ResourceLocation codec does.

Codecs have three additional instance methods for creating equivalent codecs for when we can *potentially* convert one type to another, but are not guaranteed to be able to do so. These take conversion function arguments that return DataResults, allowing validation to be performed during serialization and deserialization.

{| class="wikitable"
|+ Codec Conversion Methods
|-
! Can A always be converted to B? !! Can B always be converted to A? !! Which method of codecA should be used to create codecB?
|-
| yes || yes || codecA.xmap
|-
| yes || no || codecA.flatComapMap
|-
| no || yes || codecA.comapFlatMap
|-
| no || no || codecA.flatXmap
|}

== Registry Dispatch ==
Registry Dispatch Codecs allow us to define a registry of codecs and delegate to a specific codec to deserialize a particular json based on a type field in that json. Dispatch codecs are used extensively when deserializing worldgen data.

To create a dispatch codec for a Thing class, the following steps can be performed:
# Create a Thing abstract class class and ThingType interface. The ThingType interface should have a method that supplies a Codec<Thing>, while Thing subclasses must define a method that supplies a ThingType.
# Create a map or registry of ThingTypes, and register a ThingType for each sub-codec we want to have.
# Create a Codec<ThingType>, or have the ThingType registry implement Codec.
# Create our Codec<Thing> master codec by invoking <code>Codec#dispatch</code> on our ThingType codec. This method's arguments are:
## A field name for the ID of the sub-codec (the example json below is using "type")
## A function to retrieve a ThingType from a Thing
## A function to retrieve a Codec<Thing> from a ThingType

We can then use our Codec<Thing> to create Thing fields in other codecs whose serialized format depends on the specific sub-codec used by a Thing instance.

As an example of this, consider the ExampleCodecClass earlier. Suppose we make this class extend Thing and register our codec for it to a codec dispatch registry with the id "ourmod:exampleclass". If we were to define an instance of this class in a Thing field in some json, it would look like

<syntaxhighlight lang="json">
"some_thing":
{
"type": "ourmod:exampleclass",
"some_int": 42,
"item": "minecraft:gold_ingot",
"block_positions":
[
[0,0,0],
[10,20,-100]
]
}
</syntaxhighlight>

Other ThingTypes we register would have different fields in this json object, but would still be valid for the "some_thing" field.

Several examples of vanilla classes that use dispatch codecs:
* RuleTest and IRuleTestType
* IParticleData and ParticleType
* ConfiguredPlacement and Placement

=External Links=
* [https://github.com/Mojang/DataFixerUpper/blob/master/src/main/java/com/mojang/serialization/Codec.java Codecs in Mojang's official public DataFixerUpper repository]
* [https://kvverti.github.io/Documented-DataFixerUpper/snapshot/com/mojang/serialization/Codec.html#flatXmap-java.util.function.Function-java.util.function.Function- Unofficial Codec Javadocs]

Codecs

2021-01-06T00:38:37Z

Commoble: Add boxing/fieldOf link to Pair section, add warning for unbounded maps, add Pair example

Codecs are a serialization tool from mojang's DataFixerUpper library. Codecs are used alongside [[DynamicOps|DynamicOps]] to allow objects to be serialized to different formats and back, such as JSON or [[Using_NBT|NBT]]. While the DynamicOps describes the format the object is to be serialized to, the Codec describes the manner in which the object is to be serialized; a single Codec can be used to serialize an object to any format for which a DynamicOps exists. Codecs and DynamicOps jointly form an abstraction layer over data serialization, simplifying the effort needed to serialize or deserialize data.

= Using Codecs =

== Serialization and Deserialization ==

The primary use for Codecs is to serialize java objects to some serialized type, such as a JsonElement or an INBT, and to deserialize an serialized object back to its proper java type. This is accomplished with <code>Codec#encodeStart</code> and <code>Codec#parse</code>, respectively. Given a Codec<SomeJavaType> and a DynamicOps<SomeSerializedType>, we can convert instances of SomeJavaType to instances of SomeSerializedType and back.

Each of these methods take a [[DynamicOps]] instance and an instance of the object we are serializing or deserializing, and returns a DataResult:

<syntaxhighlight lang="java">
// let someCodec be a Codec<SomeJavaType>
// let someJavaObject be an instance of SomeJavaType
// let someNBT and someJsonElement be instances of INBT and JsonElement, respectively

// serialize some java object to INBT
DataResult<INBT> result = someCodec.encodeStart(NBTDynamicOps.INSTANCE, someJavaObject);

// deserialize some INBT instance back to a proper java object
DataResult<SomeJavaType> result = someCodec.parse(NBTDynamicOps.INSTANCE, someNBT);

// serialize some java object to a JsonElement
DataResult<JsonElement> result = someCodec.encodeStart(JsonOps.INSTANCE, someJavaObject);

// deserialize a JsonElement back to a proper java object
DataResult<SomeJavaType> result = someCodec.parse(JsonOps.INSTANCE, someJsonElement);
</syntaxhighlight>

A DataResult either holds the converted instance, or it holds some error data, depending on whether the conversion was successful or not, respectively. There are several things we can do with this DataResult; <code>DataResult#result</code> simply returns an Optional containing the converted object if the conversion was successful, while <code>DataResult#resultOrPartial</code> also runs a given function if the conversion was unsuccessful (in addition to returning the Optional). #resultOrPartial is particularly useful for logging errors during datapack deserialization:

<syntaxhighlight lang="java">
// deserialize something from json
someCodec.parse(JsonOps.INSTANCE, someJsonElement)
.resultOrPartial(errorMessage -> doSomethingIfBadData(errorMessage))
.ifPresent(someJavaObject -> doSomethingIfGoodData(someJavaObject))
</syntaxhighlight>

== Builtin Codecs ==
=== Primitive Codecs ===
The Codec class itself contains static instances of codecs for all supported primitive types, e.g. <code>Codec.STRING</code> is the canonical <code>Codec<String></code> implementation. Primitive codecs include:
* BOOL, which serializes to a boolean value
* BYTE, SHORT, INT, LONG, FLOAT, and DOUBLE, which serialize to numeric values
* STRING, which serializes to a string
* BYTE_BUFFER, INT_STREAM, and LONG_STREAM, which serialize to lists of numbers
* EMPTY, which represents null objects

=== Other Builtin Codecs ===
Vanilla minecraft has many builtin codecs for objects that it frequently serializes. These codecs are typically static instances in the class the codec is serializing; e.g. <code>ResourceLocation.CODEC</code> is the canonical <code>Codec<ResourceLocation></code>, while <code>BlockPos.CODEC</code> is the codec used for serializing a BlockPos.

Each vanilla <code>Registry</code> acts as the Codec for the type of object the registry contains; e.g. <code>Registry.BLOCK</code> is itself a <code>Codec<Block></code>. Forge Registries, however, do not currently implement Codec and cannot yet be used in this way; custom codecs must be created for forge-specific registries that are not tied to specific vanilla registries.

Of particular note here is the CompoundNBT.CODEC, which can be used to e.g. serialize a CompoundNBT into a json file. This has a notable limitation in that CompoundNBT.CODEC *cannot* safely deserialize lists of numbers from json, due to the strong typing of ListNBT and the way that the NBTDynamicOps deserializer reads numeric values.

= Creating Codecs =

Suppose we have the following class, and we want to deserialize json files to instances of this class:
<syntaxhighlight lang="java">
public class ExampleCodecClass {

private final int someInt;
private final Item item;
private final List<BlockPos> blockPositions;

public ExampleCodecClass(int someInt, Item item, List<BlockPos> blockPositions) {...}

public int getSomeInt() { return this.someInt; }
public Item getItem() { return this.someItem; }
public List<BlockPos> getBlockPositions() { return this.blockPositions; }
}
</syntaxhighlight>

Where a json file for an instance of this class might look like:

<syntaxhighlight lang="json">
{
"some_int": 42,
"item": "minecraft:gold_ingot",
"block_positions":
[
[0,0,0],
[10,20,-100]
]
}
</syntaxhighlight>

We can assemble a codec for this class by building a new codec out of smaller codecs. We'll need a codec for each of these fields:
* a <code>Codec<Integer></code>
* a <code>Codec<Item></code>
* a <code>Codec<List<BlockPos>></code>
And then we'll need to assemble these into a <code>Codec<ExampleCodecClass</code>.

As previously mentioned, we can use <code>Codec.INT</code> for the integer codec, and <code>Registry.ITEM</code> for the Item codec. We don't have a builtin codec for list-of-blockpos, but we can use BlockPos.CODEC to create one.

== Lists ==
The <code>Codec#listOf</code> instance method can be used to generate a codec for a List from an existing codec:

<syntaxhighlight lang="java">
// BlockPos.CODEC is a Codec<BlockPos>
Codec<List<BlockPos>> = BlockPos.CODEC.listOf();
</syntaxhighlight>

Codecs created via listOf() serialize things to listlike objects, such as [] json arrays or ListNBTs.

== Records ==
RecordCodecBuilder is used to generate codecs that serialize instances of classes with explicitly named fields, like our example above. Codecs created via RecordCodecBuilder serialize things to maplike objects, such as {} json objects or CompoundNBTs.

RecordCodecBuilder can be used in several ways, but the simplest form is as follows:

<syntaxhighlight lang="java">
public static final Codec<SomeJavaClass> = RecordCodecBuilder.create(instance -> instance.group(
someFieldCodecA.fieldOf("field_name_a").forGetter(SomeJavaClass::getFieldA),
someFieldCodecB.fieldOf("field_name_b").forGetter(SomeJavaClass::getFieldB),
someFieldCodecC.fieldOf("field_name_c").forGetter(SomeJavaClass::getFieldC),
// up to 16 fields can be declared here
).apply(instance, SomeJavaClass::new));
</syntaxhighlight>

Where each line in the group specifies a codec instance for the type of that field, the field name in the serialized object, and the corresponding getter function in the java class. The builder is concluded by specifying a constructor or factory for the java class whose arguments are the previously defined fields in the same order.

For example, using RecordCodecBuilder to create a codec for our example class above:
<syntaxhighlight lang="java">
public static final Codec<ExampleCodecClass> = RecordCodecBuilder.create(instance -> instance.group(
Codec.INT.fieldOf("some_int").forGetter(ExampleCodecClass::getSomeInt),
Registry.ITEM.fieldOf("item").forGetter(ExampleCodecClass::getItem),
BlockPos.CODEC.listOf().fieldOf("block_positions").forGetter(ExampleCodecClass::getBlockPositions)
).apply(instance, ExampleCodecClass::new));
</syntaxhighlight>

===Optional and Default Values in Record Fields===
When RecordCodecBuilder is used as shown above, all of the fields are *required* to be in the serialized object (the JsonObject/CompoundNBT/etc), or the entire thing will fail to parse when the codec tries to deserialize it. If we wish to have optional or default values, we have several alternatives of fieldOf() we can use.

* <code>someCodec.optionalFieldOf("field_name")</code> creates a field for an Optional. If the field in the json/nbt is not present or invalid, it will deserialize as an empty optional. Empty optionals will not be serialized; the field will be omitted from the json or nbt.
* <code>someCodec.optionalFieldOf("field_name", someDefaultValue)</code> creates an optional field that deserializes as the given default value if the field is not present in the json/nbt. When serializing, if the field in the java object equals the default value, the value will not be serialized and the field will be omitted from the json or nbt.

When using optional fields, be wary that if the field contains bad data or otherwise fails to serialize, the error will be silently caught, and the field will serialize as the default value instead!

===Boxing values as objects===
In some situations, we may need to serialize a single value as a single-field object. We can use fieldOf to box a single value in this way without needing the entire RecordCodecBuilder process:

<syntaxhighlight lang="java">
public static final Codec<Integer> BOXED_INT_CODEC = Codec.INT.fieldOf("value").codec();

JsonElement value = BOXED_INT_CODEC.encodeStart(JsonOps.INSTANCE, 5).result().get();
</syntaxhighlight>

Which serializes the following output:

<syntaxhighlight lang="json">
{"value":5}
</syntaxhighlight>

==Unit==
The <code>Codec.unit(defaultValue)</code> codec creates a Codec that always deserializes a specified default value, regardless of input. When serializing, it serializes nothing.

==Pair==
The <code>Codec.pair(codecA, codecB)</code> static method takes two codecs and generates a Codec<Pair<A,B>> from them.

The only valid arguments for this method are codecs that serialize to objects with explicit fields, such as codecs created using [[Codecs#Records|RecordCodecBuilder]] or [[Codecs#Boxing_values_as_objects|fieldOf]]. Codecs that serialize nothing (such as [[Codecs#Unit|unit codecs]]) are also valid as they act as objects-with-no-fields.

The resulting Pair codec will serialize a single object that has all of the fields of the two original codecs. For example:
<syntaxhighlight lang="java">
public static final Codec<Pair<Integer,String>> PAIR_CODEC = Codec.pair(
Codec.INT.fieldOf("value").codec(),
Codec.STRING.fieldOf("name").codec());

JsonElement encodedPair = BOXED_INT_CODEC.encodeStart(JsonOps.INSTANCE, Pair.of(5, "cheese").result().get();
</syntaxhighlight>
This codec serializes the above value to:
<syntaxhighlight lang="json">
{
"value": 5,
"name": "cheese"
}
</syntaxhighlight>

Codecs that serialize to objects with undefined fields such as [[Codecs#Maps|unboundedMap]] may cause strange and unpredictable behaviour when used here; these objects should be boxed via fieldOf when used in a pair codec.

==Either==
The <code>Codec.either(codecA, codecB)</code> static method takes two codecs and generates a Codec<Either<A,B>> from them.

When this codec is used to de/serialize an object, it first attempts to use the first codec; if and only if that conversion fails, it then attempts to use the second codec. If that conversion also fails, then the returned DataResult will contain the error data from the *second* codec's conversion attempt.

==Numeric Ranges==
The <code>Codec.intRange(min,max)</code>, <code>Codec.floatRange(min,max)</code>, and <code>Codec.doubleRange(min,max)</code> static methods generate Codecs for Integers, Floats, or Doubles, respectively, for which only a specified inclusive range is valid, and values outside that range will fail to de/serialize.

==Maps==
Suppose we want to serialize a HashMap or other Map type, where we could have indefinitely many key-value pairs and we don't know what the keys are ahead of time.

We can create a <code>Codec<Map<KEY,VALUE>></code> using the <code>Codec.unboundedMap</code> static method, which takes a key codec and a value codec and creates a codec for a map type:

<syntaxhighlight lang="java">
public static final Codec<Map<String, BlockPos>> = Codec.unboundedMap(Codec.STRING, BlockPos.CODEC);
</syntaxhighlight>

The serialized form of maps serialized by this codec will be a JsonObject or CompoundNBT, whose fields are the key-value pairs in the map.

A limitation of using unboundedMap is that it only supports key codecs that serialize to Strings (including codecs for things like ResourceLocation that aren't Strings themselves but still serialize to strings). To create a codec for a Map whose keys are not fundamentally strings, the Map must be serialized as a list of key-value entries instead of using unboundedMap. As Codec.pair() also requires that the first codec serialize to a string, the pair function cannot be used for this purpose either.

==Equivalent Types and xmap==
Suppose we have two java classes, Amalgam and Box; any Amalgam instance can be converted to a Box, and vice-versa. Now suppose we have a Codec<Amalgam>, but we'd also like to have a Codec<Box>. Rather than creating an entirely new codec for Box from scratch, we can simply xmap our Amalgam codec instead.

The <code>Codec#xmap</code> instance method is used to generate a second codec for a fundamentally equivalent type to the first codec's type. The method takes two function objects as arguments, which are used to convert the first type to the second when deserializing, and converting the second type to the first when serializing:

<syntaxhighlight lang="java">
public static final Codec<Box> = Amalgam.CODEC.xmap(Amalgam::toBox, Box::toAmalgam);
</syntaxhighlight>

Codecs created in this manner will serialize objects in the same format as the starting codec.

==Partially Equivalent Types, flatComapMap, comapFlatMap, and flatXMap==
Consider the ResourceLocation: Any ResourceLocation can be converted to a String, but not all Strings can be converted to a ResourceLocation; ResourceLocations have strict limits on their format and allowed characters.

While we *could* use xmap to convert the Codec.STRING to a codec for ResourceLocations, this would cause attempts to parse an invalid string like <code>SHOUTY:MOD:Invalid$Characters</code> to throw a runtime exception, when we really should be returning a failed DataResult to the parser instead -- which indeed is what the vanilla ResourceLocation codec does.

Codecs have three additional instance methods for creating equivalent codecs for when we can *potentially* convert one type to another, but are not guaranteed to be able to do so. These take conversion function arguments that return DataResults, allowing validation to be performed during serialization and deserialization.

{| class="wikitable"
|+ Codec Conversion Methods
|-
! Can A always be converted to B? !! Can B always be converted to A? !! Which method of codecA should be used to create codecB?
|-
| yes || yes || codecA.xmap
|-
| yes || no || codecA.flatComapMap
|-
| no || yes || codecA.comapFlatMap
|-
| no || no || codecA.flatXmap
|}

== Registry Dispatch ==
Registry Dispatch Codecs allow us to define a registry of codecs and delegate to a specific codec to deserialize a particular json based on a type field in that json. Dispatch codecs are used extensively when deserializing worldgen data.

To create a dispatch codec for a Thing class, the following steps can be performed:
# Create a Thing abstract class class and ThingType interface. The ThingType interface should have a method that supplies a Codec<Thing>, while Thing subclasses must define a method that supplies a ThingType.
# Create a map or registry of ThingTypes, and register a ThingType for each sub-codec we want to have.
# Create a Codec<ThingType>, or have the ThingType registry implement Codec.
# Create our Codec<Thing> master codec by invoking <code>Codec#dispatch</code> on our ThingType codec. This method's arguments are:
## A field name for the ID of the sub-codec (the example json below is using "type")
## A function to retrieve a ThingType from a Thing
## A function to retrieve a Codec<Thing> from a ThingType

We can then use our Codec<Thing> to create Thing fields in other codecs whose serialized format depends on the specific sub-codec used by a Thing instance.

As an example of this, consider the ExampleCodecClass earlier. Suppose we make this class extend Thing and register our codec for it to a codec dispatch registry with the id "ourmod:exampleclass". If we were to define an instance of this class in a Thing field in some json, it would look like

<syntaxhighlight lang="json">
"some_thing":
{
"type": "ourmod:exampleclass",
"some_int": 42,
"item": "minecraft:gold_ingot",
"block_positions":
[
[0,0,0],
[10,20,-100]
]
}
</syntaxhighlight>

Other ThingTypes we register would have different fields in this json object, but would still be valid for the "some_thing" field.

Several examples of vanilla classes that use dispatch codecs:
* RuleTest and IRuleTestType
* IParticleData and ParticleType
* ConfiguredPlacement and Placement

=External Links=
* [https://github.com/Mojang/DataFixerUpper/blob/master/src/main/java/com/mojang/serialization/Codec.java Codecs in Mojang's official public DataFixerUpper repository]
* [https://kvverti.github.io/Documented-DataFixerUpper/snapshot/com/mojang/serialization/Codec.html#flatXmap-java.util.function.Function-java.util.function.Function- Unofficial Codec Javadocs]

Codecs

2021-01-06T00:23:32Z

Commoble: Add section on boxing single values as objects

Codecs are a serialization tool from mojang's DataFixerUpper library. Codecs are used alongside [[DynamicOps|DynamicOps]] to allow objects to be serialized to different formats and back, such as JSON or [[Using_NBT|NBT]]. While the DynamicOps describes the format the object is to be serialized to, the Codec describes the manner in which the object is to be serialized; a single Codec can be used to serialize an object to any format for which a DynamicOps exists. Codecs and DynamicOps jointly form an abstraction layer over data serialization, simplifying the effort needed to serialize or deserialize data.

= Using Codecs =

== Serialization and Deserialization ==

The primary use for Codecs is to serialize java objects to some serialized type, such as a JsonElement or an INBT, and to deserialize an serialized object back to its proper java type. This is accomplished with <code>Codec#encodeStart</code> and <code>Codec#parse</code>, respectively. Given a Codec<SomeJavaType> and a DynamicOps<SomeSerializedType>, we can convert instances of SomeJavaType to instances of SomeSerializedType and back.

Each of these methods take a [[DynamicOps]] instance and an instance of the object we are serializing or deserializing, and returns a DataResult:

<syntaxhighlight lang="java">
// let someCodec be a Codec<SomeJavaType>
// let someJavaObject be an instance of SomeJavaType
// let someNBT and someJsonElement be instances of INBT and JsonElement, respectively

// serialize some java object to INBT
DataResult<INBT> result = someCodec.encodeStart(NBTDynamicOps.INSTANCE, someJavaObject);

// deserialize some INBT instance back to a proper java object
DataResult<SomeJavaType> result = someCodec.parse(NBTDynamicOps.INSTANCE, someNBT);

// serialize some java object to a JsonElement
DataResult<JsonElement> result = someCodec.encodeStart(JsonOps.INSTANCE, someJavaObject);

// deserialize a JsonElement back to a proper java object
DataResult<SomeJavaType> result = someCodec.parse(JsonOps.INSTANCE, someJsonElement);
</syntaxhighlight>

A DataResult either holds the converted instance, or it holds some error data, depending on whether the conversion was successful or not, respectively. There are several things we can do with this DataResult; <code>DataResult#result</code> simply returns an Optional containing the converted object if the conversion was successful, while <code>DataResult#resultOrPartial</code> also runs a given function if the conversion was unsuccessful (in addition to returning the Optional). #resultOrPartial is particularly useful for logging errors during datapack deserialization:

<syntaxhighlight lang="java">
// deserialize something from json
someCodec.parse(JsonOps.INSTANCE, someJsonElement)
.resultOrPartial(errorMessage -> doSomethingIfBadData(errorMessage))
.ifPresent(someJavaObject -> doSomethingIfGoodData(someJavaObject))
</syntaxhighlight>

== Builtin Codecs ==
=== Primitive Codecs ===
The Codec class itself contains static instances of codecs for all supported primitive types, e.g. <code>Codec.STRING</code> is the canonical <code>Codec<String></code> implementation. Primitive codecs include:
* BOOL, which serializes to a boolean value
* BYTE, SHORT, INT, LONG, FLOAT, and DOUBLE, which serialize to numeric values
* STRING, which serializes to a string
* BYTE_BUFFER, INT_STREAM, and LONG_STREAM, which serialize to lists of numbers
* EMPTY, which represents null objects

=== Other Builtin Codecs ===
Vanilla minecraft has many builtin codecs for objects that it frequently serializes. These codecs are typically static instances in the class the codec is serializing; e.g. <code>ResourceLocation.CODEC</code> is the canonical <code>Codec<ResourceLocation></code>, while <code>BlockPos.CODEC</code> is the codec used for serializing a BlockPos.

Each vanilla <code>Registry</code> acts as the Codec for the type of object the registry contains; e.g. <code>Registry.BLOCK</code> is itself a <code>Codec<Block></code>. Forge Registries, however, do not currently implement Codec and cannot yet be used in this way; custom codecs must be created for forge-specific registries that are not tied to specific vanilla registries.

Of particular note here is the CompoundNBT.CODEC, which can be used to e.g. serialize a CompoundNBT into a json file. This has a notable limitation in that CompoundNBT.CODEC *cannot* safely deserialize lists of numbers from json, due to the strong typing of ListNBT and the way that the NBTDynamicOps deserializer reads numeric values.

= Creating Codecs =

Suppose we have the following class, and we want to deserialize json files to instances of this class:
<syntaxhighlight lang="java">
public class ExampleCodecClass {

private final int someInt;
private final Item item;
private final List<BlockPos> blockPositions;

public ExampleCodecClass(int someInt, Item item, List<BlockPos> blockPositions) {...}

public int getSomeInt() { return this.someInt; }
public Item getItem() { return this.someItem; }
public List<BlockPos> getBlockPositions() { return this.blockPositions; }
}
</syntaxhighlight>

Where a json file for an instance of this class might look like:

<syntaxhighlight lang="json">
{
"some_int": 42,
"item": "minecraft:gold_ingot",
"block_positions":
[
[0,0,0],
[10,20,-100]
]
}
</syntaxhighlight>

We can assemble a codec for this class by building a new codec out of smaller codecs. We'll need a codec for each of these fields:
* a <code>Codec<Integer></code>
* a <code>Codec<Item></code>
* a <code>Codec<List<BlockPos>></code>
And then we'll need to assemble these into a <code>Codec<ExampleCodecClass</code>.

As previously mentioned, we can use <code>Codec.INT</code> for the integer codec, and <code>Registry.ITEM</code> for the Item codec. We don't have a builtin codec for list-of-blockpos, but we can use BlockPos.CODEC to create one.

== Lists ==
The <code>Codec#listOf</code> instance method can be used to generate a codec for a List from an existing codec:

<syntaxhighlight lang="java">
// BlockPos.CODEC is a Codec<BlockPos>
Codec<List<BlockPos>> = BlockPos.CODEC.listOf();
</syntaxhighlight>

Codecs created via listOf() serialize things to listlike objects, such as [] json arrays or ListNBTs.

== Records ==
RecordCodecBuilder is used to generate codecs that serialize instances of classes with explicitly named fields, like our example above. Codecs created via RecordCodecBuilder serialize things to maplike objects, such as {} json objects or CompoundNBTs.

RecordCodecBuilder can be used in several ways, but the simplest form is as follows:

<syntaxhighlight lang="java">
public static final Codec<SomeJavaClass> = RecordCodecBuilder.create(instance -> instance.group(
someFieldCodecA.fieldOf("field_name_a").forGetter(SomeJavaClass::getFieldA),
someFieldCodecB.fieldOf("field_name_b").forGetter(SomeJavaClass::getFieldB),
someFieldCodecC.fieldOf("field_name_c").forGetter(SomeJavaClass::getFieldC),
// up to 16 fields can be declared here
).apply(instance, SomeJavaClass::new));
</syntaxhighlight>

Where each line in the group specifies a codec instance for the type of that field, the field name in the serialized object, and the corresponding getter function in the java class. The builder is concluded by specifying a constructor or factory for the java class whose arguments are the previously defined fields in the same order.

For example, using RecordCodecBuilder to create a codec for our example class above:
<syntaxhighlight lang="java">
public static final Codec<ExampleCodecClass> = RecordCodecBuilder.create(instance -> instance.group(
Codec.INT.fieldOf("some_int").forGetter(ExampleCodecClass::getSomeInt),
Registry.ITEM.fieldOf("item").forGetter(ExampleCodecClass::getItem),
BlockPos.CODEC.listOf().fieldOf("block_positions").forGetter(ExampleCodecClass::getBlockPositions)
).apply(instance, ExampleCodecClass::new));
</syntaxhighlight>

===Optional and Default Values in Record Fields===
When RecordCodecBuilder is used as shown above, all of the fields are *required* to be in the serialized object (the JsonObject/CompoundNBT/etc), or the entire thing will fail to parse when the codec tries to deserialize it. If we wish to have optional or default values, we have several alternatives of fieldOf() we can use.

* <code>someCodec.optionalFieldOf("field_name")</code> creates a field for an Optional. If the field in the json/nbt is not present or invalid, it will deserialize as an empty optional. Empty optionals will not be serialized; the field will be omitted from the json or nbt.
* <code>someCodec.optionalFieldOf("field_name", someDefaultValue)</code> creates an optional field that deserializes as the given default value if the field is not present in the json/nbt. When serializing, if the field in the java object equals the default value, the value will not be serialized and the field will be omitted from the json or nbt.

When using optional fields, be wary that if the field contains bad data or otherwise fails to serialize, the error will be silently caught, and the field will serialize as the default value instead!

===Boxing values as objects===
In some situations, we may need to serialize a single value as a single-field object. We can use fieldOf to box a single value in this way without needing the entire RecordCodecBuilder process:

<syntaxhighlight lang="java">
public static final Codec<Integer> BOXED_INT_CODEC = Codec.INT.fieldOf("value").codec();

JsonElement value = BOXED_INT_CODEC.encodeStart(JsonOps.INSTANCE, 5).result().get();
</syntaxhighlight>

Which serializes the following output:

<syntaxhighlight lang="json">
{"value":5}
</syntaxhighlight>

==Unit==
The <code>Codec.unit(defaultValue)</code> codec creates a Codec that always deserializes a specified default value, regardless of input. When serializing, it serializes nothing.

==Pair==
The <code>Codec.pair(codecA, codecB)</code> static method takes two codecs and generates a Codec<Pair<A,B>> from them.

The only valid arguments for this method are codecs that serialize to objects, such as codecs created using [[Codecs#Records|RecordCodecBuilder]], [[Codecs#Maps|unboundedMap]], or fieldOf. Codecs that serialize nothing (such as [[Codecs#Unit|unit codecs]]) are also valid as they act as objects-with-no-fields.

The resulting Pair codec will serialize a single object that has all of the fields of the two original codecs.

==Either==
The <code>Codec.either(codecA, codecB)</code> static method takes two codecs and generates a Codec<Either<A,B>> from them.

When this codec is used to de/serialize an object, it first attempts to use the first codec; if and only if that conversion fails, it then attempts to use the second codec. If that conversion also fails, then the returned DataResult will contain the error data from the *second* codec's conversion attempt.

==Numeric Ranges==
The <code>Codec.intRange(min,max)</code>, <code>Codec.floatRange(min,max)</code>, and <code>Codec.doubleRange(min,max)</code> static methods generate Codecs for Integers, Floats, or Doubles, respectively, for which only a specified inclusive range is valid, and values outside that range will fail to de/serialize.

==Maps==
Suppose we want to serialize a HashMap or other Map type, where we could have indefinitely many key-value pairs and we don't know what the keys are ahead of time.

We can create a <code>Codec<Map<KEY,VALUE>></code> using the <code>Codec.unboundedMap</code> static method, which takes a key codec and a value codec and creates a codec for a map type:

<syntaxhighlight lang="java">
public static final Codec<Map<String, BlockPos>> = Codec.unboundedMap(Codec.STRING, BlockPos.CODEC);
</syntaxhighlight>

The serialized form of maps serialized by this codec will be a JsonObject or CompoundNBT, whose fields are the key-value pairs in the map.

A limitation of using unboundedMap is that it only supports key codecs that serialize to Strings (including codecs for things like ResourceLocation that aren't Strings themselves but still serialize to strings). To create a codec for a Map whose keys are not fundamentally strings, the Map must be serialized as a list of key-value entries instead of using unboundedMap. As Codec.pair() also requires that the first codec serialize to a string, the pair function cannot be used for this purpose either.

==Equivalent Types and xmap==
Suppose we have two java classes, Amalgam and Box; any Amalgam instance can be converted to a Box, and vice-versa. Now suppose we have a Codec<Amalgam>, but we'd also like to have a Codec<Box>. Rather than creating an entirely new codec for Box from scratch, we can simply xmap our Amalgam codec instead.

The <code>Codec#xmap</code> instance method is used to generate a second codec for a fundamentally equivalent type to the first codec's type. The method takes two function objects as arguments, which are used to convert the first type to the second when deserializing, and converting the second type to the first when serializing:

<syntaxhighlight lang="java">
public static final Codec<Box> = Amalgam.CODEC.xmap(Amalgam::toBox, Box::toAmalgam);
</syntaxhighlight>

Codecs created in this manner will serialize objects in the same format as the starting codec.

==Partially Equivalent Types, flatComapMap, comapFlatMap, and flatXMap==
Consider the ResourceLocation: Any ResourceLocation can be converted to a String, but not all Strings can be converted to a ResourceLocation; ResourceLocations have strict limits on their format and allowed characters.

While we *could* use xmap to convert the Codec.STRING to a codec for ResourceLocations, this would cause attempts to parse an invalid string like <code>SHOUTY:MOD:Invalid$Characters</code> to throw a runtime exception, when we really should be returning a failed DataResult to the parser instead -- which indeed is what the vanilla ResourceLocation codec does.

Codecs have three additional instance methods for creating equivalent codecs for when we can *potentially* convert one type to another, but are not guaranteed to be able to do so. These take conversion function arguments that return DataResults, allowing validation to be performed during serialization and deserialization.

{| class="wikitable"
|+ Codec Conversion Methods
|-
! Can A always be converted to B? !! Can B always be converted to A? !! Which method of codecA should be used to create codecB?
|-
| yes || yes || codecA.xmap
|-
| yes || no || codecA.flatComapMap
|-
| no || yes || codecA.comapFlatMap
|-
| no || no || codecA.flatXmap
|}

== Registry Dispatch ==
Registry Dispatch Codecs allow us to define a registry of codecs and delegate to a specific codec to deserialize a particular json based on a type field in that json. Dispatch codecs are used extensively when deserializing worldgen data.

To create a dispatch codec for a Thing class, the following steps can be performed:
# Create a Thing abstract class class and ThingType interface. The ThingType interface should have a method that supplies a Codec<Thing>, while Thing subclasses must define a method that supplies a ThingType.
# Create a map or registry of ThingTypes, and register a ThingType for each sub-codec we want to have.
# Create a Codec<ThingType>, or have the ThingType registry implement Codec.
# Create our Codec<Thing> master codec by invoking <code>Codec#dispatch</code> on our ThingType codec. This method's arguments are:
## A field name for the ID of the sub-codec (the example json below is using "type")
## A function to retrieve a ThingType from a Thing
## A function to retrieve a Codec<Thing> from a ThingType

We can then use our Codec<Thing> to create Thing fields in other codecs whose serialized format depends on the specific sub-codec used by a Thing instance.

As an example of this, consider the ExampleCodecClass earlier. Suppose we make this class extend Thing and register our codec for it to a codec dispatch registry with the id "ourmod:exampleclass". If we were to define an instance of this class in a Thing field in some json, it would look like

<syntaxhighlight lang="json">
"some_thing":
{
"type": "ourmod:exampleclass",
"some_int": 42,
"item": "minecraft:gold_ingot",
"block_positions":
[
[0,0,0],
[10,20,-100]
]
}
</syntaxhighlight>

Other ThingTypes we register would have different fields in this json object, but would still be valid for the "some_thing" field.

Several examples of vanilla classes that use dispatch codecs:
* RuleTest and IRuleTestType
* IParticleData and ParticleType
* ConfiguredPlacement and Placement

=External Links=
* [https://github.com/Mojang/DataFixerUpper/blob/master/src/main/java/com/mojang/serialization/Codec.java Codecs in Mojang's official public DataFixerUpper repository]
* [https://kvverti.github.io/Documented-DataFixerUpper/snapshot/com/mojang/serialization/Codec.html#flatXmap-java.util.function.Function-java.util.function.Function- Unofficial Codec Javadocs]

Codecs

2021-01-06T00:09:10Z

Commoble: Corrected valid arguments for pair codecs

Codecs are a serialization tool from mojang's DataFixerUpper library. Codecs are used alongside [[DynamicOps|DynamicOps]] to allow objects to be serialized to different formats and back, such as JSON or [[Using_NBT|NBT]]. While the DynamicOps describes the format the object is to be serialized to, the Codec describes the manner in which the object is to be serialized; a single Codec can be used to serialize an object to any format for which a DynamicOps exists. Codecs and DynamicOps jointly form an abstraction layer over data serialization, simplifying the effort needed to serialize or deserialize data.

= Using Codecs =

== Serialization and Deserialization ==

The primary use for Codecs is to serialize java objects to some serialized type, such as a JsonElement or an INBT, and to deserialize an serialized object back to its proper java type. This is accomplished with <code>Codec#encodeStart</code> and <code>Codec#parse</code>, respectively. Given a Codec<SomeJavaType> and a DynamicOps<SomeSerializedType>, we can convert instances of SomeJavaType to instances of SomeSerializedType and back.

Each of these methods take a [[DynamicOps]] instance and an instance of the object we are serializing or deserializing, and returns a DataResult:

<syntaxhighlight lang="java">
// let someCodec be a Codec<SomeJavaType>
// let someJavaObject be an instance of SomeJavaType
// let someNBT and someJsonElement be instances of INBT and JsonElement, respectively

// serialize some java object to INBT
DataResult<INBT> result = someCodec.encodeStart(NBTDynamicOps.INSTANCE, someJavaObject);

// deserialize some INBT instance back to a proper java object
DataResult<SomeJavaType> result = someCodec.parse(NBTDynamicOps.INSTANCE, someNBT);

// serialize some java object to a JsonElement
DataResult<JsonElement> result = someCodec.encodeStart(JsonOps.INSTANCE, someJavaObject);

// deserialize a JsonElement back to a proper java object
DataResult<SomeJavaType> result = someCodec.parse(JsonOps.INSTANCE, someJsonElement);
</syntaxhighlight>

A DataResult either holds the converted instance, or it holds some error data, depending on whether the conversion was successful or not, respectively. There are several things we can do with this DataResult; <code>DataResult#result</code> simply returns an Optional containing the converted object if the conversion was successful, while <code>DataResult#resultOrPartial</code> also runs a given function if the conversion was unsuccessful (in addition to returning the Optional). #resultOrPartial is particularly useful for logging errors during datapack deserialization:

<syntaxhighlight lang="java">
// deserialize something from json
someCodec.parse(JsonOps.INSTANCE, someJsonElement)
.resultOrPartial(errorMessage -> doSomethingIfBadData(errorMessage))
.ifPresent(someJavaObject -> doSomethingIfGoodData(someJavaObject))
</syntaxhighlight>

== Builtin Codecs ==
=== Primitive Codecs ===
The Codec class itself contains static instances of codecs for all supported primitive types, e.g. <code>Codec.STRING</code> is the canonical <code>Codec<String></code> implementation. Primitive codecs include:
* BOOL, which serializes to a boolean value
* BYTE, SHORT, INT, LONG, FLOAT, and DOUBLE, which serialize to numeric values
* STRING, which serializes to a string
* BYTE_BUFFER, INT_STREAM, and LONG_STREAM, which serialize to lists of numbers
* EMPTY, which represents null objects

=== Other Builtin Codecs ===
Vanilla minecraft has many builtin codecs for objects that it frequently serializes. These codecs are typically static instances in the class the codec is serializing; e.g. <code>ResourceLocation.CODEC</code> is the canonical <code>Codec<ResourceLocation></code>, while <code>BlockPos.CODEC</code> is the codec used for serializing a BlockPos.

Each vanilla <code>Registry</code> acts as the Codec for the type of object the registry contains; e.g. <code>Registry.BLOCK</code> is itself a <code>Codec<Block></code>. Forge Registries, however, do not currently implement Codec and cannot yet be used in this way; custom codecs must be created for forge-specific registries that are not tied to specific vanilla registries.

Of particular note here is the CompoundNBT.CODEC, which can be used to e.g. serialize a CompoundNBT into a json file. This has a notable limitation in that CompoundNBT.CODEC *cannot* safely deserialize lists of numbers from json, due to the strong typing of ListNBT and the way that the NBTDynamicOps deserializer reads numeric values.

= Creating Codecs =

Suppose we have the following class, and we want to deserialize json files to instances of this class:
<syntaxhighlight lang="java">
public class ExampleCodecClass {

private final int someInt;
private final Item item;
private final List<BlockPos> blockPositions;

public ExampleCodecClass(int someInt, Item item, List<BlockPos> blockPositions) {...}

public int getSomeInt() { return this.someInt; }
public Item getItem() { return this.someItem; }
public List<BlockPos> getBlockPositions() { return this.blockPositions; }
}
</syntaxhighlight>

Where a json file for an instance of this class might look like:

<syntaxhighlight lang="json">
{
"some_int": 42,
"item": "minecraft:gold_ingot",
"block_positions":
[
[0,0,0],
[10,20,-100]
]
}
</syntaxhighlight>

We can assemble a codec for this class by building a new codec out of smaller codecs. We'll need a codec for each of these fields:
* a <code>Codec<Integer></code>
* a <code>Codec<Item></code>
* a <code>Codec<List<BlockPos>></code>
And then we'll need to assemble these into a <code>Codec<ExampleCodecClass</code>.

As previously mentioned, we can use <code>Codec.INT</code> for the integer codec, and <code>Registry.ITEM</code> for the Item codec. We don't have a builtin codec for list-of-blockpos, but we can use BlockPos.CODEC to create one.

== Lists ==
The <code>Codec#listOf</code> instance method can be used to generate a codec for a List from an existing codec:

<syntaxhighlight lang="java">
// BlockPos.CODEC is a Codec<BlockPos>
Codec<List<BlockPos>> = BlockPos.CODEC.listOf();
</syntaxhighlight>

Codecs created via listOf() serialize things to listlike objects, such as [] json arrays or ListNBTs.

== Records ==
RecordCodecBuilder is used to generate codecs that serialize instances of classes with explicitly named fields, like our example above. Codecs created via RecordCodecBuilder serialize things to maplike objects, such as {} json objects or CompoundNBTs.

RecordCodecBuilder can be used in several ways, but the simplest form is as follows:

<syntaxhighlight lang="java">
public static final Codec<SomeJavaClass> = RecordCodecBuilder.create(instance -> instance.group(
someFieldCodecA.fieldOf("field_name_a").forGetter(SomeJavaClass::getFieldA),
someFieldCodecB.fieldOf("field_name_b").forGetter(SomeJavaClass::getFieldB),
someFieldCodecC.fieldOf("field_name_c").forGetter(SomeJavaClass::getFieldC),
// up to 16 fields can be declared here
).apply(instance, SomeJavaClass::new));
</syntaxhighlight>

Where each line in the group specifies a codec instance for the type of that field, the field name in the serialized object, and the corresponding getter function in the java class. The builder is concluded by specifying a constructor or factory for the java class whose arguments are the previously defined fields in the same order.

For example, using RecordCodecBuilder to create a codec for our example class above:
<syntaxhighlight lang="java">
public static final Codec<ExampleCodecClass> = RecordCodecBuilder.create(instance -> instance.group(
Codec.INT.fieldOf("some_int").forGetter(ExampleCodecClass::getSomeInt),
Registry.ITEM.fieldOf("item").forGetter(ExampleCodecClass::getItem),
BlockPos.CODEC.listOf().fieldOf("block_positions").forGetter(ExampleCodecClass::getBlockPositions)
).apply(instance, ExampleCodecClass::new));
</syntaxhighlight>

===Optional and Default Values in Record Fields===
When RecordCodecBuilder is used as shown above, all of the fields are *required* to be in the serialized object (the JsonObject/CompoundNBT/etc), or the entire thing will fail to parse when the codec tries to deserialize it. If we wish to have optional or default values, we have several alternatives of fieldOf() we can use.

* <code>someCodec.optionalFieldOf("field_name")</code> creates a field for an Optional. If the field in the json/nbt is not present or invalid, it will deserialize as an empty optional. Empty optionals will not be serialized; the field will be omitted from the json or nbt.
* <code>someCodec.optionalFieldOf("field_name", someDefaultValue)</code> creates an optional field that deserializes as the given default value if the field is not present in the json/nbt. When serializing, if the field in the java object equals the default value, the value will not be serialized and the field will be omitted from the json or nbt.

When using optional fields, be wary that if the field contains bad data or otherwise fails to serialize, the error will be silently caught, and the field will serialize as the default value instead!

==Unit==
The <code>Codec.unit(defaultValue)</code> codec creates a Codec that always deserializes a specified default value, regardless of input. When serializing, it serializes nothing.

==Pair==
The <code>Codec.pair(codecA, codecB)</code> static method takes two codecs and generates a Codec<Pair<A,B>> from them.

The only valid arguments for this method are codecs that serialize to objects, such as codecs created using [[Codecs#Records|RecordCodecBuilder]], [[Codecs#Maps|unboundedMap]], or fieldOf. Codecs that serialize nothing (such as [[Codecs#Unit|unit codecs]]) are also valid as they act as objects-with-no-fields.

The resulting Pair codec will serialize a single object that has all of the fields of the two original codecs.

==Either==
The <code>Codec.either(codecA, codecB)</code> static method takes two codecs and generates a Codec<Either<A,B>> from them.

When this codec is used to de/serialize an object, it first attempts to use the first codec; if and only if that conversion fails, it then attempts to use the second codec. If that conversion also fails, then the returned DataResult will contain the error data from the *second* codec's conversion attempt.

==Numeric Ranges==
The <code>Codec.intRange(min,max)</code>, <code>Codec.floatRange(min,max)</code>, and <code>Codec.doubleRange(min,max)</code> static methods generate Codecs for Integers, Floats, or Doubles, respectively, for which only a specified inclusive range is valid, and values outside that range will fail to de/serialize.

==Maps==
Suppose we want to serialize a HashMap or other Map type, where we could have indefinitely many key-value pairs and we don't know what the keys are ahead of time.

We can create a <code>Codec<Map<KEY,VALUE>></code> using the <code>Codec.unboundedMap</code> static method, which takes a key codec and a value codec and creates a codec for a map type:

<syntaxhighlight lang="java">
public static final Codec<Map<String, BlockPos>> = Codec.unboundedMap(Codec.STRING, BlockPos.CODEC);
</syntaxhighlight>

The serialized form of maps serialized by this codec will be a JsonObject or CompoundNBT, whose fields are the key-value pairs in the map.

A limitation of using unboundedMap is that it only supports key codecs that serialize to Strings (including codecs for things like ResourceLocation that aren't Strings themselves but still serialize to strings). To create a codec for a Map whose keys are not fundamentally strings, the Map must be serialized as a list of key-value entries instead of using unboundedMap. As Codec.pair() also requires that the first codec serialize to a string, the pair function cannot be used for this purpose either.

==Equivalent Types and xmap==
Suppose we have two java classes, Amalgam and Box; any Amalgam instance can be converted to a Box, and vice-versa. Now suppose we have a Codec<Amalgam>, but we'd also like to have a Codec<Box>. Rather than creating an entirely new codec for Box from scratch, we can simply xmap our Amalgam codec instead.

The <code>Codec#xmap</code> instance method is used to generate a second codec for a fundamentally equivalent type to the first codec's type. The method takes two function objects as arguments, which are used to convert the first type to the second when deserializing, and converting the second type to the first when serializing:

<syntaxhighlight lang="java">
public static final Codec<Box> = Amalgam.CODEC.xmap(Amalgam::toBox, Box::toAmalgam);
</syntaxhighlight>

Codecs created in this manner will serialize objects in the same format as the starting codec.

==Partially Equivalent Types, flatComapMap, comapFlatMap, and flatXMap==
Consider the ResourceLocation: Any ResourceLocation can be converted to a String, but not all Strings can be converted to a ResourceLocation; ResourceLocations have strict limits on their format and allowed characters.

While we *could* use xmap to convert the Codec.STRING to a codec for ResourceLocations, this would cause attempts to parse an invalid string like <code>SHOUTY:MOD:Invalid$Characters</code> to throw a runtime exception, when we really should be returning a failed DataResult to the parser instead -- which indeed is what the vanilla ResourceLocation codec does.

Codecs have three additional instance methods for creating equivalent codecs for when we can *potentially* convert one type to another, but are not guaranteed to be able to do so. These take conversion function arguments that return DataResults, allowing validation to be performed during serialization and deserialization.

{| class="wikitable"
|+ Codec Conversion Methods
|-
! Can A always be converted to B? !! Can B always be converted to A? !! Which method of codecA should be used to create codecB?
|-
| yes || yes || codecA.xmap
|-
| yes || no || codecA.flatComapMap
|-
| no || yes || codecA.comapFlatMap
|-
| no || no || codecA.flatXmap
|}

== Registry Dispatch ==
Registry Dispatch Codecs allow us to define a registry of codecs and delegate to a specific codec to deserialize a particular json based on a type field in that json. Dispatch codecs are used extensively when deserializing worldgen data.

To create a dispatch codec for a Thing class, the following steps can be performed:
# Create a Thing abstract class class and ThingType interface. The ThingType interface should have a method that supplies a Codec<Thing>, while Thing subclasses must define a method that supplies a ThingType.
# Create a map or registry of ThingTypes, and register a ThingType for each sub-codec we want to have.
# Create a Codec<ThingType>, or have the ThingType registry implement Codec.
# Create our Codec<Thing> master codec by invoking <code>Codec#dispatch</code> on our ThingType codec. This method's arguments are:
## A field name for the ID of the sub-codec (the example json below is using "type")
## A function to retrieve a ThingType from a Thing
## A function to retrieve a Codec<Thing> from a ThingType

We can then use our Codec<Thing> to create Thing fields in other codecs whose serialized format depends on the specific sub-codec used by a Thing instance.

As an example of this, consider the ExampleCodecClass earlier. Suppose we make this class extend Thing and register our codec for it to a codec dispatch registry with the id "ourmod:exampleclass". If we were to define an instance of this class in a Thing field in some json, it would look like

<syntaxhighlight lang="json">
"some_thing":
{
"type": "ourmod:exampleclass",
"some_int": 42,
"item": "minecraft:gold_ingot",
"block_positions":
[
[0,0,0],
[10,20,-100]
]
}
</syntaxhighlight>

Other ThingTypes we register would have different fields in this json object, but would still be valid for the "some_thing" field.

Several examples of vanilla classes that use dispatch codecs:
* RuleTest and IRuleTestType
* IParticleData and ParticleType
* ConfiguredPlacement and Placement

=External Links=
* [https://github.com/Mojang/DataFixerUpper/blob/master/src/main/java/com/mojang/serialization/Codec.java Codecs in Mojang's official public DataFixerUpper repository]
* [https://kvverti.github.io/Documented-DataFixerUpper/snapshot/com/mojang/serialization/Codec.html#flatXmap-java.util.function.Function-java.util.function.Function- Unofficial Codec Javadocs]

Codecs

2021-01-04T23:14:33Z

Commoble: Add link to NBT page

Codecs are a serialization tool from mojang's DataFixerUpper library. Codecs are used alongside [[DynamicOps|DynamicOps]] to allow objects to be serialized to different formats and back, such as JSON or [[Using_NBT|NBT]]. While the DynamicOps describes the format the object is to be serialized to, the Codec describes the manner in which the object is to be serialized; a single Codec can be used to serialize an object to any format for which a DynamicOps exists. Codecs and DynamicOps jointly form an abstraction layer over data serialization, simplifying the effort needed to serialize or deserialize data.

= Using Codecs =

== Serialization and Deserialization ==

The primary use for Codecs is to serialize java objects to some serialized type, such as a JsonElement or an INBT, and to deserialize an serialized object back to its proper java type. This is accomplished with <code>Codec#encodeStart</code> and <code>Codec#parse</code>, respectively. Given a Codec<SomeJavaType> and a DynamicOps<SomeSerializedType>, we can convert instances of SomeJavaType to instances of SomeSerializedType and back.

Each of these methods take a [[DynamicOps]] instance and an instance of the object we are serializing or deserializing, and returns a DataResult:

<syntaxhighlight lang="java">
// let someCodec be a Codec<SomeJavaType>
// let someJavaObject be an instance of SomeJavaType
// let someNBT and someJsonElement be instances of INBT and JsonElement, respectively

// serialize some java object to INBT
DataResult<INBT> result = someCodec.encodeStart(NBTDynamicOps.INSTANCE, someJavaObject);

// deserialize some INBT instance back to a proper java object
DataResult<SomeJavaType> result = someCodec.parse(NBTDynamicOps.INSTANCE, someNBT);

// serialize some java object to a JsonElement
DataResult<JsonElement> result = someCodec.encodeStart(JsonOps.INSTANCE, someJavaObject);

// deserialize a JsonElement back to a proper java object
DataResult<SomeJavaType> result = someCodec.parse(JsonOps.INSTANCE, someJsonElement);
</syntaxhighlight>

A DataResult either holds the converted instance, or it holds some error data, depending on whether the conversion was successful or not, respectively. There are several things we can do with this DataResult; <code>DataResult#result</code> simply returns an Optional containing the converted object if the conversion was successful, while <code>DataResult#resultOrPartial</code> also runs a given function if the conversion was unsuccessful (in addition to returning the Optional). #resultOrPartial is particularly useful for logging errors during datapack deserialization:

<syntaxhighlight lang="java">
// deserialize something from json
someCodec.parse(JsonOps.INSTANCE, someJsonElement)
.resultOrPartial(errorMessage -> doSomethingIfBadData(errorMessage))
.ifPresent(someJavaObject -> doSomethingIfGoodData(someJavaObject))
</syntaxhighlight>

== Builtin Codecs ==
=== Primitive Codecs ===
The Codec class itself contains static instances of codecs for all supported primitive types, e.g. <code>Codec.STRING</code> is the canonical <code>Codec<String></code> implementation. Primitive codecs include:
* BOOL, which serializes to a boolean value
* BYTE, SHORT, INT, LONG, FLOAT, and DOUBLE, which serialize to numeric values
* STRING, which serializes to a string
* BYTE_BUFFER, INT_STREAM, and LONG_STREAM, which serialize to lists of numbers
* EMPTY, which represents null objects

=== Other Builtin Codecs ===
Vanilla minecraft has many builtin codecs for objects that it frequently serializes. These codecs are typically static instances in the class the codec is serializing; e.g. <code>ResourceLocation.CODEC</code> is the canonical <code>Codec<ResourceLocation></code>, while <code>BlockPos.CODEC</code> is the codec used for serializing a BlockPos.

Each vanilla <code>Registry</code> acts as the Codec for the type of object the registry contains; e.g. <code>Registry.BLOCK</code> is itself a <code>Codec<Block></code>. Forge Registries, however, do not currently implement Codec and cannot yet be used in this way; custom codecs must be created for forge-specific registries that are not tied to specific vanilla registries.

Of particular note here is the CompoundNBT.CODEC, which can be used to e.g. serialize a CompoundNBT into a json file. This has a notable limitation in that CompoundNBT.CODEC *cannot* safely deserialize lists of numbers from json, due to the strong typing of ListNBT and the way that the NBTDynamicOps deserializer reads numeric values.

= Creating Codecs =

Suppose we have the following class, and we want to deserialize json files to instances of this class:
<syntaxhighlight lang="java">
public class ExampleCodecClass {

private final int someInt;
private final Item item;
private final List<BlockPos> blockPositions;

public ExampleCodecClass(int someInt, Item item, List<BlockPos> blockPositions) {...}

public int getSomeInt() { return this.someInt; }
public Item getItem() { return this.someItem; }
public List<BlockPos> getBlockPositions() { return this.blockPositions; }
}
</syntaxhighlight>

Where a json file for an instance of this class might look like:

<syntaxhighlight lang="json">
{
"some_int": 42,
"item": "minecraft:gold_ingot",
"block_positions":
[
[0,0,0],
[10,20,-100]
]
}
</syntaxhighlight>

We can assemble a codec for this class by building a new codec out of smaller codecs. We'll need a codec for each of these fields:
* a <code>Codec<Integer></code>
* a <code>Codec<Item></code>
* a <code>Codec<List<BlockPos>></code>
And then we'll need to assemble these into a <code>Codec<ExampleCodecClass</code>.

As previously mentioned, we can use <code>Codec.INT</code> for the integer codec, and <code>Registry.ITEM</code> for the Item codec. We don't have a builtin codec for list-of-blockpos, but we can use BlockPos.CODEC to create one.

== Lists ==
The <code>Codec#listOf</code> instance method can be used to generate a codec for a List from an existing codec:

<syntaxhighlight lang="java">
// BlockPos.CODEC is a Codec<BlockPos>
Codec<List<BlockPos>> = BlockPos.CODEC.listOf();
</syntaxhighlight>

Codecs created via listOf() serialize things to listlike objects, such as [] json arrays or ListNBTs.

== Records ==
RecordCodecBuilder is used to generate codecs that serialize instances of classes with explicitly named fields, like our example above. Codecs created via RecordCodecBuilder serialize things to maplike objects, such as {} json objects or CompoundNBTs.

RecordCodecBuilder can be used in several ways, but the simplest form is as follows:

<syntaxhighlight lang="java">
public static final Codec<SomeJavaClass> = RecordCodecBuilder.create(instance -> instance.group(
someFieldCodecA.fieldOf("field_name_a").forGetter(SomeJavaClass::getFieldA),
someFieldCodecB.fieldOf("field_name_b").forGetter(SomeJavaClass::getFieldB),
someFieldCodecC.fieldOf("field_name_c").forGetter(SomeJavaClass::getFieldC),
// up to 16 fields can be declared here
).apply(instance, SomeJavaClass::new));
</syntaxhighlight>

Where each line in the group specifies a codec instance for the type of that field, the field name in the serialized object, and the corresponding getter function in the java class. The builder is concluded by specifying a constructor or factory for the java class whose arguments are the previously defined fields in the same order.

For example, using RecordCodecBuilder to create a codec for our example class above:
<syntaxhighlight lang="java">
public static final Codec<ExampleCodecClass> = RecordCodecBuilder.create(instance -> instance.group(
Codec.INT.fieldOf("some_int").forGetter(ExampleCodecClass::getSomeInt),
Registry.ITEM.fieldOf("item").forGetter(ExampleCodecClass::getItem),
BlockPos.CODEC.listOf().fieldOf("block_positions").forGetter(ExampleCodecClass::getBlockPositions)
).apply(instance, ExampleCodecClass::new));
</syntaxhighlight>

===Optional and Default Values in Record Fields===
When RecordCodecBuilder is used as shown above, all of the fields are *required* to be in the serialized object (the JsonObject/CompoundNBT/etc), or the entire thing will fail to parse when the codec tries to deserialize it. If we wish to have optional or default values, we have several alternatives of fieldOf() we can use.

* <code>someCodec.optionalFieldOf("field_name")</code> creates a field for an Optional. If the field in the json/nbt is not present or invalid, it will deserialize as an empty optional. Empty optionals will not be serialized; the field will be omitted from the json or nbt.
* <code>someCodec.optionalFieldOf("field_name", someDefaultValue)</code> creates an optional field that deserializes as the given default value if the field is not present in the json/nbt. When serializing, if the field in the java object equals the default value, the value will not be serialized and the field will be omitted from the json or nbt.

When using optional fields, be wary that if the field contains bad data or otherwise fails to serialize, the error will be silently caught, and the field will serialize as the default value instead!

==Unit==
The <code>Codec.unit(defaultValue)</code> codec creates a Codec that always deserializes a specified default value, regardless of input. When serializing, it serializes nothing.

==Pair==
The <code>Codec.pair(codecA, codecB)</code> static method takes two codecs and generates a Codec<Pair<A,B>> from them.

Using this function requires that codecA be a codec that serializes to a string.

==Either==
The <code>Codec.either(codecA, codecB)</code> static method takes two codecs and generates a Codec<Either<A,B>> from them.

When this codec is used to de/serialize an object, it first attempts to use the first codec; if and only if that conversion fails, it then attempts to use the second codec. If that conversion also fails, then the returned DataResult will contain the error data from the *second* codec's conversion attempt.

==Numeric Ranges==
The <code>Codec.intRange(min,max)</code>, <code>Codec.floatRange(min,max)</code>, and <code>Codec.doubleRange(min,max)</code> static methods generate Codecs for Integers, Floats, or Doubles, respectively, for which only a specified inclusive range is valid, and values outside that range will fail to de/serialize.

==Maps==
Suppose we want to serialize a HashMap or other Map type, where we could have indefinitely many key-value pairs and we don't know what the keys are ahead of time.

We can create a <code>Codec<Map<KEY,VALUE>></code> using the <code>Codec.unboundedMap</code> static method, which takes a key codec and a value codec and creates a codec for a map type:

<syntaxhighlight lang="java">
public static final Codec<Map<String, BlockPos>> = Codec.unboundedMap(Codec.STRING, BlockPos.CODEC);
</syntaxhighlight>

The serialized form of maps serialized by this codec will be a JsonObject or CompoundNBT, whose fields are the key-value pairs in the map.

A limitation of using unboundedMap is that it only supports key codecs that serialize to Strings (including codecs for things like ResourceLocation that aren't Strings themselves but still serialize to strings). To create a codec for a Map whose keys are not fundamentally strings, the Map must be serialized as a list of key-value entries instead of using unboundedMap. As Codec.pair() also requires that the first codec serialize to a string, the pair function cannot be used for this purpose either.

==Equivalent Types and xmap==
Suppose we have two java classes, Amalgam and Box; any Amalgam instance can be converted to a Box, and vice-versa. Now suppose we have a Codec<Amalgam>, but we'd also like to have a Codec<Box>. Rather than creating an entirely new codec for Box from scratch, we can simply xmap our Amalgam codec instead.

The <code>Codec#xmap</code> instance method is used to generate a second codec for a fundamentally equivalent type to the first codec's type. The method takes two function objects as arguments, which are used to convert the first type to the second when deserializing, and converting the second type to the first when serializing:

<syntaxhighlight lang="java">
public static final Codec<Box> = Amalgam.CODEC.xmap(Amalgam::toBox, Box::toAmalgam);
</syntaxhighlight>

Codecs created in this manner will serialize objects in the same format as the starting codec.

==Partially Equivalent Types, flatComapMap, comapFlatMap, and flatXMap==
Consider the ResourceLocation: Any ResourceLocation can be converted to a String, but not all Strings can be converted to a ResourceLocation; ResourceLocations have strict limits on their format and allowed characters.

While we *could* use xmap to convert the Codec.STRING to a codec for ResourceLocations, this would cause attempts to parse an invalid string like <code>SHOUTY:MOD:Invalid$Characters</code> to throw a runtime exception, when we really should be returning a failed DataResult to the parser instead -- which indeed is what the vanilla ResourceLocation codec does.

Codecs have three additional instance methods for creating equivalent codecs for when we can *potentially* convert one type to another, but are not guaranteed to be able to do so. These take conversion function arguments that return DataResults, allowing validation to be performed during serialization and deserialization.

{| class="wikitable"
|+ Codec Conversion Methods
|-
! Can A always be converted to B? !! Can B always be converted to A? !! Which method of codecA should be used to create codecB?
|-
| yes || yes || codecA.xmap
|-
| yes || no || codecA.flatComapMap
|-
| no || yes || codecA.comapFlatMap
|-
| no || no || codecA.flatXmap
|}

== Registry Dispatch ==
Registry Dispatch Codecs allow us to define a registry of codecs and delegate to a specific codec to deserialize a particular json based on a type field in that json. Dispatch codecs are used extensively when deserializing worldgen data.

To create a dispatch codec for a Thing class, the following steps can be performed:
# Create a Thing abstract class class and ThingType interface. The ThingType interface should have a method that supplies a Codec<Thing>, while Thing subclasses must define a method that supplies a ThingType.
# Create a map or registry of ThingTypes, and register a ThingType for each sub-codec we want to have.
# Create a Codec<ThingType>, or have the ThingType registry implement Codec.
# Create our Codec<Thing> master codec by invoking <code>Codec#dispatch</code> on our ThingType codec. This method's arguments are:
## A field name for the ID of the sub-codec (the example json below is using "type")
## A function to retrieve a ThingType from a Thing
## A function to retrieve a Codec<Thing> from a ThingType

We can then use our Codec<Thing> to create Thing fields in other codecs whose serialized format depends on the specific sub-codec used by a Thing instance.

As an example of this, consider the ExampleCodecClass earlier. Suppose we make this class extend Thing and register our codec for it to a codec dispatch registry with the id "ourmod:exampleclass". If we were to define an instance of this class in a Thing field in some json, it would look like

<syntaxhighlight lang="json">
"some_thing":
{
"type": "ourmod:exampleclass",
"some_int": 42,
"item": "minecraft:gold_ingot",
"block_positions":
[
[0,0,0],
[10,20,-100]
]
}
</syntaxhighlight>

Other ThingTypes we register would have different fields in this json object, but would still be valid for the "some_thing" field.

Several examples of vanilla classes that use dispatch codecs:
* RuleTest and IRuleTestType
* IParticleData and ParticleType
* ConfiguredPlacement and Placement

=External Links=
* [https://github.com/Mojang/DataFixerUpper/blob/master/src/main/java/com/mojang/serialization/Codec.java Codecs in Mojang's official public DataFixerUpper repository]
* [https://kvverti.github.io/Documented-DataFixerUpper/snapshot/com/mojang/serialization/Codec.html#flatXmap-java.util.function.Function-java.util.function.Function- Unofficial Codec Javadocs]

Codecs

2021-01-04T23:10:02Z

Commoble: Clarify numeric range methods

Codecs are a serialization tool from mojang's DataFixerUpper library. Codecs are used alongside [[DynamicOps|DynamicOps]] to allow objects to be serialized to different formats and back, such as JSON or NBT. While the DynamicOps describes the format the object is to be serialized to, the Codec describes the manner in which the object is to be serialized; a single Codec can be used to serialize an object to any format for which a DynamicOps exists. Codecs and DynamicOps jointly form an abstraction layer over data serialization, simplifying the effort needed to serialize or deserialize data.

= Using Codecs =

== Serialization and Deserialization ==

The primary use for Codecs is to serialize java objects to some serialized type, such as a JsonElement or an INBT, and to deserialize an serialized object back to its proper java type. This is accomplished with <code>Codec#encodeStart</code> and <code>Codec#parse</code>, respectively. Given a Codec<SomeJavaType> and a DynamicOps<SomeSerializedType>, we can convert instances of SomeJavaType to instances of SomeSerializedType and back.

Each of these methods take a [[DynamicOps]] instance and an instance of the object we are serializing or deserializing, and returns a DataResult:

<syntaxhighlight lang="java">
// let someCodec be a Codec<SomeJavaType>
// let someJavaObject be an instance of SomeJavaType
// let someNBT and someJsonElement be instances of INBT and JsonElement, respectively

// serialize some java object to INBT
DataResult<INBT> result = someCodec.encodeStart(NBTDynamicOps.INSTANCE, someJavaObject);

// deserialize some INBT instance back to a proper java object
DataResult<SomeJavaType> result = someCodec.parse(NBTDynamicOps.INSTANCE, someNBT);

// serialize some java object to a JsonElement
DataResult<JsonElement> result = someCodec.encodeStart(JsonOps.INSTANCE, someJavaObject);

// deserialize a JsonElement back to a proper java object
DataResult<SomeJavaType> result = someCodec.parse(JsonOps.INSTANCE, someJsonElement);
</syntaxhighlight>

A DataResult either holds the converted instance, or it holds some error data, depending on whether the conversion was successful or not, respectively. There are several things we can do with this DataResult; <code>DataResult#result</code> simply returns an Optional containing the converted object if the conversion was successful, while <code>DataResult#resultOrPartial</code> also runs a given function if the conversion was unsuccessful (in addition to returning the Optional). #resultOrPartial is particularly useful for logging errors during datapack deserialization:

<syntaxhighlight lang="java">
// deserialize something from json
someCodec.parse(JsonOps.INSTANCE, someJsonElement)
.resultOrPartial(errorMessage -> doSomethingIfBadData(errorMessage))
.ifPresent(someJavaObject -> doSomethingIfGoodData(someJavaObject))
</syntaxhighlight>

== Builtin Codecs ==
=== Primitive Codecs ===
The Codec class itself contains static instances of codecs for all supported primitive types, e.g. <code>Codec.STRING</code> is the canonical <code>Codec<String></code> implementation. Primitive codecs include:
* BOOL, which serializes to a boolean value
* BYTE, SHORT, INT, LONG, FLOAT, and DOUBLE, which serialize to numeric values
* STRING, which serializes to a string
* BYTE_BUFFER, INT_STREAM, and LONG_STREAM, which serialize to lists of numbers
* EMPTY, which represents null objects

=== Other Builtin Codecs ===
Vanilla minecraft has many builtin codecs for objects that it frequently serializes. These codecs are typically static instances in the class the codec is serializing; e.g. <code>ResourceLocation.CODEC</code> is the canonical <code>Codec<ResourceLocation></code>, while <code>BlockPos.CODEC</code> is the codec used for serializing a BlockPos.

Each vanilla <code>Registry</code> acts as the Codec for the type of object the registry contains; e.g. <code>Registry.BLOCK</code> is itself a <code>Codec<Block></code>. Forge Registries, however, do not currently implement Codec and cannot yet be used in this way; custom codecs must be created for forge-specific registries that are not tied to specific vanilla registries.

Of particular note here is the CompoundNBT.CODEC, which can be used to e.g. serialize a CompoundNBT into a json file. This has a notable limitation in that CompoundNBT.CODEC *cannot* safely deserialize lists of numbers from json, due to the strong typing of ListNBT and the way that the NBTDynamicOps deserializer reads numeric values.

= Creating Codecs =

Suppose we have the following class, and we want to deserialize json files to instances of this class:
<syntaxhighlight lang="java">
public class ExampleCodecClass {

private final int someInt;
private final Item item;
private final List<BlockPos> blockPositions;

public ExampleCodecClass(int someInt, Item item, List<BlockPos> blockPositions) {...}

public int getSomeInt() { return this.someInt; }
public Item getItem() { return this.someItem; }
public List<BlockPos> getBlockPositions() { return this.blockPositions; }
}
</syntaxhighlight>

Where a json file for an instance of this class might look like:

<syntaxhighlight lang="json">
{
"some_int": 42,
"item": "minecraft:gold_ingot",
"block_positions":
[
[0,0,0],
[10,20,-100]
]
}
</syntaxhighlight>

We can assemble a codec for this class by building a new codec out of smaller codecs. We'll need a codec for each of these fields:
* a <code>Codec<Integer></code>
* a <code>Codec<Item></code>
* a <code>Codec<List<BlockPos>></code>
And then we'll need to assemble these into a <code>Codec<ExampleCodecClass</code>.

As previously mentioned, we can use <code>Codec.INT</code> for the integer codec, and <code>Registry.ITEM</code> for the Item codec. We don't have a builtin codec for list-of-blockpos, but we can use BlockPos.CODEC to create one.

== Lists ==
The <code>Codec#listOf</code> instance method can be used to generate a codec for a List from an existing codec:

<syntaxhighlight lang="java">
// BlockPos.CODEC is a Codec<BlockPos>
Codec<List<BlockPos>> = BlockPos.CODEC.listOf();
</syntaxhighlight>

Codecs created via listOf() serialize things to listlike objects, such as [] json arrays or ListNBTs.

== Records ==
RecordCodecBuilder is used to generate codecs that serialize instances of classes with explicitly named fields, like our example above. Codecs created via RecordCodecBuilder serialize things to maplike objects, such as {} json objects or CompoundNBTs.

RecordCodecBuilder can be used in several ways, but the simplest form is as follows:

<syntaxhighlight lang="java">
public static final Codec<SomeJavaClass> = RecordCodecBuilder.create(instance -> instance.group(
someFieldCodecA.fieldOf("field_name_a").forGetter(SomeJavaClass::getFieldA),
someFieldCodecB.fieldOf("field_name_b").forGetter(SomeJavaClass::getFieldB),
someFieldCodecC.fieldOf("field_name_c").forGetter(SomeJavaClass::getFieldC),
// up to 16 fields can be declared here
).apply(instance, SomeJavaClass::new));
</syntaxhighlight>

Where each line in the group specifies a codec instance for the type of that field, the field name in the serialized object, and the corresponding getter function in the java class. The builder is concluded by specifying a constructor or factory for the java class whose arguments are the previously defined fields in the same order.

For example, using RecordCodecBuilder to create a codec for our example class above:
<syntaxhighlight lang="java">
public static final Codec<ExampleCodecClass> = RecordCodecBuilder.create(instance -> instance.group(
Codec.INT.fieldOf("some_int").forGetter(ExampleCodecClass::getSomeInt),
Registry.ITEM.fieldOf("item").forGetter(ExampleCodecClass::getItem),
BlockPos.CODEC.listOf().fieldOf("block_positions").forGetter(ExampleCodecClass::getBlockPositions)
).apply(instance, ExampleCodecClass::new));
</syntaxhighlight>

===Optional and Default Values in Record Fields===
When RecordCodecBuilder is used as shown above, all of the fields are *required* to be in the serialized object (the JsonObject/CompoundNBT/etc), or the entire thing will fail to parse when the codec tries to deserialize it. If we wish to have optional or default values, we have several alternatives of fieldOf() we can use.

* <code>someCodec.optionalFieldOf("field_name")</code> creates a field for an Optional. If the field in the json/nbt is not present or invalid, it will deserialize as an empty optional. Empty optionals will not be serialized; the field will be omitted from the json or nbt.
* <code>someCodec.optionalFieldOf("field_name", someDefaultValue)</code> creates an optional field that deserializes as the given default value if the field is not present in the json/nbt. When serializing, if the field in the java object equals the default value, the value will not be serialized and the field will be omitted from the json or nbt.

When using optional fields, be wary that if the field contains bad data or otherwise fails to serialize, the error will be silently caught, and the field will serialize as the default value instead!

==Unit==
The <code>Codec.unit(defaultValue)</code> codec creates a Codec that always deserializes a specified default value, regardless of input. When serializing, it serializes nothing.

==Pair==
The <code>Codec.pair(codecA, codecB)</code> static method takes two codecs and generates a Codec<Pair<A,B>> from them.

Using this function requires that codecA be a codec that serializes to a string.

==Either==
The <code>Codec.either(codecA, codecB)</code> static method takes two codecs and generates a Codec<Either<A,B>> from them.

When this codec is used to de/serialize an object, it first attempts to use the first codec; if and only if that conversion fails, it then attempts to use the second codec. If that conversion also fails, then the returned DataResult will contain the error data from the *second* codec's conversion attempt.

==Numeric Ranges==
The <code>Codec.intRange(min,max)</code>, <code>Codec.floatRange(min,max)</code>, and <code>Codec.doubleRange(min,max)</code> static methods generate Codecs for Integers, Floats, or Doubles, respectively, for which only a specified inclusive range is valid, and values outside that range will fail to de/serialize.

==Maps==
Suppose we want to serialize a HashMap or other Map type, where we could have indefinitely many key-value pairs and we don't know what the keys are ahead of time.

We can create a <code>Codec<Map<KEY,VALUE>></code> using the <code>Codec.unboundedMap</code> static method, which takes a key codec and a value codec and creates a codec for a map type:

<syntaxhighlight lang="java">
public static final Codec<Map<String, BlockPos>> = Codec.unboundedMap(Codec.STRING, BlockPos.CODEC);
</syntaxhighlight>

The serialized form of maps serialized by this codec will be a JsonObject or CompoundNBT, whose fields are the key-value pairs in the map.

A limitation of using unboundedMap is that it only supports key codecs that serialize to Strings (including codecs for things like ResourceLocation that aren't Strings themselves but still serialize to strings). To create a codec for a Map whose keys are not fundamentally strings, the Map must be serialized as a list of key-value entries instead of using unboundedMap. As Codec.pair() also requires that the first codec serialize to a string, the pair function cannot be used for this purpose either.

==Equivalent Types and xmap==
Suppose we have two java classes, Amalgam and Box; any Amalgam instance can be converted to a Box, and vice-versa. Now suppose we have a Codec<Amalgam>, but we'd also like to have a Codec<Box>. Rather than creating an entirely new codec for Box from scratch, we can simply xmap our Amalgam codec instead.

The <code>Codec#xmap</code> instance method is used to generate a second codec for a fundamentally equivalent type to the first codec's type. The method takes two function objects as arguments, which are used to convert the first type to the second when deserializing, and converting the second type to the first when serializing:

<syntaxhighlight lang="java">
public static final Codec<Box> = Amalgam.CODEC.xmap(Amalgam::toBox, Box::toAmalgam);
</syntaxhighlight>

Codecs created in this manner will serialize objects in the same format as the starting codec.

==Partially Equivalent Types, flatComapMap, comapFlatMap, and flatXMap==
Consider the ResourceLocation: Any ResourceLocation can be converted to a String, but not all Strings can be converted to a ResourceLocation; ResourceLocations have strict limits on their format and allowed characters.

While we *could* use xmap to convert the Codec.STRING to a codec for ResourceLocations, this would cause attempts to parse an invalid string like <code>SHOUTY:MOD:Invalid$Characters</code> to throw a runtime exception, when we really should be returning a failed DataResult to the parser instead -- which indeed is what the vanilla ResourceLocation codec does.

Codecs have three additional instance methods for creating equivalent codecs for when we can *potentially* convert one type to another, but are not guaranteed to be able to do so. These take conversion function arguments that return DataResults, allowing validation to be performed during serialization and deserialization.

{| class="wikitable"
|+ Codec Conversion Methods
|-
! Can A always be converted to B? !! Can B always be converted to A? !! Which method of codecA should be used to create codecB?
|-
| yes || yes || codecA.xmap
|-
| yes || no || codecA.flatComapMap
|-
| no || yes || codecA.comapFlatMap
|-
| no || no || codecA.flatXmap
|}

== Registry Dispatch ==
Registry Dispatch Codecs allow us to define a registry of codecs and delegate to a specific codec to deserialize a particular json based on a type field in that json. Dispatch codecs are used extensively when deserializing worldgen data.

To create a dispatch codec for a Thing class, the following steps can be performed:
# Create a Thing abstract class class and ThingType interface. The ThingType interface should have a method that supplies a Codec<Thing>, while Thing subclasses must define a method that supplies a ThingType.
# Create a map or registry of ThingTypes, and register a ThingType for each sub-codec we want to have.
# Create a Codec<ThingType>, or have the ThingType registry implement Codec.
# Create our Codec<Thing> master codec by invoking <code>Codec#dispatch</code> on our ThingType codec. This method's arguments are:
## A field name for the ID of the sub-codec (the example json below is using "type")
## A function to retrieve a ThingType from a Thing
## A function to retrieve a Codec<Thing> from a ThingType

We can then use our Codec<Thing> to create Thing fields in other codecs whose serialized format depends on the specific sub-codec used by a Thing instance.

As an example of this, consider the ExampleCodecClass earlier. Suppose we make this class extend Thing and register our codec for it to a codec dispatch registry with the id "ourmod:exampleclass". If we were to define an instance of this class in a Thing field in some json, it would look like

<syntaxhighlight lang="json">
"some_thing":
{
"type": "ourmod:exampleclass",
"some_int": 42,
"item": "minecraft:gold_ingot",
"block_positions":
[
[0,0,0],
[10,20,-100]
]
}
</syntaxhighlight>

Other ThingTypes we register would have different fields in this json object, but would still be valid for the "some_thing" field.

Several examples of vanilla classes that use dispatch codecs:
* RuleTest and IRuleTestType
* IParticleData and ParticleType
* ConfiguredPlacement and Placement

=External Links=
* [https://github.com/Mojang/DataFixerUpper/blob/master/src/main/java/com/mojang/serialization/Codec.java Codecs in Mojang's official public DataFixerUpper repository]
* [https://kvverti.github.io/Documented-DataFixerUpper/snapshot/com/mojang/serialization/Codec.html#flatXmap-java.util.function.Function-java.util.function.Function- Unofficial Codec Javadocs]

Codecs

2021-01-04T23:06:39Z

Commoble: Expand and rewrite, describe codec utilities in greater detail

Codecs are a serialization tool from mojang's DataFixerUpper library. Codecs are used alongside [[DynamicOps|DynamicOps]] to allow objects to be serialized to different formats and back, such as JSON or NBT. While the DynamicOps describes the format the object is to be serialized to, the Codec describes the manner in which the object is to be serialized; a single Codec can be used to serialize an object to any format for which a DynamicOps exists. Codecs and DynamicOps jointly form an abstraction layer over data serialization, simplifying the effort needed to serialize or deserialize data.

= Using Codecs =

== Serialization and Deserialization ==

The primary use for Codecs is to serialize java objects to some serialized type, such as a JsonElement or an INBT, and to deserialize an serialized object back to its proper java type. This is accomplished with <code>Codec#encodeStart</code> and <code>Codec#parse</code>, respectively. Given a Codec<SomeJavaType> and a DynamicOps<SomeSerializedType>, we can convert instances of SomeJavaType to instances of SomeSerializedType and back.

Each of these methods take a [[DynamicOps]] instance and an instance of the object we are serializing or deserializing, and returns a DataResult:

<syntaxhighlight lang="java">
// let someCodec be a Codec<SomeJavaType>
// let someJavaObject be an instance of SomeJavaType
// let someNBT and someJsonElement be instances of INBT and JsonElement, respectively

// serialize some java object to INBT
DataResult<INBT> result = someCodec.encodeStart(NBTDynamicOps.INSTANCE, someJavaObject);

// deserialize some INBT instance back to a proper java object
DataResult<SomeJavaType> result = someCodec.parse(NBTDynamicOps.INSTANCE, someNBT);

// serialize some java object to a JsonElement
DataResult<JsonElement> result = someCodec.encodeStart(JsonOps.INSTANCE, someJavaObject);

// deserialize a JsonElement back to a proper java object
DataResult<SomeJavaType> result = someCodec.parse(JsonOps.INSTANCE, someJsonElement);
</syntaxhighlight>

A DataResult either holds the converted instance, or it holds some error data, depending on whether the conversion was successful or not, respectively. There are several things we can do with this DataResult; <code>DataResult#result</code> simply returns an Optional containing the converted object if the conversion was successful, while <code>DataResult#resultOrPartial</code> also runs a given function if the conversion was unsuccessful (in addition to returning the Optional). #resultOrPartial is particularly useful for logging errors during datapack deserialization:

<syntaxhighlight lang="java">
// deserialize something from json
someCodec.parse(JsonOps.INSTANCE, someJsonElement)
.resultOrPartial(errorMessage -> doSomethingIfBadData(errorMessage))
.ifPresent(someJavaObject -> doSomethingIfGoodData(someJavaObject))
</syntaxhighlight>

== Builtin Codecs ==
=== Primitive Codecs ===
The Codec class itself contains static instances of codecs for all supported primitive types, e.g. <code>Codec.STRING</code> is the canonical <code>Codec<String></code> implementation. Primitive codecs include:
* BOOL, which serializes to a boolean value
* BYTE, SHORT, INT, LONG, FLOAT, and DOUBLE, which serialize to numeric values
* STRING, which serializes to a string
* BYTE_BUFFER, INT_STREAM, and LONG_STREAM, which serialize to lists of numbers
* EMPTY, which represents null objects

=== Other Builtin Codecs ===
Vanilla minecraft has many builtin codecs for objects that it frequently serializes. These codecs are typically static instances in the class the codec is serializing; e.g. <code>ResourceLocation.CODEC</code> is the canonical <code>Codec<ResourceLocation></code>, while <code>BlockPos.CODEC</code> is the codec used for serializing a BlockPos.

Each vanilla <code>Registry</code> acts as the Codec for the type of object the registry contains; e.g. <code>Registry.BLOCK</code> is itself a <code>Codec<Block></code>. Forge Registries, however, do not currently implement Codec and cannot yet be used in this way; custom codecs must be created for forge-specific registries that are not tied to specific vanilla registries.

Of particular note here is the CompoundNBT.CODEC, which can be used to e.g. serialize a CompoundNBT into a json file. This has a notable limitation in that CompoundNBT.CODEC *cannot* safely deserialize lists of numbers from json, due to the strong typing of ListNBT and the way that the NBTDynamicOps deserializer reads numeric values.

= Creating Codecs =

Suppose we have the following class, and we want to deserialize json files to instances of this class:
<syntaxhighlight lang="java">
public class ExampleCodecClass {

private final int someInt;
private final Item item;
private final List<BlockPos> blockPositions;

public ExampleCodecClass(int someInt, Item item, List<BlockPos> blockPositions) {...}

public int getSomeInt() { return this.someInt; }
public Item getItem() { return this.someItem; }
public List<BlockPos> getBlockPositions() { return this.blockPositions; }
}
</syntaxhighlight>

Where a json file for an instance of this class might look like:

<syntaxhighlight lang="json">
{
"some_int": 42,
"item": "minecraft:gold_ingot",
"block_positions":
[
[0,0,0],
[10,20,-100]
]
}
</syntaxhighlight>

We can assemble a codec for this class by building a new codec out of smaller codecs. We'll need a codec for each of these fields:
* a <code>Codec<Integer></code>
* a <code>Codec<Item></code>
* a <code>Codec<List<BlockPos>></code>
And then we'll need to assemble these into a <code>Codec<ExampleCodecClass</code>.

As previously mentioned, we can use <code>Codec.INT</code> for the integer codec, and <code>Registry.ITEM</code> for the Item codec. We don't have a builtin codec for list-of-blockpos, but we can use BlockPos.CODEC to create one.

== Lists ==
The <code>Codec#listOf</code> instance method can be used to generate a codec for a List from an existing codec:

<syntaxhighlight lang="java">
// BlockPos.CODEC is a Codec<BlockPos>
Codec<List<BlockPos>> = BlockPos.CODEC.listOf();
</syntaxhighlight>

Codecs created via listOf() serialize things to listlike objects, such as [] json arrays or ListNBTs.

== Records ==
RecordCodecBuilder is used to generate codecs that serialize instances of classes with explicitly named fields, like our example above. Codecs created via RecordCodecBuilder serialize things to maplike objects, such as {} json objects or CompoundNBTs.

RecordCodecBuilder can be used in several ways, but the simplest form is as follows:

<syntaxhighlight lang="java">
public static final Codec<SomeJavaClass> = RecordCodecBuilder.create(instance -> instance.group(
someFieldCodecA.fieldOf("field_name_a").forGetter(SomeJavaClass::getFieldA),
someFieldCodecB.fieldOf("field_name_b").forGetter(SomeJavaClass::getFieldB),
someFieldCodecC.fieldOf("field_name_c").forGetter(SomeJavaClass::getFieldC),
// up to 16 fields can be declared here
).apply(instance, SomeJavaClass::new));
</syntaxhighlight>

Where each line in the group specifies a codec instance for the type of that field, the field name in the serialized object, and the corresponding getter function in the java class. The builder is concluded by specifying a constructor or factory for the java class whose arguments are the previously defined fields in the same order.

For example, using RecordCodecBuilder to create a codec for our example class above:
<syntaxhighlight lang="java">
public static final Codec<ExampleCodecClass> = RecordCodecBuilder.create(instance -> instance.group(
Codec.INT.fieldOf("some_int").forGetter(ExampleCodecClass::getSomeInt),
Registry.ITEM.fieldOf("item").forGetter(ExampleCodecClass::getItem),
BlockPos.CODEC.listOf().fieldOf("block_positions").forGetter(ExampleCodecClass::getBlockPositions)
).apply(instance, ExampleCodecClass::new));
</syntaxhighlight>

===Optional and Default Values in Record Fields===
When RecordCodecBuilder is used as shown above, all of the fields are *required* to be in the serialized object (the JsonObject/CompoundNBT/etc), or the entire thing will fail to parse when the codec tries to deserialize it. If we wish to have optional or default values, we have several alternatives of fieldOf() we can use.

* <code>someCodec.optionalFieldOf("field_name")</code> creates a field for an Optional. If the field in the json/nbt is not present or invalid, it will deserialize as an empty optional. Empty optionals will not be serialized; the field will be omitted from the json or nbt.
* <code>someCodec.optionalFieldOf("field_name", someDefaultValue)</code> creates an optional field that deserializes as the given default value if the field is not present in the json/nbt. When serializing, if the field in the java object equals the default value, the value will not be serialized and the field will be omitted from the json or nbt.

When using optional fields, be wary that if the field contains bad data or otherwise fails to serialize, the error will be silently caught, and the field will serialize as the default value instead!

==Unit==
The <code>Codec.unit(defaultValue)</code> codec creates a Codec that always deserializes a specified default value, regardless of input. When serializing, it serializes nothing.

==Pair==
The <code>Codec.pair(codecA, codecB)</code> static method takes two codecs and generates a Codec<Pair<A,B>> from them.

Using this function requires that codecA be a codec that serializes to a string.

==Either==
The <code>Codec.either(codecA, codecB)</code> static method takes two codecs and generates a Codec<Either<A,B>> from them.

When this codec is used to de/serialize an object, it first attempts to use the first codec; if and only if that conversion fails, it then attempts to use the second codec. If that conversion also fails, then the returned DataResult will contain the error data from the *second* codec's conversion attempt.

==Numeric Ranges==
The <code>intRange(min,max)</code>, <code>floatRange(min,max)</code>, and <code>doubleRange(min,max)</code> static methods in Codec generate Codecs for Integers, Floats, or Doubles, respectively, for which only a specified inclusive range is valid, and values outside that range will fail to de/serialize.

==Maps==
Suppose we want to serialize a HashMap or other Map type, where we could have indefinitely many key-value pairs and we don't know what the keys are ahead of time.

We can create a <code>Codec<Map<KEY,VALUE>></code> using the <code>Codec.unboundedMap</code> static method, which takes a key codec and a value codec and creates a codec for a map type:

<syntaxhighlight lang="java">
public static final Codec<Map<String, BlockPos>> = Codec.unboundedMap(Codec.STRING, BlockPos.CODEC);
</syntaxhighlight>

The serialized form of maps serialized by this codec will be a JsonObject or CompoundNBT, whose fields are the key-value pairs in the map.

A limitation of using unboundedMap is that it only supports key codecs that serialize to Strings (including codecs for things like ResourceLocation that aren't Strings themselves but still serialize to strings). To create a codec for a Map whose keys are not fundamentally strings, the Map must be serialized as a list of key-value entries instead of using unboundedMap. As Codec.pair() also requires that the first codec serialize to a string, the pair function cannot be used for this purpose either.

==Equivalent Types and xmap==
Suppose we have two java classes, Amalgam and Box; any Amalgam instance can be converted to a Box, and vice-versa. Now suppose we have a Codec<Amalgam>, but we'd also like to have a Codec<Box>. Rather than creating an entirely new codec for Box from scratch, we can simply xmap our Amalgam codec instead.

The <code>Codec#xmap</code> instance method is used to generate a second codec for a fundamentally equivalent type to the first codec's type. The method takes two function objects as arguments, which are used to convert the first type to the second when deserializing, and converting the second type to the first when serializing:

<syntaxhighlight lang="java">
public static final Codec<Box> = Amalgam.CODEC.xmap(Amalgam::toBox, Box::toAmalgam);
</syntaxhighlight>

Codecs created in this manner will serialize objects in the same format as the starting codec.

==Partially Equivalent Types, flatComapMap, comapFlatMap, and flatXMap==
Consider the ResourceLocation: Any ResourceLocation can be converted to a String, but not all Strings can be converted to a ResourceLocation; ResourceLocations have strict limits on their format and allowed characters.

While we *could* use xmap to convert the Codec.STRING to a codec for ResourceLocations, this would cause attempts to parse an invalid string like <code>SHOUTY:MOD:Invalid$Characters</code> to throw a runtime exception, when we really should be returning a failed DataResult to the parser instead -- which indeed is what the vanilla ResourceLocation codec does.

Codecs have three additional instance methods for creating equivalent codecs for when we can *potentially* convert one type to another, but are not guaranteed to be able to do so. These take conversion function arguments that return DataResults, allowing validation to be performed during serialization and deserialization.

{| class="wikitable"
|+ Codec Conversion Methods
|-
! Can A always be converted to B? !! Can B always be converted to A? !! Which method of codecA should be used to create codecB?
|-
| yes || yes || codecA.xmap
|-
| yes || no || codecA.flatComapMap
|-
| no || yes || codecA.comapFlatMap
|-
| no || no || codecA.flatXmap
|}

== Registry Dispatch ==
Registry Dispatch Codecs allow us to define a registry of codecs and delegate to a specific codec to deserialize a particular json based on a type field in that json. Dispatch codecs are used extensively when deserializing worldgen data.

To create a dispatch codec for a Thing class, the following steps can be performed:
# Create a Thing abstract class class and ThingType interface. The ThingType interface should have a method that supplies a Codec<Thing>, while Thing subclasses must define a method that supplies a ThingType.
# Create a map or registry of ThingTypes, and register a ThingType for each sub-codec we want to have.
# Create a Codec<ThingType>, or have the ThingType registry implement Codec.
# Create our Codec<Thing> master codec by invoking <code>Codec#dispatch</code> on our ThingType codec. This method's arguments are:
## A field name for the ID of the sub-codec (the example json below is using "type")
## A function to retrieve a ThingType from a Thing
## A function to retrieve a Codec<Thing> from a ThingType

We can then use our Codec<Thing> to create Thing fields in other codecs whose serialized format depends on the specific sub-codec used by a Thing instance.

As an example of this, consider the ExampleCodecClass earlier. Suppose we make this class extend Thing and register our codec for it to a codec dispatch registry with the id "ourmod:exampleclass". If we were to define an instance of this class in a Thing field in some json, it would look like

<syntaxhighlight lang="json">
"some_thing":
{
"type": "ourmod:exampleclass",
"some_int": 42,
"item": "minecraft:gold_ingot",
"block_positions":
[
[0,0,0],
[10,20,-100]
]
}
</syntaxhighlight>

Other ThingTypes we register would have different fields in this json object, but would still be valid for the "some_thing" field.

Several examples of vanilla classes that use dispatch codecs:
* RuleTest and IRuleTestType
* IParticleData and ParticleType
* ConfiguredPlacement and Placement

=External Links=
* [https://github.com/Mojang/DataFixerUpper/blob/master/src/main/java/com/mojang/serialization/Codec.java Codecs in Mojang's official public DataFixerUpper repository]
* [https://kvverti.github.io/Documented-DataFixerUpper/snapshot/com/mojang/serialization/Codec.html#flatXmap-java.util.function.Function-java.util.function.Function- Unofficial Codec Javadocs]

DynamicOps

2021-01-04T17:56:04Z

Commoble: Expand and rewrite; emphasize relationship between dynamicops and codecs

The <code>DynamicOps</code> class is part of mojang's DataFixerUpper serialization library; DynamicOps are used alongside [[codecs]] to convert java objects to a serialized format and back. While Codecs describe how a java object is to be serialized, DynamicOps describe the format the object is to be serialized to.

The DataFixerUpper library includes several DynamicOps for serializing to json; vanilla minecraft also includes a DynamicOps for serializing to minecraft's [[Using_NBT|NBT]] format.

= Builtin DynamicOps =

== JsonOps ==

The <code>JsonOps</code> DynamicOps are used to serialize and deserialize json data. These are instances of <code>DynamicOps<JsonElement></code>, meaning that they are used to convert java objects to and from <code>JsonElement</code> instances. These can used in conjunction with codecs to serialize/deserialize objects from assets and datapacks.

There are two public instances of JsonOps: <code>JsonOps.INSTANCE</code> and <code>JsonOps.COMPRESSED</code>. Compressed data is represented as a single string to read/write. However, this is never used within vanilla itself.

== NBTDynamicOps ==

The <code>NBTDynamicOps.INSTANCE</code> is an instance of <code>DynamicOps<INBT></code>, meaning it is used to convert java objects to and from <code>INBT</code> instances. This can be used for serializing data into packets to send across networks, as well as serializing persistant data for entities and similar objects.

There is only one public instance of NBTDynamicOps: <code>NBTDynamicOps.INSTANCE</code>.

= Using DynamicOps =

== Serializing and Deserializing ==

By combining a <code>Codec<SomeJavaType></code> with a <code>DynamicOps<SomeSerializedFormat></code>, the proper java object can be converted to the serialized form and back. See [[codecs]] for details on this subject.

== Format Conversion ==

By using the <code>DynamicOps#convertTo</code> instance method with a second DynamicOps instance, data can be converted between two different serialized formats, such as JsonElement to INBT and back.

<syntaxhighlight lang=java>
// converting INBT to JsonElement
JsonElement someJsonElement = NBTDynamicOps.INSTANCE.convertTo(JsonOps.INSTANCE, someNBT);

// converting JsonElement to INBT
INBT someNBT = JsonOps.INSTANCE.convertTo(NBTDynamicOps.INSTANCE, someJsonElement);
</syntaxhighlight>

Depending on the implementation of the DynamicOps used, this may throw an exception or return an empty object if the two formats are incompatible.

In particular, it is *not* safe to convert lists of numbers to NBT in this manner, due to the NBT format's strong typing and the way the NBTDynamicOps attempts to create lists of numbers.

= External Links =
* [https://kvverti.github.io/Documented-DataFixerUpper/snapshot/com/mojang/serialization/DynamicOps.html DynamicOps unofficial javadocs]