Dependencies

• Working knowledge of entities and actors - Schema Versions and Schema Version Migrations are part of the backend's entities ecosystem. Take a look at Introduction to Entities and Actors if you're not familiar with them yet.
• Basic understanding of Logic Versions - Logic Versions are closely tied to Schema Versions. While Schema Versions control the data structure versions and migrations, Logic Versions control the network protocol compatibility.

Key Concepts

• Every entity has an associated integer called Schema Version that tracks what version of the data model it currently has. This number should be incremented when a change requires a Migration to initialize new fields. When an entity is persisted to the database, its Schema Version is stored alongside the payload.
• A Schema Version Migration (or just Migration) is an operation that increments the Schema Version and upgrades the entity to match the new model data layout.

Overview

Over a game's lifetime, it changes, receives updates, and eventually, old logic gets deprecated. Schema Version Migrations allow you to update older persisted entities to match the current data models. It's also possible to eliminate the old code if necessary, but we don't recommend it.

Here are the Schema Version operations covered on this page:

1. Defining the Initial Schema Version
2. Implementing Migrations
3. Retiring Legacy Members
4. Manually Migrating Entities

Entity Schema Version vs Database Schema

The Entity Schema Version is not the same as the Database Schema. The two are versioned independently, and changing one does not imply changing the other:

• The Entity Schema Version defines the payload format of a persisted entity: the serialized blob stored inside a row.
• The Database Schema defines the database structure: its tables, columns, and indexes. See Working with the Database.

Defining the Schema Version

You can mark a persisted state object to support Migrations by implementing the ISchemaMigratable interface and then using the [SupportedSchemaVersions(oldestSupportedSchemaVersion, currentSchemaVersion)] attribute to specify the range of Schema Versions that you wish to support for the type. The oldestSupportedSchemaVersion is the oldest Schema Version we're willing to support migrating from.

Note that the base class often implements the ISchemaMigratable interface and does not need to be explicitly implemented anymore. For example, with PlayerModel, the interface is already implemented by PlayerModelBase.

As an example, let's define a simplified initial player model that holds information on the number of fruits that the player has:

PlayerModel.cs

// Right now, the backend only supports Schema Version 
// 1, with it being both the CurrentSchemaVersion 
// and the OldestSupportedSchemaVersion.
[MetaSerializableDerived(1)]
[SupportedSchemaVersions(1, 1)]
public class PlayerModel : PlayerModelBase<...>
{
    [MetaMember(1)] int NumApples;
    [MetaMember(2)] int NumOranges;

    ...
}

When a new entity is created, its Schema Version is initialized to its maximum supported Schema Version. The maximum Schema Version is also the current Schema Version, while OldestSupportedSchemaVersion is the oldest version we want to support Migrations from.

Implementing the Migration

The Migration is implemented with a set of Migration functions, each representing a Migration operation from a given Schema Version and updates the model to the next.

Continuing our example, we could introduce a new set of members to record the highest number of fruits a player has ever held. These new members would need to be initialized through a Migration.

The simplest way of implementing the Migrations is to declare the individual Migration functions as Model methods tagged with the [MigrationFromVersion(fromVersion)] attribute, grouped together into a C# region:

PlayerModel.cs

[MetaSerializableDerived(1)]
[SupportedSchemaVersions(1, 2)] // CurrentSchemaVersion increased from v1 to v2
public class PlayerModel : PlayerModelBase<...>
{
    [MetaMember(1)] int NumApples;
    [MetaMember(2)] int NumOranges;
    [MetaMember(3)] int MaxNumApples; 
    [MetaMember(4)] int MaxNumOranges; 

    ...

    #region Schema migrations

    // Migration from version 1 to 2:
    // Added a new field to record the highest number of fruit
    // that a player has ever held.
    [MigrationFromVersion(1)]
    void Migrate1To2()
    {
        MaxNumApples = NumApples;
        MaxNumOranges = NumOranges;
    }

    #endregion
}

Now, let's imagine that we want to remove apples from the game and that we'll compensate players at an exchange rate of 3 oranges for every apple:

PlayerModel.cs

[MetaSerializableDerived(1)]
[SupportedSchemaVersions(1, 3)] // Bumped CurrentSchemaVersion to v3
public class PlayerModel : PlayerModelBase<...>
{
    [MetaMember(1)] int LegacyNumApples;    // No longer used in-game
    [MetaMember(2)] int NumOranges;
    [MetaMember(3)] int LegacyMaxNumApples; // No longer used in-game
    [MetaMember(4)] int MaxNumOranges;

    ...

    #region Schema migrations

    // Migration from version v1 to v2:
    [MigrationFromVersion(1)]
    void Migrate1To2()
    {
        LegacyMaxNumApples = LegacyNumApples;
        MaxNumOranges = NumOranges;
    }

    // Migration from version v2 to v3:
    [MigrationFromVersion(2)]
    void Migrate2To3()
    {
        // Compensate players with three oranges for every apple that they held.
        NumOranges += LegacyNumApples * 3;

        // It's good practice to clear these legacy values as we will no longer use them.
        LegacyNumApples = 0;
        LegacyMaxNumApples = 0;

        // Update MaxNumOranges after compensation.
        MaxNumOranges = Math.Max(MaxNumOranges, NumOranges);
    }

    #endregion
}

The SDK calls the Migration functions automatically when a persisted entity is being "woken up", i.e., restored from the database, if the entity was persisted with a Schema Version lower than current. The entity sequentially invokes each Migration step required to bring it to the current version, and only then is the entity ready to start running.

The Migrations are performed whenever a persisted entity wakes up. This can happen for many reasons, but the most typical ones are players logging in, an entity (player or other) being viewed from the dashboard, or an entity spawning automatically when the server starts. Migrations are also invoked when an entity is imported with a Schema Version older than the current one.

After you deploy the Migration code, it does not run automatically for all entities. An entity stays at the old Schema Version until it is woken up and then persisted again. This means that you may need to keep the old code around indefinitely to support the old Migration code. See the section Manually Migrating Entities for a technique to address this.

Using RegisterMigrationsFunction

As your model's number of Migration functions grows, maintaining separate methods for each Migration might become impractical. Therefore, it is also possible to declare Migration functions per version by declaring a single RegisterMigrationsFunction static method that returns a lambda function per Migration version:

PlayerModel.cs

[SupportedSchemaVersions(1, 3)]
public class PlayerModel : PlayerModelBase<...>
{
    [RegisterMigrationsFunction]
    static Action<PlayerModel> RegisterMigrations(int fromVersion)
    {
        switch (fromVersion)
        {
            // Migrate from v1 to v2: Initialize LegacyMaxNumApples and MaxNumOranges
            case 1: return model => {
                model.LegacyMaxNumApples = model.LegacyNumApples;
                model.MaxNumOranges = model.NumOranges;
            };
            // Migrate from v2 to v3: Remove apples from the game
            case 2: return model => {
                model.NumOranges += model.LegacyNumApples * 3;
                model.LegacyNumApples = 0;
                model.LegacyMaxNumApples = 0;
                model.MaxNumOranges = Math.Max(model.MaxNumOranges, model.NumOranges);
            };
            default: return null;
        }
    }

    ...

}

These two examples yield identical results, the only difference being the convenience of declaring the functions. The latter RegisterMigrationsFunction approach is especially convenient if your Version Migrations reuse code. For example, you want to clear some version-dependent state on the model every time a Migration executes.

It is also possible to declare both individual MigrationFromVersion functions and a RegisterMigrationsFunction, but you can only declare a single Schema Version Migration function via one of the methods. If a MigrationFromVersion function is provided, then the corresponding call to RegisterMigrations must return null. On conflicting declarations, the Metaplay SDK will raise an error during initialization.

Retiring Legacy Members

When you remove a feature, the members that held its data often stick around only so a Migration can still read them. In our example, LegacyNumApples and LegacyMaxNumApples are the original apple fields, renamed and kept so that Migrate2To3() can compensate players for them. Once no Migration needs them anymore, they become dead weight. Removing one is a two-step process. First, drop support for the Schema Versions whose Migrations still reference the member. Then delete the member itself.

Dropping support for old Schema Versions

OldestSupportedSchemaVersion is the lowest Schema Version your code still supports. The SDK cannot wake up any entity persisted at a lower version, because the Migration it would need no longer exists. As long as the game backend may run against a database holding such entities, the corresponding Migration code cannot be removed.

⚠️ Be careful

Only raise OldestSupportedSchemaVersion once you are certain every entity has been Migrated to at least that version. Otherwise the SDK fails to deserialize any entity that is still on an older version, and those entities cannot wake up. For a player entity, this means the player can no longer play. For a live game, raising the value is rarely safe unless you first actively migrate all entities. See Manually Migrating Entities.

If you accidentally raise OldestSupportedSchemaVersion and remove the old migration code before all entities have migrated, you can reinstate the old version range and migration code to let the remaining entities migrate.

Alternatively, you can configure entities to reset to their initial state instead of failing to wake up, by overriding PersistedEntityConfig.ResetOnSchemaVersionTooOld to return true for the entity type in question.

Removing the members

Once no remaining Migration references a member, you can delete it. Removing a [MetaMember] is safe on its own, because the deserializer simply ignores any persisted members whose Id no longer maps to a member.

One possible hazard is reusing a freed Id. A member declared with a removed member's Id would deserialize old data into the wrong field. To prevent this, list the freed Ids in a [MetaBlockedMembers(...)] attribute on the class.

Continuing the example, once all players are on v3 we can drop v1 and v2 support, delete the Legacy* apple members (MetaMembers 1 and 3), and block those Ids:

PlayerModel.cs

[MetaSerializableDerived(1)]
[SupportedSchemaVersions(3, 3)] // Dropped support for v1 and v2
[MetaBlockedMembers(1, 3)]      // Members freed by removing LegacyNumApples and LegacyMaxNumApples
public class PlayerModel : PlayerModelBase<...>
{
    [MetaMember(1)] int LegacyNumApples;    // No longer used in-game
    [MetaMember(2)] int NumOranges;
    [MetaMember(3)] int LegacyMaxNumApples; // No longer used in-game
    [MetaMember(4)] int MaxNumOranges;

    #region Schema migrations

    [MigrationFromVersion(1)]
    void Migrate1To2()
    {
        // ...
    }
    [MigrationFromVersion(2)]
    void Migrate2To3()
    {
        // ...
    }

    #endregion
}

Manually Migrating Entities

For many types of entities, there's no natural time after which they'd all have Migrated. For example, there's no time period after which we'd know for sure that all existing players have logged in at least once after a new Schema Version was added.

A practical way to address this is to actively wake up all Entities of a given type. Metaplay has a schema migrator maintenance job for this purpose; see the Scan Jobs System vs. Maintenance Jobs section in Database Scan Jobs. The schema migrator job scans through an entity table, wakes up all the entities on an old Schema Version, and produces a report indicating whether all entities were successfully Migrated to the current Schema Version. Note that the job is intended to be run during the normal operation of a game server and can take a significant amount of time to complete, depending on the number of entities.

⚠️ Caution

When removing code dealing with old Schema Versions, consider all the environments in which the code may run. For example, even if all the player entities in your production environment have migrated, you might still need to consider your staging and development environments.

After you are sure all entities of a given type have migrated to the maximum Schema Version, the code dealing with the old Schema Versions can be safely removed.