Key Concepts

• Actor in an actor system is an isolated stateful object that communicates through message passing.
• Message is an immutable unidirectional serializable object with no built-in behavior or semantics.
• Entity Actor is an Actor responsible for implementing the behavior of a Game Entity on the Backend.

Entity Actor Lifecycle

The Game Backend is a distributed actor system where the game logic is implemented using Entities. The Entity lifecycles come in two flavors, as Ephemeral and Persisted entities.

Ephemeral Entity

Create ---> ( Starting ) --> ( Running ) --> ( Stopping ) ---> removed from system

An Ephemeral Entity is an Entity that lives only as long as it is running. Its state will be forgotten and EntityId will become unused when the entity instance stops. This is often used for service entities that do not need to store state over server node reboots and for helper entities that only live for a moment. For example, InAppPurchaseValidatorActor is an ephemeral entity. If this entity is lost, the ongoing validation operations will be lost, but only temporarily as the validation will be retried.

Persisted Entity

                                .------- crash recovery ----------.
                                |                                 v
Create --> ( Starting ) --> ( Running ) --> ( Stopping ) --> ( Persisted )
               ^                                                  |
               '-------- Wake up on demand -----------------------'

A persisted entity is an entity whose state is stored into a persisted storage when it is not running. When it is created for the first time, it is given an immutable EntityId, and from this point onward the EntityId will always refer to this particular entity. The next time the entity is needed and the state of the entity is restored from the database. Hence, persisted entities are eternal.

Entity Communication

Entities communicate by sending messages to each other. When sending a message, the sender specifies a destination entity id and a payload. The system will deliver the message to the destination entity if such exists, waking it up if necessary. Communication is fundamentally unreliable. If the destination entity does not exist or an error occurs, the message is dropped and no retries are attempted. If the sender needs to know whether the message was processed, the recipient must send an acknowledgment message back to the sender. This pattern is available thru EntityAsk<> method.

In addition to raw messaging and EntityAsk<>, Metaplay provides a PubSub system in which an entity may Subscribe to a topic of another Entity. When the subscribed entity publishes a message to a given topic, the message is delivered to the entities subscribing to that topic on that entity.

All participants in a PubSub must be in a Running state. When an entity shuts down or crashes, all subscribers and subscribees are notified and the shutdown entity is unsubscribed. However, the communication is fundamentally unreliable and no timely lifecycle notifications can be guaranteed. Only the order of the messages and the delivery of preceding messages in a single PubSub topic is guaranteed; for anything more sophisticated, an EntityAsk<>-like acknowledgment mechanism needs to be used.

For more details, see Deep Dive: Entity-to-Entity Communication.

Entity Shutdown

As entities use memory, CPU, and potentially other resources when running, it is important to shut down entities as they are no longer needed. As such, the developer must choose a suitable EntityActor.ShutdownPolicy for each entity type in the game.

The most straightforward ShutdownPolicyis AutoShutdownPolicy::ShutdownNever. As its name implies, the entity will not be automatically shut down. In this mode, the developer must manually call RequestShutdown() where appropriate. This is most commonly used for long-lived service entities.

For automatic shutdown, the developer may use AutoShutdownPolicy::ShutdownAfterSubscribersGone. In this mode, the entity lives only as long there has been recently a PubSub subscriber to the entity. After the last subscriber is lost and the specified time has passed with no new subscribers appearing, the entity will be shut down. This is the most commonly used for persisted entities.

Entity Sharding and Configuration

Metaplay's Entities are stored in shards on the server, and they have a configuration class that defines some characteristics like EntityShardGroup, ShardingStrategy, and NodeSetPlacement.Depending on what their purpose is, they should be initialized with different configurations.

CustomEntityConfig.cs

namespace Game.Server;

[EntityConfig]
class CustomEntityConfig : PersistedEntityConfig
{
    public override EntityKind        EntityKind           => EntityKindGame.CustomEntity;
    public override Type              EntityActorType      => typeof(CustomActor);


    public override EntityShardGroup  EntityShardGroup     => EntityShardGroup.Workloads;
    public override IShardingStrategy ShardingStrategy     => ShardingStrategies.CreateStaticSharded();
    public override NodeSetPlacement  NodeSetPlacement     => NodeSetPlacement.Service;

    ...
}

Here are what the configurations define:

Entity Shard Group

The EntityShardGroup specifies the order in which the Entities in question are initialized, relative to the other Entities. In most cases, EntityShardGroup.Workloads is the correct group choice and it is the default value. The other values are intended for core services that need to be initialized before everything else. The options are:

• EntityShardGroup.BaseServices: Initial core services like GlobalStateManager and DiagnosticTool that other services depend on
• EntityShardGroup.ServicesProxies: Local proxy actors for the base services.
• EntityShardGroup.Workloads: Everything else. Usually the correct choice.

Sharding Strategy

The ShardingStrategy specifies how the Entities of a given type are placed across the cluster when multiple nodes are handling the same kind of Entity. Here are the possible options:

• ShardingStrategies.CreateSingletonService() for service Entities that should only have one copy running. The service Entity will automatically spawn when the server starts.
• ShardingStrategies.CreateDynamicService() for service Entities that should have a copy on every node responsible for this Entity kind. The service Entities are automatically started when the server starts.
• ShardingStrategies.CreateMultiService() for service Entities that should have a predetermined number of copies running, regardless of the number of nodes in the cluster. The service Entities are automatically distributed between the nodes that are responsible for this Entity kind when the server starts.
• ShardingStragies.CreateStaticSharded() for Entities that should be sharded across a static set of nodes. Use this for Entities that should scale across multiple nodes in the cluster. It is used by Entities such as players, guilds, and guild divisions that need to be scalable.
• ShardingStrategies.CreateManual() for Entities that get their EntityIds assigned manually. This can be used to encode the location of ephemeral Entities into the EntityId and therefore allow more dynamic placement of Entities than the static sharding strategy allows.

Node Set Placement

The NodeSetPlacement specifies the default nodes in the cluster on which the entities should be placed. See Configuring Cluster Topology to better understand how clustering works in Metaplay as well as how to override these placements for non-standard cluster configurations. The valid options are:

• NodeSetPlacement.Service should be used for singleton entities where only one copy of the entity exists and other service-style entities that don't need to scale across the cluster.
• NodeSetPlacement.Logic should be used for entities that need to scale across multiple cluster nodes.
• NodeSetPlacement.All should be used for entities that need to be located on all nodes in the cluster. These are usually various daemon-style service entities.