Dependencies

• An overview of automated testing in the Metaplay SDK - You should have an idea of how automated testing works across the Metaplay SDK as described in Introduction to Automated Testing.

Overview

The BotClient is a headless client designed to run multiple bots against a server. It can serve multiple purposes, such as load testing, functional testing, and game economy simulation. Well-implemented bots can be instrumental in catching functional bugs, particularly exceptions. The effectiveness of the testing coverage depends on how thoroughly features are represented to mirror real player behavior as closely as possible.

Instead of rigid test cases, the BotClient requires game-specific behavior to be implemented as "AI". This AI should be roughly representative of how real players might behave, as well as have "try everything once" chaotic behavior. A well-programmed AI that can simulate real player behavior can then be used to generate a realistic load on the game servers and populate multiplayer systems with participants, making them significantly easier to test and debug.

You can run BotClients locally to iterate on their logic quickly.

Running the BotClient

The BotClient is a standalone C# project that you can start the same way you start the game server. The BotClient can be configured to run a specific number of concurrent bots, and it can be configured to run against a specific server.

First, start the server on your machine. Any other way of starting the server is also fine.

Backend/Server$ dotnet run -LogLevel=Information

Then, run the BotClient. In this case, we use 5 concurrent bots. It targets the server running locally by default. You can find more configuration options on the Backend/BotClient/Config/Options.base.yaml file.

Backend/BotClient$ dotnet run -MaxBots=5 -LogLevel=Information

You can also run the BotClient locally against a cloud deployment. Just replace the hostname with the name of your game:

Backend/BotClient$ dotnet run --Bot:ServerHost=idler-develop.p1.metaplay.io --Bot:EnableTls=true

Writing Bot Logic

The bot logic should emulate real player behavior to a reasonable degree to make the load testing representative. This means that the bot AI logic should cover as much of the game features as reasonably possible and perform the operations on a roughly similar frequency as real players are expected to do.

The feature coverage is important primarily to ensure that some features don't have surprising performance bugs, like a player action using 100ms of CPU due to a loop running too many iterations, for example.

Regarding the frequency of operations, it's generally enough that each operation is done a similar amount of times as the others, since individual operations tend to be cheap and it doesn't matter which operation exactly is run. If some individual operations are known to be expensive, their behavior should be modeled more accurately.

Game-Specific Logic

Implementing the bot AI is simple in principle, and just implement the game-specific behavior in BotClient.OnUpdate():

Backend/BotClient/BotClient.cs

public class BotClient
{
  protected override async Task OnUpdate()
  {
    // Placeholder logic for executing an action with 5% chance on each tick
    RandomPCG rnd = RandomPCG.CreateNew();
    if (rnd.NextInt(100) < 5)
      _playerContext.ExecuteAction(new MyPlayerAction(...));
  }
}

In practice, your AI logic will likely become much more complicated over time as it should cover as many features of your game as reasonably possible.

You can take a look at the Idler sample's Backend/BotClient/BotClient.cs for a more complete example of how the AI logic can be implemented.

Load Testing

The BotClient can be run either locally for modest-sized load tests or in the cloud for large load tests scaling up to millions of concurrents when needed.

Load Testing Locally

Typically, it's best to start locally with a few hundred concurrent bots as a sanity test that there are no major performance issues. It's possible to scale the local test up to some thousands of bots, depending on how beefy of a machine you're using for testing.

To analyze any issues with CPU or memory consumption, please refer to the Troubleshooting Servers for instructions on CPU profiling and debugging memory leaks.

For typical games without expensive real-time simulations or other CPU-intensive features, you should expect roughly a thousand concurrents to run on each CPU, though both the server and BotClient will be consuming resources. On a reasonably sized machine, you can still expect to run some thousands of concurrent bots.

You should use logging level Information for both the game server and BotClient to give a more representative load test as debug logging can consume a significant amount of CPU time.

Load Testing in the Cloud

Support Contract Needed

Our tool for running high-scale load tests in the cloud is still a preview feature and not yet part of the public SDK. Please contact us if you are interested and we'll help you set it up in a way that best works for your project.

Further Reading

A small load test is also run as part of the integration tests. See Integration Testing for more details.