🧶 Yarn Spinner 1.0

The popular open source narrative game development framework, Yarn Spinner, which is maintained by Secret Lab and a fabulous community, has reached version 1.0. As part of our 1.0 release, we’ve debuted 5 exciting new features:

  1. Compiled Scripts — Yarn Spinner now compiles to a binary format.
  2. Automatic Compiling — In Unity, your Yarn scripts will automatically be compiled when they change.
  3. Line Tagging — You can automatically add unique tags to lines of dialogue, and generate a .csv file to send to translators with the click of a button.
  4. Code Extension — There’s a syntax highlighting extension, available from the marketplace, for Visual Studio Code.
  5. No more .yarn.txt — The file extension is now .yarn! It was time.

We want Yarn Spinner to be the best tool that it can be. As part of that, we’ve launched a Patreon page, and we’d love for you to help support its development!

We’ve got big plans, so please check out the website, follow us on Twitter, support the Patreon if you can, and join our Narrative Game Development Slack. And we’d really appreciate it if you shared the news!

📊 Replacing Your Gearbox at 100 MPH: How live games monitor and change with millions playing

At Velocity Berlin 2019, we were asked to give a talk to a crowd of largely non-game developers, on how games make gather and analyse telemetry in order to improve the experience of players. In this post, we’ll recap what we talked about, and provide links for further reading!

Jon and Paris on stage at Velocity Berlin 2019. Photo: Tim Nugent

This talk was largely a literature review of advice and techniques from game developers, and we’re tremendously grateful for their generous sharing of knowledge. In particular, three stand-out talks from GDC were hugely useful:

We’d also like to thank Tony Albrecht from Riot Games, whose advice helped us put this talk together.

In order to talk about how games use data, we’ll break the discussion down into four main topics: what data is gathered, how that data is gathered, how the data is analysed, and how the changes are deployed.

What Data is Gathered

Much of the data that games gather is not unique to games. Just about every product out there collects data on when it’s launched, how long the session lasts for, and how much interaction the user has with it; by gathering this data, it becomes possible to understand patterns of usage in terms of sessions.

There are three critical metrics that need to be gathered in order to gain a rough understanding of how a game is used:

  • Session duration: how much time elapses between the user starting and finishing a stretch of interaction.
  • Session interval: how long the user waits before starting another session.
  • Session depth: how many interactions the user has in each session.

It’s important to note that you can’t analyse these individually. If the average session interval is decreasing, that may indicate that people are coming back to play more and more, but it could also mean that players are opening the game, checking to see if there’s anything new, and then immediately leaving.

By gathering this data, it becomes possible to get a picture of the number of daily, weekly and monthly active users for the game. These figures represent how many unique users had a session in the game over the specified period; generally, you want this to be going up over time, though players tend to drop off a game over time. Daily active users counts tend to be quite spiky, because of players who hear about new content and updates and jump into the game, but don’t stick around for long. As a result, monthly active users tends to be the main reported figure, because it effectively smoothes out trends in player population; in the 2018 annual report for Activision Blizzard, one of the key figures that they highlighted was MAU across Activision, Blizzard and King.

The quarterly MAU figures for Activision Blizzard, from September 30 2017 to December 31 2018.

In addition to these standard metrics, there are also data points that are unique to games. These tend to vary based on the type of game, but generally include things like score, death, level number, and position. Position is a particularly interesting one, because it’s very easy to visualise and plot against other key events – we’ll come back to this in a moment.

However, these concrete measurements of player behaviour aren’t good at getting an understanding of whether the player enjoyed themselves in the game. To fix this gap, Call of Duty: World War II directly asks their players if they had fun, using a single yes/no question that’s designed to minimise the burden of answering. Interestingly, the development team reports an average non-skip rate of between 60-80%, even when the answer order randomisation places the option to skip as the default. This is significantly higher than they expect.

The Call of Duty: WW2 fun survey.

Finally, games typically record performance data on how smoothly the game is running. Games typically aim to play at 60 frames per second, which means that each frame has only 16.6 milliseconds to render. When you have only about a dozen milliseconds to render, every one of them counts.

As a result, League of Legends records two kinds of performance data: first, regular telemetry reports are sent during a game, giving an idea of the impact that the most recent patch has had on performance. Additionally, the game records performance data for each frame as it’s played, and then compresses and uploads the data at the end of a round.

Most games perform some kind of internal time profiling; a common tool that we’ve seen multiple game developers use is the Chromium tracing system. This system was originally built to profile the performance of the Chrome web browser, but it’s able to load arbitrary timing data for analysis. For example, Aras Pranckevičius has written about how Unity Technologies uses it to profile the performance of their build system, and Colt McAnlis has written about why using Chrome’s profiler is a better idea than building your own.

How the Data is Gathered

Data collection is the process of delivering the data to the developer for analysis. There are a few ways to do it, and a great talk from Tom Mathews from 343 on how they built their telemetry systems for Halo 5 is a great place to look at. Some highlights of his talk include the fact that they converted their logging system away from unstructured logging strings to a formal, schema-based format based on Microsoft’s Bond format.

Among a few other benefits, this logging system allowed for sub-second response to telemetry, which meant that the game itself is able to respond to the data. As a result, in-game elements like end-of-game reports and leaderboards are driven by the telemetry system, rather than having to build a separate system for this purpose.

How the Data is Analysed

The data received from games can be divided into two main categories: spatial, and non-spatial. Spatial game data is anything that’s related to the player’s position in the game, whereas non-spatial data is everything else, and includes data like in-game performance, skill, and time spent in the game.

Non-spatial data is generally used to get a picture of how players are enjoying the game, and to generate a predictive understanding of whether players are coming back to play more. The fun survey from Call of Duty: World War II is particularly interesting, because they gather data on whether the player enjoyed the game or not – the developers don’t have to infer this data from simpler things like the fact that players are coming back.

This allows them to link player fun to other variables, with some surprising results. As might be expected, in-game performance – that is, how many kills a player got versus how many deaths they had – is a strong predictor of how much fun the player had in the game, but it’s not the only important variable. Player tenure (the total duration of time spent in game, across all sessions) and player skill (total kills versus total deaths, across all sessions) were found to be strong predictors of players reporting that they didn’t have fun. The Call of Duty development team’s theory for this is that the longer a person plays a game, the more critical of it they become; additionally, they found that player fun reports drop off once a player’s skill level becomes greater than that of the median player.

A particularly interesting note made by the development team was that the margin by which a team won was less impactful on fun score than the margin by which the team lost by. That is, a player whose team won by 50 points was just as likely to say that they had fun as a player who won by 5 points. This wasn’t the case for the losing team, however – a player whose team lost by a large margin was much more likely to report that they didn’t have fun than a player who only lost by a few points. This is good empirical evidence for the widely held belief that games that end in close ties are better for everyone.

In the area of spatial data, an excellent paper by Anders Drachen and Matthias Schubert, “Spatial game analytics and visualization” (PDF) proposes four primary types of spatial data analysis: univariate/bivariate, multivariate, trajectory, and behavioural analysis.

Univariate/bivariate analysis focuses on either one or two variables, in which one of those variables is player position. This is usually seen in the form of heat maps for levels, which allow developers to get a good understanding of where players die in levels. For example, consider this heat map for the level de_dust2, from Counter-Strike. Red areas indicate places where lots of players die.

de_dust2
A heatmap showing player deaths in de_dust2, from Counter-Strike 1.6. Source: gameME

Some interesting observations from this heat map:

  • Corridors and doorways are hotspots for player death
  • Doorways exhibit diagonal lines of player death, indicating where players lie in wait
  • Certain long lines of player death indicate places where players have long sight lines
  • A grid pattern can be seen at the bottom and top areas; these are the player spawn locations, and represent players starting the game there, deciding they don’t like their team, and quitting.

An excellent discussion on using heat maps to analyse level flow and gameplay balance is Sean Houghton’s post Balance and Flow Maps, which discusses an analysis of gameplay balance in maps used in Transformers: War for Cybertron.

Multivariate analysis allows for some more complex and sophisticated analysis of level content. Georg Zoeller’s excellent 2011 talk at GDC about the analysis tools used in Star Wars: The Old Republic shows how they combine information about player death with the locations and levels of monsters in the level, which were used to figure out balance problems with the level progression curve.

Trajectory information can be used to plot the movement of individual players through the game’s space, and is useful for detecting outliers and unintended paths through the environment. Jonathan Dankoff’s post on using trajectory analysis in Assassin’s Creed Brotherhood highlights how players could bypass tutorial content by not following the expected path through the environment.

Player trajectory data in Assassin’s Creed: Brotherhood. Players are intended to jump off a tower and parachute towards the destination (green lines at top), but some players were instead finding a way down the walls (red and green lines in the lower right), bypassing tutorial content. Source: Jonathan Dankoff

Finally, spatial information can be used to derive data about player behaviour in the game. Mahlmann et al’s paper, “Predicting player behavior in Tomb Raider: Underworld” (PDF) was able to use information like how long players spent in the early parts of the game to predict how far through the game they’d get before they gave up – something that’s extremely useful in balancing game difficulty and production investment in the game’s content.

How Changes are Deployed

When a game has made changes, it’s time to get the updated version out to players. There are a variety of ways to do this, with varying levels of disruption to players.

The most straightforward way of doing it is to release a new version of the game via digital distribution – that is, via Steam, itch, and the various App Stores. This is conceptually simple, but has a few downsides: players may not be aware of the update, or may choose not to update, which fragments the installed player base. Additionally, the size of the updates may be large, which reduces the chance of all players updating.

The local patching model adopted by the Nintendo Switch is quite interesting: players who want to form a local network and play a game are able to compare their installed version, figure out who has the most recent update, and then distribute the patch locally, without relying on internet access. This feature is especially important when you consider that one of the main marketing points of the Switch is the ability pick it up and take it outside; the Switch has no cellular internet connectivity, so local wireless communication is all it has.

If a game is designed to be competitive, opt-in beta streams can be used. In this model, players choose to receive beta versions of patches, and play the game in a testing mode. Because game changes frequently change the balance of play, players who want to maintain a competitive edge have an incentive to play the beta stream, and accept the risk of bugs and data loss.

Game changes don’t necessarily require updates to the code or assets, and small hot fixes can be applied. Some notable games that do this include Borderlands and Fortnite, which download a small patch on every game load that tunes gameplay content. These patches are typically only kept in memory, and are lost when the game exits; hot fixes are generally rolled up into a permanent patch after some time.

In order to minimise downtime, a blue-green deployment model for server updates is frequently common. When a new patch becomes available, existing servers are kept online for as long as there are players connected. All new players connect to servers running the latest version, and older servers are shut down as players disconnect from them. This means that players aren’t required to leave the game when a new patch lands; however, this model only works in games where players are separated into discrete sessions, and doesn’t work in single, shared-world environments like massively multiplayer games. For example, Star Wars: The Old Republic shuts down every Tuesday night for a few hours for patch deployment, and all players are kicked from the server.

Wrapping Up

We had a great time presenting this to a room full of operations and deployment experts, and we feel that there’s a lot that games can bring to the wider world of operations management. The video recording of our session will be available soon, and we’ll add it to this post when it arrives.

Installing Unity ML-Agents

⚠️ At O’Reilly AI Conference San Jose, attending our tutorial? This is for you! Complete all the steps in this document to be ready for the tutorial.

Want to explore the Unity Machine Learning Agents Toolkit (“ML-Agents”)? Here’s the easiest way to get up and running on Windows or macOS.

Unity ML-Agents is a great way to explore machine learning, whether you’re interested in building AI for games, or simulating an environment to solve a broader ML problem, why not try Unity’s ML-Agents?

We’ll be posting a variety of guides and material covering various aspects of Unity’s ML-Agents, but we thought we’d start with an installation guide!


To use ML-Agents, you’ll need to install three things:

  1. Unity
  2. Python and ML-Agents (and associated environment and support)
  3. The ML-Agents Unity project

Unity

Installing Unity is the easiest bit. We recommend downloading and using the official Unity Hub to manage your installs of Unity.

➡️ Download the Unity Hub for Windows or macOS

The Unity Hub allows you to manage multiple installs of different versions of Unity, and lets you select which version of Unity you open and create projects with.

⚠️ We recommend installing Unity 2019.2.4f1 for the tutorial at O’Reilly AI Conference. If you install a different version, we might not be able to help you.

If you don’t want to use the Unity Hub, you can download different versions of Unity for your platform manually:

➡️ Download a specific version of Unity for Windows or macOS

We strongly recommend that you use the Unity Hub to manage your Unity installs, as it’s the easiest way to stick to a specific version of Windows, and manage your installs. It really makes things easier.

If you like using command line tools, you can also try the U3d tool to download and manage Unity install’s from the terminal.

Python and ML-Agents

Our preferred way of installing and managing Python, particularly for machine learning tasks, is to use the Anaconda Environment.

⚠️ Anaconda’s environments don’t quite work like virtualenv, or other Python environment systems that you might be familiar with. They don’t store things in the location you specify, they store things in the system-wide Anaconda directory (e.g. on macOS in “/Users/USER/anaconda3/envs/”). Just remember that once you activate them, all commands are inside the environment.

Anaconda bundles a package manager, an environment manager, and a variety of other tools that make using and managing Python environments easier.

➡️ Download the Anaconda installer for Windows or macOS

Once you’ve installed Anaconda, following these instructions to make an Anaconda Environment to use with Unity ML-Agents.

➡️ First, download 🔗 this yaml file, and execute the following command (pointing to the yaml file you just downloaded):

conda env create -f /path/to/unity_ml.yaml

➡️ Once the new Anaconda Environment (named UnityML) has been created, activate it using the following command in your terminal:

conda activate UnityML

The yaml file we provided specifies all the Python packages, from both Anaconda’s package manager, as well pip, the Python package manager, that you need to make an environment that will work with ML-Agents.

Doing it manually

You can also do this manually (instead of asking Anaconda to create an environment based on our environment file).

⚠️ You do not need to do this if you created the environment with the yaml file, as above. If you did that go straight to “Testing the environment”, below.

➡️ Create a new Anaconda Environment named UnityML and running Python 3.6 (the version of Python you need to be running to work with TensorFlow at the moment):

conda create -n UnityML python=3.6

➡️ Activate the Conda environment:

conda activate UnityML

➡️ Install TensorFlow 1.7.1 (the version of TensorFlow you need to be running to work with ML-Agents):

pip install tensorflow==1.7.1

➡️ Once TensorFlow is installed, installing the Unity ML-Agents:

pip install mlagents

Testing the environment

➡️ To check everything is installed properly, run the following command:

mlagents-learn --help

You should see something that looks like the following image. This shows that everything is installed properly.

If you’re coming to our conference tutorial, you’re now ready to go.

The ML-Agents Unity Project

The best way to start exploring ML-Agents is to use their provided Unity project. To get it, you’ll need a copy of the Unity ML-Agents repository.

⚠️ You do not need to do this bit if you’re coming to our tutorial at the O’Reilly AI Conference. We will provide a project on the day.

➡️ Clone the Unity ML-Agents repository to your system (see the note below if you’re coming to our tutorial!):

git clone https://github.com/Unity-Technologies/ml-agents.git

⚠️ If you’re coming to our O’Reilly AI Conference tutorial, we will provide a project on the day.

You should now have a directory called ml-agents. This directory contains the source code for ML-Agents, a whole of lot useful configuration files, as well starting point Unity projects for you to use.

➡️ You’re ready to go! If you’re coming to our tutorial, you’ll need a slightly different project which we’ll help you out with on the day!

We’ll have another article on getting started (now that you’ve got it installed) next week!


In San Jose? At O’Reilly AI Conference? Attend our tutorial!