EVE's Carbon Engine: Quasar And Data Driven Design

Every so often I come across a reminder than both EVE Online and EVE Frontier use the Carbon game engine. Players from EVE Online probably want to check the latest Frontier dev blogs for information, especially those interested in the technical aspects of the game. Last week the Frontier team published a dev blog describing how the information from Quasar is used to evaluate events and gameplay loops during development.

First, what is Quasar? Quasar is CCP’s custom-built gRPC-based networking layer within the Carbon Engine. The framework acts as a high-speed data pipeline between the game client/server and external services—bypassing traditional databases for efficiency. For purposes of design from the dev blog, the author concentrates on the tracking of protobuf information.

What are protobufs? The dev blog explained.

These protobufs can be quite literally anything we want. The process is simple:

• Someone identifies a need for a particular point of data (e.g. a ship refueling or someone constructing a smart assembly like a Smart Storage Unit).
• The gameplay engineering team ensure a protobuf for that event is emitted to Quasar.
• The data engineering team receive these events and they go into the ‘cold data lake’. So called cold because this type of data doesn’t need to be real-time, it can take up to an hour as the system aggregates data for faster querying (there are also real-time events for things like missions).
• Data scientists do data magic and produce pages in Tableau for designers and engineers to review.

I don't think the example used to illustrate the usage of Quasar would surprise anyone who has played EVE Online. That's right, Frontier's New Player Experience, which I usually call the tutorial when writing about the game. But as I've run through the NPE several times now the example makes sense to me.

Timing of each element of Frontier's tutorial

I have to appreciate as someone who purchased a Founder's pack that I don't need to fill out journal entries for the developers. That task is handled by Quasar. Which leads me to concentrate on the mechanics so I can learn about Frontier instead.

Now, like AI programs, the data does need some interpretation.

In the table we can clearly see some steps take significantly longer than others. For example, players leaving the system after fitting modules to their ship. Now I should note some long steps are expected and not problematic. For example leaving the system being where it is makes sense because, once you have built your first jump capable ship, it takes a few more mining/refining runs to fuel up. Moreover, with the lens that as these are primarily new players, they will not have figured out the most optimal solutions / routes / methods for doing these things in-game.

The next example really appeals to long-time EVE players: player retention. Where in the tutorial do players decide to quit the tutorial?

Where do players quit?

A quick explainer on this one, the % shown at each line is the % of new players that don’t complete the previous step and move onwards. For example, of all the players that start the tutorial by entering the game, only 0.01% (or 1 in every 10.000 new players) don’t move onwards to refuel their ship. Now, suddenly, that large time ‘leave system’ looks a lot more understandable as only 7.13% of all new players drop out here. We can take this to mean if you’ve reached that point, you understand the gameplay to the level where you have figured out what to do next.

Instead, we can see that the step ‘gather resources' loses nearly 20% of  new players despite taking a median time of just 9.20 minutes. Relatively short steps with steep drop-offs are our biggest design red flags and offer clear areas for improvement. We can look into that step specifically and attempt to determine the factors that cause this drop-off. Once we have ideas, we can start testing changes and, thanks to our robust data pipeline, do comparisons between cohorts to see the effect our changes have. We can, of course, add as many steps into this chart as we think we need to get a solid understanding of the key moments that play into them.

Just from my experience I have to wonder how many of those new players quit because they couldn't find the necessary resources? I have to guess that people complaining on the Frontier Discord server weren't the only reason more resources seemed to pop up near systems adjacent to the tutorial areas. Or did I just get lucky?

Of course, what works in Frontier works in Online. I expect that the Online game designers use similar information to adjust both seasonal events and expansion content. That quick iteration on content that never used to happen before? Apparently the logs don't show anything but Quasar does.