Fanfest 2025: Upgrading CARBON to Python 3

About six weeks ago I wrote about CCP's efforts to upgrade the Carbon engine that runs both EVE Online and EVE Frontier from Python 2.7 to Python 3. Today I attended a session the provided more details about the effort.

CCP Aporia

Given by CCP Aporia, the presentation provided some additional details not known before. Or perhaps details provided a long time ago and just forgotten. One fact I probably should have expected is that the Carbon Engine which powers both EVE Online and EVE Frontier was extracted from EVE Online. I'm not sure how often such a process is as I believe today studios begin with a game engine (or create their own) before developing a game. Then again, the development of EVE began in 1997. Best practices in game development have probably changed over the last 30 years.

Now, I knew that Python 2.7 was depreciated in 2020 from the previous dev blog. One point I didn't think about was the lack of new programmers who didn't have experience in Python 2.7. I should have realized as the classes I've taken in Python over the years have all been conducted in Python 3.x, with the latest taking place in Python 3.12. 

A clever explanation

CCP Aporia did receive a question about why, if Python 2.7 was depreciated so long ago, was CCP just now getting around to upgrading the engine. He gave a rather generic answer but I think I know why. The developer told up that work on the upgrade began in May 2023 with initial planning beginning in March of the same year. Looking back, CCP released a press release announcing what was then known as Project Awakening on 21 March. Coincidence? Perhaps, but I think the information cements my theory that the Carbon engine upgrade occurred because of EVE Frontier.

Another interesting fact is that CCP, in conjunction with their partner Recked Digital, actually performed two upgrades. The first, upgrading from Stackless Python 2.7 to Stackless Python 3.8.1, was finished in November 2023. But then CCP made the decision to upgrade Carbon from Stackless Python 3.8.1 to Python 3.12. A forward-looking move as Python 3.8 was scheduled to be depreciated in October 2024 and Stackless Python would be archived in a read-only state in February 2025. The version of the Carbon engine upgraded to Python 3.12 was first used by EVE Frontier in June 2024.

Some of the benefits of making the switch to Python 3.12 is better observability of problems. A new tracemalloc package in Python 3.12 is much more effective at finding memory leaks than CCP's existing in-house tools. tracemalloc is a built-in module in Python used for memory allocation tracking. It helps developers understand how much memory their code is using and where potential memory leaks might be occurring. Key features of tracemalloc are:

• Tracks memory allocations, including the size and location.
• Provides snapshot comparisons to identify unexpected memory growth.
• Helps debug high memory usage issues by identifying problem areas in the code.
• Displays statistics on memory allocation per file and line number.

In addition, upgrading to Python 3.12 means receiving the latest bug fixes and performance improvements as they are released. Also, hopefully the developers will have the ability to implement a native replacement for the global interpreter lock once one is developed that is faster than the GIL.

The upgrade improved the performance of the Carbon engine by an average of 20%.

Now for the big question. When will EVE Online receive the upgraded engine? According to CCP Aporia the question remains up in the air. Two of the reasons are pretty straightforward. EVE Online is a live game and has a lot of players. Does CCP want to provide an unstable server for players for a week or two? I'm pretty sure the answer is no.

The third reason CCP Aporia talked about is one I intellectually knew about but didn't really consider before. EVE Online has a lot of history. A lot of history means a lot of data. Probably pentabytes (1024 terabytes) of data. Python 2.7 & Python 3.12 handle data in different ways, meaning data stored for handling by Python 2.7 may not be compatible with the way Python 3.12 processes data. I'm pretty sure players would hate to see big stacks of items disappear during a transition.

We may hear more information about the Carbon engine on the final two days of Fanfest. But for now I'm waiting for more news on when the Tranquility shard will receive another upgrade.