Thursday, December 27, 2012

CCP's War On Bots: 2012 In Review

"Security issues from our perspective extend beyond things like infrastructure, hardening the firewalls and all those fancy security things but I believe they also extend to the RMT and botting situation.  I think that is a security problem.  I see catching bad guys as catching bad guys.  I don't see it as a customer service thing."

CCP Sreegs, Fanfest 2012

The year 2012 started off with one big question for those concerned about botting and illicit RMT: could CCP afford to continue the war it began in 2011?  Following the Summer of Rage caused by the Incarna expansion and related events the number of subscriptions had fallen 8% down to 340,000 in December 2012.  Many players had dug into CCP's finances in June 2011 and discovered a loan that was due on 28 October that probably contributed heavily to CCP announcing a 20% reduction in its workforce on 19 October.  With all of this bad news could CCP really afford to ban any accounts no matter how shady?


CCP Sreegs Unleashed - Players received an answer to this question early in the year when the newly formed Team Security resumed CCP's anti-botting operations on 26 February 2012.  Within 48 hours tears from the H-Bot forums leaked onto the official Eve Online forums, prompting a dev blog from CCP Sreegs on 1 March.  In the dev blog he addressed the lull in the War On Bots™.
"As you are all aware the company has gone through a lot of changes in the recent months. Because of this there was a period of time where nobody had responsibility for handling the technology responsible for nuking botters. As of now there is a formal team on the EVE project devoted entirely to security, of which I am the product owner which is a fancy word for manager. This means that we've now thrown the switch again and turned on the catching bad guys machine because we own it and we don't like cheaters."
In addition to announcing the banning of 1000-2000 accounts CCP Sreegs also announced that accounts receiving any type of ban for botting will have all the characters locked to the account.  This was an additional signal that CCP didn't need the money from botters as botters had made a practice of laundering their discovered botting characters on the character bazaar.  Before CCP Sreegs published his dev blog, a botter on the Questor mission bot forums wrote:
vicious666 (28 February) - "tbh is more than a year and never got such issue, so seams now ccp have some "will" to stop questor , or is an economycs trick, 3k account banished,so in 2 weeks all 3k gonna change account-transfer/sell chars ? is a 100000$ dollar operation for them (in plex used) and they not lose a single account"
I'm not sure of the botter's math but at 20 EUR/USD per character transferred CCP had made a lot of money off of the War on Bots™.  If CCP were still in financial difficulties I don't believe this move would have occurred.

Perhaps more importantly for the War on Bots™ CCP now has a formal team dedicated to security consisting of:
  • CCP Sreegs - Product Owner
  • CCP Stillman - Security analyst specializing in application vulnerability
  • CCP Arkanon - Internal Affairs Investigator
  • CCP Peligro - Internal Affairs Investigator

While this team is responsible for all internet security CCP Sreegs believes that anti-botting and anti-RMT efforts fall under his team's responsibility.  As he stated at Fanfest, "I see catching bad guys as catching bad guys.  I don't see it as a customer service thing."

Not A PR Stunt - CCP uses Fanfest not only as an event to reward players who brave Iceland's weather but as a giant marketing event.  Many players, including veteran botters, look upon any security measures taken around the event as a public relations stunt and if botters just hunker down and act with care that CCP will eventually turn its attention somewhere else.  CCP Sreegs job was to shatter that belief.



In a dev blog published on 4 April CCP Sreegs announced that the automatic banning process would run daily instead of weekly in order to deny botters any safe windows in which to operate.  A bigger move that produced tears was that Team Security would seize all ISK gained from botting when the ban takes effect.  During Fanfest weekend Team Security also struck illicit RMT operations, permanently banning 105 accounts, seizing between 1-3 trillion ISK in assets and reversing 500 billion ISK in RMT transactions.  In the three weeks following that dev blog, Team Security continued its illicit RMT crackdown permanently banning 1268 accounts and seizing 4.2 trillion in assests.  The crackdown also affected illicit ISK buyers as CCP reversed 1.5 trillion ISK in transactions from 524 additional accounts. 

On the botting front bot improved detection methods appeared and were noticed by bot developers.  Slav2, the developer of the Eve Pilot family of bots, had this exchange at the end of April:
Famine:  "But i think Slav has mentioned that single users of his program have not been banned since the middle of 2011 or something."

Slav2 (developer of Eve Pilot):  "This statement was true before resent bans. Pre festival time has traditionally higher risk of a ban, when CCP collecting statistics to show in graphs. I think CCP made some changes in bot detection after festival. Some new ideas could appear when GMs collected statistics. Now single bots who use VMWare may be banned too."
Unintended consequences - Team Security is not the only group of developers at CCP who can influence the War On Bots™.  The news from May and June was dominated by the effects on Inferno on botters and thus on RMT operations.  CCP Arrow's Unified Inventory system probably created more botting tears than any other single event in Eve Online's nine year history with the possible exception of Unholy Rage.  Within 24 hours the tears started to flow onto the forums of all Eve bots.  Usually after an expansion launches the bot devs can usually get their bots up and running within 1-2 days.  On an expansion with a lot of changes possibly 3 or in extreme cases 4 days.  But with Inferno I saw complaints up to 2 weeks later and some functionality was never restored, partly because of the continuous software fixes CCP was implementing that would break new things in bots.  And during this whole time Team Security's automatic banning process churned away banning botters already frustrated with the patch.  The Eve Pilot users were hammered once their bots were fixed, making June a particularly tearful month.

A Sleepy Summer - The months of July, August and September were fairly quiet as Team Security's software continued churning out bans.  This quote from the H-Bot forums summed up the feelings of many botters:
frblyes (H-Bot, 31 August) - "so are the bans still being given out like free candy from a pedafile to little school girls?"
One of the most amusing episodes in the War On Bots™ occurred on 12 July as a botter decided to live stream his botting session on Own3d.tv.  Not only was the botter used in a game of bot bowling by a group of developers but banned and then fired from his position working for Eve Hold'em.

In early August users of the Eve Pilot bots discovered that Team Security had put into place a system that would not allow players to log into the game from a computer previously detected as used by a banned player.  Either the launcher of the client uploads hardware system data that allows CCP to digitally fingerprint a computer.  Those attempting to create a new account reportedly just receive a message stating the account is waiting to be verified.

The Next Phase - With botting use suppressed Team Security decided to devote more effort in battling illicit RMT operations in New Eden.  On 6 October at Eve Vegas CCP Stillman made the following statement:
"The point of this video is that the social acceptance of botting/RMT is going to come to a stop.  We said at Fanfest this year that we were going to eventually look into it and we feel that we are at a point where we've dealt with the low-hanging fruit we can do in terms of detection of botting. 

"The next big issue is really the alliance acceptance of this kind of bad behavior.  So this was the first that we took a bunch of corporations we banned; we're going to wipe the POSes and everything they have.  And what we are going to end up doing coming up is that we're going to make sure that if an alliance is hosting botting/RMT they're not going to continue doing that. We're going to make sure that they feel the pain.  But we don't want this going on.  We don't want it to be an alliance (unintelligable). It's crazy what's going to happen. And we're not going to do anything drastic or anything, really, like an Unholy Rage against alliances. 

"We're going to adopt the same philosophy as we do with our botting/anti-botting work is that we are going to go for a slow burn.  We're going to measure every step we take.  We're going to see what impact it has.  We're not going to kill the whole alliance overnight, like, 'Hey guys, bad luck,' at least not initially.  We're going to make sure that everybody understands what's going on. 

"If you're doing this it is not going to be a fun time for you.  And we're going to see where it takes us because its one of the big elephants in the room right now.  We're going to address that and its going to happen over the next couple of months." [emphasis mine]
CCP Stillman's presentation created a furor amongst those in null sec.  In an interview published on 15 October CCP Sreegs clarified his position.
"I believe there are some items being blogged about and coming out of EVE Vegas which are being severely misunderstood and I’d like to provide some clarification. In essence the alliance action system as discussed is being billed as making alliances responsible for botters and this is far from the mark I wanted us to be at. My first question in that when we discussed it internally was 'What tools would we give people to police actions against bots from an alliance level?' and the answer was clearly that we couldn’t so we weren’t going to go in that direction merely for botting. This is coincidentally one of the questions Stillman was asked in Vegas lest anyone feel our thoughts are out of sync with the players. That is not to say that we will never focus on bot-friendly alliances but that it’s not the scope of where we’re focusing today. 
"Our efforts and actions we’re talking about here are purely related to RMT, which can take the form of botting but which takes other forms as well. We’ve seen alliances which have RMT as a part and parcel of their leadership structure. They use various methods to attempt to shield that from us but we’re seeing it anyway. We’re not naming names YET (though we may well do so) but those alliances should feel on notice that they can end their RMT related activities whether that be in the form of using 'Rental' agreements or private forums on their own alliance websites or any other methods they’re currently using to violate the EULA or we will end it for them. 
"Forming an alliance and having a bunch of people join it or taking over an established alliance and twisting it into a personal business for the leadership does nothing to change people’s obligations under the EULA and we want to send a very clear message that if they think somehow it does they’re incorrect. Players who see this behavior should both report it to us and inform their alliance leadership that it’s unacceptable, or move on to other alliances because to be honest with you that alliance does not have a future given its current path. 
"At this point in time, as Stillman stated later, we’re only referring to a very small number of alliances and the VAST majority of alliances need not be concerned. In this case we’re talking less than 10." 
The first indication of CCP making a major move against illicit RMT operations in null sec occurred on 31 October with a leak of an Eve mail stating that an Against ALL Authorities CTA was cancelled due to the banning of 20 logistics pilots for illicit RMT.  Three weeks later CCP cleared the -A- pilot running all 20 accounts of wrongdoing but the rising price of illicit ISK indicated that Team Security succeeded in banning other operations supplying ISK sellers.  One site, InGameDelivery, stopped selling illicit ISK sometime between 4-11 November.  Another major gold selling site, the Hong-Kong based Koala Credits, ceased selling ISK on 21 November and resumed sales sometime between 3-6 December.


The rise in the prices Koala Credits offered to customers began to rise dramatically just after CCP apparently began their campaign.  Another large gold selling site, the U.S.-based Avatar Bank located in Pompano Beach, Florida, never stopped selling ISK but kept its price to just under what a player could purchase ISK for buy purchasing PLEX from CCP from 18 November to 9 December.  Interestingly Avatar Bank and InGameDelivery have some sort of relationship for delivering in-game currency in several games.  The fact that InGameDelivery is still out of the ISK selling business could indicate that Avatar Bank is still facing supply issues.

Conclusion - This year opened with the question of whether CCP would decide to continue its war against botters and the illicit RMT trade.  Some people argue that CCP want the extra revenue from botting accounts and don't mind the activity in their game.  I personally believe that allowing botting and illicit in-game currency sales harm a game; the most recent example is when ArenaNet's Guild Wars 2 was overrun with botters earlier this year.  That game eventually needed to hire an outside company to help take care of the problem.  Getting a reputation as a game hospitable to botters is bad for business and I am glad that CCP looked at the longer term health of the game rather than go for a quick money grab.

The other major story for 2012 is how much progress Team Security made during the year.  Beginning operations in late February, Team Security managed to suppress botting operations enough that efforts to tackle illicit RMT activities in null sec alliances was seen as a more efficient use of resources than increasing efforts against botters.  The fact that such anti-RMT operations apparently affected supply for some of the largest gold-selling companies in the weeks leading up to the Retribution expansion calls into question just how big an influence some of the null sec alliances have on the illicit RMT trade.  Perhaps that relationship is driving the next phase in Team Security's plans.

7 comments:

  1. Team Security is an absolute gold mine for the company. Most people wanting cheap isk will take the path of least resistance, weighing risk of getting caught against amount of isk gained. Some people may have been botting to support expensive pvp habits - they may turn to selling plex instead. Some people may have bought illegal isk because it's cheap and reliable, they'll turn to selling plex instead.

    Additionally professional outfits very often use stolen credit cards to pay for accounts operating illegally as they suspect the account won't be used for long anyway. Scott Hartsman last year said: "I would dare anybody to ask an exec at a gaming company how much they've had to pay in MasterCard and Visa fines, because of fraud. It happens a lot."

    http://massively.joystiq.com/2011/07/21/scott-hartsman-says-gold-farming-hurts-our-games-more-than-we-kn/

    So although Team Security may reduce player numbers it more than compensates by raising profit per player.

    ReplyDelete
    Replies
    1. There are other articles around with SOE President John Smedley stating back in 2008 that credit card charge backs cost SOE $1 million in a 6 month period.

      Delete
  2. You may want to correct your Incarna reference in the first paragraph. It reads Inferno.

    That said, nice summation. Thank you for keeping up with this and showing us the repercussions outside the community. It may be naive to think botters and RMT will one day be eliminated, but I'm still hopeful.

    ReplyDelete
    Replies
    1. Fixed. Thanks for pointing that out. I'm always getting the names mixed up.

      Delete
  3. Nice article, but you need a clearer separation of RMT and bots. They are not the same thing. An RMT may use bots, but a botter is not necessarily participating in RMT.

    Unauthorized RMT is stealing money from CCP. Plain and simple. Every 500M ISK which is bought from an RMT is one less PLEX that should have been bought from CCP. RMT should never be allowed, nor supported by players - in any way, shape or form - just as trafficking in stolen goods is generally not allowed by most countries. Do you buy stolen goods in RL? No? Then you should not be buying ISK from an unauthorized RMT.

    Bots are nothing but a tool to grind ISK in the game. RMT doesn't really need bots - they can always use cheap overseas labor and trial accounts to do the same thing, at a slightly lower profit. Bots, in of themselves, do not affect CCP's profit margin.

    CCP has chosen to ban bots in their EULA, since they say that bots can destabilize the game. That is fine - just as is banning griefing, inappropriate behavior, etc. CCP can ban whatever they chose to ban - they own the game.

    However, if botting can destabilize the game, then the same thing can be said of any AFK activity which allows players to grind massive amounts of ISK, and CCP should be more proactive about dealing with these issues. And, yet, they were irresponsibly slow to fix the FW plexing issue and have made AFK ice mining even more profitable and easy. This lack of clarity is why botting often remains controversial.

    Separating the issues makes it easier to draw a line. Many players can be easily convinced against supporting RMT, but don't see a problem with botting. RMT is stealing; botting is merely cheating. If you chose to group them together, though, you are going to inevitably get players who end up supporting RMT, just because they think the ban on bots is unreasonable, given that CCP supports/allows other forms of AFK ISK grinding.

    ReplyDelete
    Replies
    1. With the reduction of especially in the mining industry, it has made that occupation viable to the non Botters. Before I was having to compete with a bot for mined mineral. Now I don't, the value of the minerals I mine is now up 50 to 75%

      In null sec it was easier to just jump the cheap minerals up after making them into compressed materials like armor plating.

      For Ratting bots made LP point less valuable for mission runners who would grind their isk earning that way.

      The only people it didn't effect were the PVP players who did sanctums who got isk quick and were able to buy the Minerals and LP items from High sec.

      Bots devalued Mission running and Mining occupations for newer players. Bots also were a bog on Resources. 2% of the active users were taking 30% of the computer resources. These Bots were a major contributor to LAG.There has been fewer accounts of Soul crushing Lag and TIE die, since the botter's not may be sharing that of a pvp battle.

      Botters also steal accounts subscriptions.
      Botter is able to plex their account easily with the stuff they mine and sell,
      I on the other hand have to pay for my subscription because I don't have enough isk to plex and buy my ships.
      As a result I would have had more accounts, but I can't because it is too expensive.
      Now with the increase in the price of my goods and and I can afford a plex once in a while to supplement my accounts.


      Delete
    2. Bots are key to RMT, With out them, RMT is hampered.
      Since you now have an isk printing machine, your bot, you can now sell that isk for Real money or make things to sell for Real money.

      Your bot is an Isk printer. where do you draw the line, feed you starving internet space family or actually start feeding your starving Real Life family.
      Do you only do enough to get by or do you start making a minimal living.
      Do you make a minimal living or upgrade to several accounts and computers and start making an enterprise from it.
      Do you follow the dark side like http://www.markeedragon.com/ once did, (Check his online videos) but then went fully legit later. (I buy time codes from him though Somberlottery.)

      Delete