The CSM 12 Election As Research

One of the common descriptions of EVE Online's Council of Stellar Management is as a focus group. If I may use Wikipedia as a source...

"A focus group is a small, but demographically diverse group of people whose reactions are studied especially in market research or political analysis in guided or open discussions about a new product or something else to determine the reactions that can be expected from a larger population. It is a form of qualitative research consisting of interviews in which a group of people are asked about their perceptions, opinions, beliefs, and attitudes towards a product, service, concept, advertisement, idea, or packaging. Questions are asked in an interactive group setting where participants are free to talk with other group members. During this process, the researcher either takes notes or records the vital points he or she is getting from the group. Researchers should select members of the focus group carefully for effective and authoritative responses."

Kind of sounds like the two yearly CSM summits, doesn't it? For the current topic, I want to concentrate on the last sentence in the passage from Wikipedia. How does CCP select their focus group? In a typical, EVE Online game design move, CCP throws caution to the wind and lets the players decide and sees what happens.

The result is the annual CSM election. In effect, CCP conducts a one question online survey asking its customers who should participate in the focus group. By allowing the players to choose the members themselves, the developers can avoid charges of favoritism. Well, perhaps minimize the charges. The composition of the group itself could lead CCP to modify its development plans. What happens if two wormhole candidates or two lowsec candidates win? Is that a sign that the players in those areas of the game are disenchanted and may need some attention to avoid subscription losses?

If we look at the CSM election as a market research exercise instead of a political election, some of the numbers look quite different. In the real world, a turnout of 12.5% for an election is horrible. But according to SurveyGizmo, a company that makes market research software, the average response rate for surveys distributed to external audiences such as customers is 10-15%. For the years in which we have data, the participation rate in CSM elections falls within that range.

Continuing with the market research theme, SurveyGizmo came up with five factors that affect survey results, four of which I believe apply to CSM elections. They are:

• Customer Loyalty: Do the respondents feel a connection to the brand conducting the survey? A high level of loyalty will lead to a higher number of responses.
  
• Perceived Benefit: Whether it’s getting early access to results or being entered into a raffle, most respondents need to clearly understand the benefit of completing a survey.
  
• Demographics: Some sections of the population are simply more likely to response to survey invitations than others.
  
• Survey Distribution: If your audience consists mostly of digital natives, sending out links via social media will net you some good response rates. If you’re surveying retirees, you may want to choose an alternative distribution method.

Customer loyalty is a pretty basic concept. If someone only plays EVE for a couple of hours on the weekend, getting caught up in small changes in the game probably doesn't seem worth the effort. For those who play 3-4 hours a day, sending people to Reykjavik to make sure CCP doesn't mess up the game probably comes pretty high on the list of things to do.

The perceived benefit directly references how a player looks at the CSM. With 64 candidates on the ballot this year, one could spend 3-4 hours just trying to figure out who to vote for. If a player thinks the CSM is just a bunch of self-important nerds who are only running to win a trip to Iceland, the player probably won't take the time to vote. Of course, if the same player is handed a pre-made voting link and told that if he votes that way, he gets credit for participating in a PvP fleet, then the chances of voting go way up. Also, if CCP offers incentives for voting, as occurred in the CSM 10 election, that also raises participation.

Demographics is the major subject every year. Players in null sec vote, players in high sec don't. Players in wormholes and low sec vote when they get pissed off.

Survey distribution in my mind is just the ease with which one can vote. The voting through the launcher is kind of awkward, although better than in the past. Some people don't even look at the launcher when logging in, which depresses the participation rate. If CCP made voting easier, like putting the system inside the client, rates might improve.

The paragraph after the list of factors in the SurveyGizmo article applies directly to the CSM:

"An important participation incentive to survey respondents is that their opinions will be heard and that action will be taken based on their feedback. If respondents believe that participating in a survey will result in real improvements response rates may increase, as will the quality of the feedback."

The same holds true for the CSM. If players believe that picking quality candidates will result in improvements in the game, or at least prevent CCP from doing dumb things, then the participation rate in CSM elections will increase.

I know this post took an off-beat point-of-view of the CSM election process. I thought that perhaps looking at the process from a more business perspective in the form of market research might breathe some fresh life into an old, worn-out topic.