On March 12, 2012, a disgruntled fan of the Mass Effect series posted a message on the official BioWare forum outlining the reasons why he was frustrated with the Shepard trilogy’s ending. The post is addressed to Mass Effect 3’s developers and writers and outlines, under a series of bolded headings, a litany of their supposed failings. Some of the author’s frustrations sprung from the shortfall between the promise of making meaningful choices across three games and the reality of how players’ actions affected the three endings provided. Other grievances were based on perceived inconsistencies in the science fiction. In conclusion, he wrote, “BioWare… this game deserves a better ending. We know you can do better than this. Please, do not let us down in this way.”
Later that week, another member of the same forum posted a message saying that he had taken his complaint to a higher power than even the developer: the US Federal Trade Commission. He called on any other unhappy fans who believed that BioWare had somehow failed to deliver a satisfactory experience to do the same. Another user posted a forum poll in which you could vote on whether or not the developer should patch the ending in order to replace it with a new, “brighter” one. Implausibly, almost 100,000 people voted in the poll, an overwhelming 91 per cent in favour of the notion that BioWare should rewrite and re-release the ending. Yet another player launched a charity drive, asking players to donate money to Child’s Play as a way to demonstrate the depth of their discontent and put a curious kind of emotional pressure on the company (‘Fix the ending or else the children get… support?’). The fundraiser collected more than $80,000 before being shut down by Child’s Play.
All this rabble-rousing worked. Seven days after that first forum post, Dr Ray Muzyka, co-founder of BioWare and its then-CEO, released a strange statement in which he simultaneously defended the work of Mass Effect 3’s writing and design team, while pledging to “accept the criticism and feedback with humility”. Muzyka also revealed that, as a result of the pressure, his staff were now working on supplementary material that would “provide more clarity” to players “seeking closure” to their Mass Effect journey. In his post, Muzyka articulated a conundrum that creators of popular fiction have wrestled with through the centuries: how to square the artistic vision of a creator with the demands of the audience who pay for the work. “We’re working hard to maintain the right balance between the artistic integrity of the original story while addressing the fan feedback we’ve received,” he wrote.
This tussle between an artist and their audience has historical precedent. For instance, Charles Dickens’ The Old Curiosity Shop was published in instalments in the author’s own weekly periodical, Master Humphrey’s Clock. Within three days of publishing Chapter 53, in which a lovable character, Little Nell, visits an old church and has a conversation in a graveyard, Dickens had received several letters warning the writer to refrain from what many believed he was planning: murder. These readers took Nell’s visit to the graveyard in conjunction with the line that she looked “pale but very happy” as a foreshadowing of her death.
There’s an anecdotal, possibly apocryphal, story that in subsequent weeks American readers stormed New York City’s piers, demanding to know from visitors from England whether or not Nell dies. And Dickens was later swarmed with letters expressing anger and heartbreak over her eventual fate. Similarly, after finding out that the character Aeris dies midway through 1997’s Final Fantasy 7, fans sent letters and emails to the game’s developer, Square, either berating the team’s decision or demanding to know how it might be undone.
While neither Dickens nor Square acted upon their audience’s complaints, the gap between videogame creator and videogame consumer has closed considerably since FF7’s release, making it far harder for studios to ignore such impassioned pleas. No more does the game-maker who kills off a beloved set of characters, or who adds a controversial design, receive a mere sack full of letters. They are now inundated with petitions, forum posts and even guilt-inducing charity drives. In the most extreme cases, some developers have received personal threats and online harassment. In 2013, for instance, David Vonderhaar, design director of Call of Duty: Black Ops 2 was told he should “die in a fire” or “kill himself” by upset players after he announced via Twitter that the firing rate of one of the game’s most popular weapons had been tweaked from 0.2 seconds to 0.4 seconds. The volume of harassment became so great that Activision community manager Dan Amrich wrote a blog post in which he called the attackers “immature, whiny assholes”.
“The way in which game developers interact with players has been the most significant change in the industry in recent years,” says Zac Antonaci, who is head of community management at Frontier Developments and acts as a go-between, managing the demands of Elite: Dangerous’s playerbase and the development team. “Today, every element of game production has to consider and be mindful of the players. With all games moving more online and the development of the industry in so many ways becoming a more social experience, the value of a strong relationship between game maker and players is more important than ever.”
"There are logical reasons why we have chosen to not go down certain paths. That can be tricky to communicate to the community"
Things have grown yet more fraught in the crowdfunding era and with the rise of the amateur consumer-publisher. Many backers of nascent videogames such as Frontier’s Elite: Dangerous (which was funded via Kickstarter) hold the belief that their patronage should bestow special privileges – as stakeholders, they think that they should be given a say in a game’s direction, even at the granular level of game and character design. “[Crowdfunding] does change the expectations of both the game studio and the players,” says Antonaci. “Developers understand that by running a crowdfunding campaign, they are inviting players even closer to the process, which, as a community manager, is fantastically exciting.”
Perhaps, but as another community manager (who asked to remain anonymous) puts it, the relationship is also complicated and challenging. “Some community members who fund a game become entitled and think that they know better than the designers,” he says. “If they feel like their demands are being ignored, things can become ugly.”
Garth DeAngelis, lead producer on XCOM 2, agrees: “Listening to the community has become absolutely critical for game designers in recent years. But there is a fine line. The team making the game understand the ramifications of certain features and the knock-on effects in a way that players looking in from the outside simply cannot. We can’t act upon every single thing that the community asks for. Design by committee never works.”
Communicating this sensitively to irate players is a challenge. “The most passionate fans are often the most vocal,” says DeAngelis. “These characters are usually screaming for features that we have already examined very carefully internally. There are logical reasons why we have chosen to not go down certain paths. That can be tricky to communicate to the community.”
However, when the feedback coming from players is unanimous, it can offer one of the most useful gauges for a developer to use in prioritising a sequel’s design or features. “While working on XCOM: Enemy Unknown, we had in the back of our minds that it would be cool to have procedural levels,” DeAngelis explains. “After the game launched, we aggregated all of the reviews, the forum posts and the YouTube commentaries. Then we took them apart and created piles of feedback so we could see which criticisms were most widely maintained. We found that players wanted more variety in levels. That allowed us to prioritise the procedural feature for the sequel based on hard data. For us, that’s how it tends to work. Player feedback allows us to prioritise features on which we were already working.”
Other studios have taken this process of gathering and implementing feedback to an industrial-scale level. John Hopson heads up the user research team at Bungie in Bellevue, Washington. He leads a team of researchers who provide Destiny’s designers with player feedback, delivering data that can be used to inform design decisions. The studio invited players into the studio to begin this process long before the game was finished, using a range of techniques, from simple fundamentals to the highly advanced.
“We observed participants directly and made notes,” Hopson explains. “Then we had the participants fill out surveys to let us know what they thought. We also recorded every play session and connected it to our logging data so we could go back and watch key moments. Those video clips became one of our most effective communications tools with the rest of the studio, because when you can watch a dozen or so videos of players experiencing the same problem, that’s much more persuasive than any report.”
Hopson’s team designed a tool that would allow players to offer instant feedback while they were playing. “Participants could press combinations of buttons on their controller to tell us that the game at that moment was ‘awesome’ or ‘frustrating’ and so on.” These instant feedback reports were then connected to the video recordings, so a designer could sit down after a test and watch the exact moments where players said they were having a problem, or even the moments where players said they were having the most fun.
As well as active feedback offered by players, Hopson and his team collected and examined passive feedback. Destiny was the first Bungie game for which the team made extensive use of eye-tracking data, which showed how effectively players were able to navigate the game’s complicated user interface. “Knowing where players were looking and what they were reading was really useful in telling when players were having trouble understanding something, and when they were simply not noticing it,” he says.
"But knowing that [the outcry] was based on emotion rather than fact meant we could take the time to make the right fix instead of rushing something out the door"
Hopson and his team’s work had only just begun when Destiny launched, however, and he estimates that today they spend around a third of their time monitoring feedback from the public, and two-thirds testing future addons. “We start every morning by looking at data reports of how people are playing the game right now and monitoring forum discussions to keep abreast of player attitudes. We use this data to tell which changes will have the most impact and which fixes are the most urgent. For example, back when the Vex Mythoclast was first used in PVP, Internet forums were filled with players claiming that it was a super-weapon that made the Crucible unplayable. But from looking at the data, we could see that the actual impact of the weapon was a lot smaller than the emotional reaction.”
In the incident of the Vex Mythoclast, the customer, it turned out, was objectively wrong. But, according to Hopson, that doesn’t necessarily matter. “The emotional reaction was still true,” he says. “But knowing that [the outcry] was based on emotion rather than fact meant we could take the time to make the right fix instead of rushing something out the door and possibly breaking the game in some other fashion.”
The structured influence of player data and feedback on game design is a relatively recent development. “While analysing player data has been around for quite a while in games, it really took off with the success of the free-to-play genre,” says Seth Killian, former special combat advisor at Capcom, who worked on games such as Street Fighter 4 and Ultimate Marvel Vs Capcom 3. “In the world of packaged console games on a shelf, having a ton of data never meant that much, because you could only make limited changes to your game after it had shipped. Updates required a lot of time for approvals, testing by the platform holder, and often involved a significant fee. The developer also had less incentive to offer updates as the players you were studying had already spent the money to purchase your game, so there was limited or no opportunity to make additional money. With competitive free-to-play games, you expect to be making a lot of updates, so you have a chance to actually use the data you collect to address players’ problems and improve in-game sales.”
This may be true, but there was one far earlier era when player feedback exerted a significant influence on game design: the arcade heyday. Back in the industry’s earliest days, Atari would monitor how much money new games made at test locations. If these cabinets failed to pass a certain financial threshold, the game would be quietly canned. This rather simplistic implementation of player feedback become more sophisticated with the earlier Street Fighters. Capcom’s designers would launch a new entry in the series, and then carefully monitor top-level players in order to see if certain characters were overpowered. They could then fine tune the roster balance in the subsequent iteration.
“For us, using player feedback to tweak and improve competitive videogames isn’t a new phenomenon,” says Peter ‘Combofiend’ Rosas, associate producer on the forthcoming Street Fighter 5. In more recent years, however, the process has been improved thanks to data gathered from the game’s servers as well as the various communication channels that players around the world use to communicate with the company. “Our methodology and data targets have shifted recently,” Rosas says. “Nowadays, with YouTube, Twitter, Capcom-Unity, and Facebook, there are many ways that players can reach out to us to share feedback. A lot of those players aren’t professionals, so it provides us with a variety of data, from those just getting started all the way to our hardcore.”
Balancing a game like Street Fighter, which is played in high-stakes cash-prize tournaments and which features a large character roster, is a tremendous challenge, but it’s undoubtedly one of the most important elements of the series’ design. If the community comes to collectively consider a competitive fighting game as being imbalanced, it can ruin its reputation irredeemably. “We’ve found [from looking at the data] that there are certain types of losses to players that are more acceptable than others,” says Rosas. “By this, I mean that if a player loses to something their character does not have the tools to stop, or in an imbalanced match-up, the disparity will dissuade them from playing the game further or from taking the loss; the blame is shifted from them to what they might perceive as a poorly designed game. If they feel as though the loss was justified, the odds of them sticking around to play again greatly increase: it becomes a personal challenge that they want to overcome.”
CCP, developer of EVE Online, has gone further than most in creating systems whereby its audience can feel heard in steering the game’s direction. In 2008, the developer commissioned a study into the political state of the MMOG’s game world. The study’s authors argued the need for a player-run political body, concluding: “EVE’s society must be granted a larger role in exerting influence on the legislative powers
of CCP.” In response, CCP established the Council Of Stellar Management, currently the only example of a game-based democratic organisation designed to represent a playerbase. The council has 14 places, and each year scores of candidates stand for election, campaigning in the game on particular issues, making promises to other players to effect certain changes.
Every six months, CCP flies the successful candidates to its headquarters in Reykjavik for three days. During that time, the council meets with the studio’s staff and hears about the new features planned for the game’s future. Much like on Internet forums or in Kickstarter campaign comment sections, the debates can become heated; often council members disagree with one another. But the council performs a crucial role: it bridges the gap between the game’s players and creators.
Some players believe the council is little more than a token gesture, but in 2011 they were offered some strong evidence to the contrary. At the time, CCP introduced a new feature, one that had been 18 months in the making: a digital store for in-game items that could be purchased for real money, including – brazenly – a monocle with a $70 price tag. The feature was announced in a grand reveal by CCP, but it was the last thing that the majority of players were interested in, especially when the game was, at the time, suffering from numerous technical issues. Players complained and these complaints turned to anger when an internal memo from CCP’s CEO, Hilmar Pétursson, leaked. In it, he dismissed the players’ reaction to the new store as mere noise.
That week, thousands of players gathered in the game and began staging symbolic riots. Many fired their ships’ weapons on a giant monument stationed outside a major trading hub. The protest marked the beginning of what later became known as the ‘summer of rage’. CCP immediately called an emergency summit in Iceland for the council. It proved effective. Acting on the council’s advice, Pétursson wrote an open letter to the game’s players admitting that he had made a mistake. Furthermore, the developer acknowledged the events in game: it erected a battered memorial at the place where the majority of the protests had taken place.
For Frontier’s Antonaci, regardless of whether or not a developer chooses to act upon the feedback given by a game’s players, it’s crucial to demonstrate that grievances have been heard. “I think it’s about intelligent listening,” he says. “Any idea that is proffered in a considerate way is a good idea, since it means someone actually loves your product and wants to be involved. Ideas need to be filtered, but regardless of how you do this, or whether or not you implement the idea, the key is to always be sure to let the community know how the process is working. Always, always close that loop.”
Read more from Edge here. Or take advantage of our subscription offers for print and digital editions.
