The Zero-Sum Funomaly pt.5
Sunday, November 6, 2011 at 9:00PM In game design and psychology rewards and punishments are rich, complex topics to say the least. I'll explain why step by step.
First we have to replace our common notions of rewards and punishments with the definitions from psychology and operant conditioning. The common definition of punishment is "to subject to pain, loss, confinement, death, etc., as a penalty for some offense, transgression, or fault" ~ dictionary.com. In the general sense, a punishment is any undesirable environmental change or stimulus, or a change issued from an authoritative position in response to a wrongdoing. The problem with these definitions is the ambiguity of authority (who has the power), the subjectivity of what is undesirable, and the lack of a functional component.
It's about the same with the common definition of rewards: "something given or received in return or recompense for service, merit, hardship, etc." ~dictionary.com. This definition is good. The only problem is that many conflate the definition of rewards with reinforcements. We can sort through everything after considering the following. From wikipedia:
It is important to note that actors [players] are not spoken of as being reinforced, punished, or extinguished; it is the actions that are reinforced, punished, or extinguished. Additionally, reinforcement, punishment, and extinction are not terms... restricted to the laboratory. Naturally occurring consequences can also be said to reinforce, punish, or extinguish behavior and are not always delivered by people.
Reinforcement is a consequence that causes a behavior to occur with greater frequency.
Punishment is a consequence that causes a behavior to occur with less frequency.
To put it clearly, the majority of the fun players can have interacting with video games is a result of the intrinsic motivators of power and curiosity. In order to overcome goals, which are necessary for games, players typically need to learn (develop DKART skills) and execute. Since trial-and-error is our most commonly used learning method, it's very important to look at the feedback a game gives to player actions. The feedback that we need to learn from our trials is called results, output, reactions, or simply consequences. Assuming that the goals players accept are the goals of the game (mandatory or optional) we can look at any action-consequence pair of any gameplay action and categorize it as reinforcing or punishing.
Looking at design this way, it's clear that rewards and punishments are more about learning than making the player feel good or bad as ends to themselves. It's also clear that for all games where it is easier to fail than win (which is most video games), there will be more punishing consequences for emergent actions than rewards. Unwanted consequences are inherent quality of emergent gameplay. This is also why it is erroneous to consider eliminating punishment from game design. Designers who seek to create such a zero-sum funomaly game don't understand fun, skill, learning, punishment, or emergence.
There's more to this discussion. When gamers and designers talk of rewards and punishments, I think they most often speak of a slightly different issue. When a gamer says they don't want to be punished, they're not saying they don't want unfavorable consequences to their actions. They're saying that they don't want to be discouraged from learning or executing. Because if this happens and the intrinsically motivated foundation falls apart, the fun of the gameplay experience falls with it. Like with any student or learner, what works for one doesn't work for all. While one player may get discouraged from having to play part of a level over, another player may be encouraged. Everyone has different limits, preferences, and different experience learning in different ways. For these reasons rewards and punishments are personal concepts that are largely shaped by player attitude.
For most gameplay challenges the journey and winning is the reward. In other words, the experience is the positive compensation for all the hard work you put in reaching the goal. At least, this is what goal-setting theory tells us. The idea is that we're motivated to accept goals, pursue, and accomplish them perhaps because it appeals to our intrinsic needs to exert power and to collect experiences. For beating a game we don't get money, food, or fame (well maybe outside of MLG). We do it because we like playing games and overcoming challenges. We do it to learn more about the story, the gameplay, or ourselves in the process. But, just because the game screen says "you win" doesn't mean we automatically feel satisfied.
Sometimes I win and feel like a loser. When I get through a challenge by brute forcing a problem or rolling the dice after far too many attempts, sometimes I feel like I've cheated. Knowing that I didn't figure out a solution on my own or solve it elegantly takes away most of my fun and sense of accomplishment. It's about the same when I look up how to get through a tough section in game or I get someone else to beat a challenge for me.
On the other hand, there are times when the game says "game over" and I feel like a winner. After spending many hours learning how not to die in the platformer rougelike Spelunky, I finally made it to the final boss. Since I hadn't previously encountered the boss, a giant golden head statue (see image above), I didn't know how to beat it. While I safely assessed the situation resting on top of the bosses head, the game glitched and I instantly died. I'm pretty confident that I would have beaten the game that day. But because of the glitch, I was never able to make it that far again. In my mind, I beat Spelunky.
Arcade style games like Pac-Man, Geometry Wars, Tetris, and Flydrill are designed to be endless. So eventually, you will lose and see that "game over" screen. If you do well, beat your own high score, or advance in the leader boards you may feel like a winner.
With competitive multiplayer games, there are times when I've fought and lost to much higher level opponents. With Smash Brothers Melee, I've played some of the best players in the world in their prime (Mew2King and SephirothKen). In these battles, I didn't worry about winning or losing. I focused on learning as much as I could. In the end, if I picked up a new technique, took off a few stock, or learn something that would aid my future battles, I felt like a winner.
The psychology of fun and game design runs far deeper than I'll cover in this series. To conclude this article, I'll briefly discuss a few more concepts.
From what we know about extrinsic motivators, poor reinforcement design seems more dangerous to fun than poor punishments. At their worst, punishments from video games are undesirable because of lost time and effort (see more about tension here). Another way to put it is that we cannot suffer any real loss by playing a video game (physical harm, restricting food/sleep, etc.). The more skill you exert playing skill based games, the less frequent punishments occur. Plus we know to avoid punishments to improve our overall experience. On the other hand rewards are generally great. We want them, we seek them, and we enjoy obtaining them. But rewards can also take away from intrinsic motivation. As I've explained previously in this series, losing intrinsic motivation is a big deal.
Some video games systems are designed like casino games, using unpredictable rewards to hook players into addiction. From the psychology of loot by Jamie Madigan:
Why? What is it that makes loot drops so effective at enticing us to keep going when we could be playing something else that by all accounts should be more fun? The answer, it turns out, is not so much the loot, as it is the loot drop. Of course, we love some equipment we find because it lets us do new things or do old things better. But it turns out that what really pushes the buttons in our brains is the act of finding something new. Actually, that’s not even it, exactly—it’s the anticipation of finding something great. How do we know this? Look no further than the nearest neuroscientist with a lab monkey, and a fist full of electrodes.
Extinction is the lack of any consequence following a behavior. When a behavior is inconsequential (i.e., producing neither favorable nor unfavorable consequences) it will occur with less frequency. When a previously reinforced behavior is no longer reinforced with either positive or negative reinforcement, it leads to a decline in that behavior.
The concepts behind the psychology of loot explains why ?-blocks in Super Mario Brothers retain their sense of excitement. When exploring these clued secrets for the first time, you never know if you're going to get a powerup, a starman, a 1up Mushroom, or measly coin. The anticipating and surprise of discovery is key here. However, the concept of extinction explains why look-alike secrets retain their effectiveness after many examples even repeated. Take the plain looking brick for example. For every brick you break, you can get coins, a 1up Mushroom, Starman, a beanstalk, or nothing. In fact, most bricks give you nothing. The reason why these look-alike secrets are so effective is that when players explore the duds they get discouraged. When nothing happens after the 10th brick you break or the 5th pipe you fail to slide down, you tend to stop searching every possibility. This is extinction effect working. This decline in behaviour then sets the stage to be surprised and excited again at the next unanticipated discovery.
The following factors affect the effectiveness of shaping behavior via consequences (reinforcement or punishment). From wikipedia...
Immediacy: After a response, how immediately a consequence is then felt determines the effectiveness of the consequence. More immediate feedback will be more effective than less immediate feedback.
Consequences come quickly in most video games, especially action games. When you get hit, you see the consequences play out in the seconds that follow. It's harder to see the consequences of very dynamic, emergent, or suspended consequences. If you lose a character early on in Fire Emblem, it's hard to impossible to see how it will affect your strategic viability 20 missions later. Or in a difficult puzzle challenge that takes 20+ moves to solve, it can be difficult to see if you're heading down a dead end move by move. In both of these cases, the lack of immediacy makes it harder to learn some lessons.
Contingency: If a consequence does not contingently (reliably, or consistently) follow the target response, its effectiveness upon the response is reduced. But if a consequence follows the response consistently after successive instances, its ability to modify the response is increased. The schedule of reinforcement, when consistent, leads to faster learning. When the schedule is variable the learning is slower. ~wikipedia
This factor explains why gamers generally have a hard time with random elements. Whether from percentage chances to miss an attack or a random roll of stats, getting inconsistent results is hard to learn from because the effectiveness of the consequence is weakened. Read more about randomness and how gamers respond to it here.
Size: This is a "cost-benefit" determinant of whether a consequence will be effective. If the size, or amount, of the consequence is large enough to be worth the effort, the consequence will be more effective upon the behavior.
If the consequence isn't desirable or undesirable enough, the effectiveness of shaping behavior drops. With infinite continues, checkpoints, and respawns the sting of death can be taken away. This can have serious adverse consequences to learning. The vita chambers system in BioShock is a very lenient checkpoint system. When you die, you respawn at specific chamber locations with all your guns, ammo, and all changes you've made to the game world preserved. So if you take down the boss to half health before dying, you can return to the spot and resume. This design made seriously avoiding death less of a priority than other behaviors for me.
I know that I presented lots of ideas in this article. The big take away is that punishments are just as important to fun as rewards. In fact, you cannot separate both from emergent, skill based games. In part 6 we'll look at how difficult it is to use fun when evaluating games.
Reader Comments (2)
Ha, Flydrill. :)
Immediacy, contingency, size. I was just helping to demo Foldit, watching a bunch of kids trying the intro puzzles. I am reminded how tough it can be to make feedback clear (contingency) when a system is so convoluted - it can easily seem random. And similarly, size is hard to correlate with action when the dynamics are so chaotic - what is seemingly the same action can have very different consequences, and some very important factors (packing, hydrophobic hiding in Foldit) can easily get drowned out because they have a much more subtle effect on score than others (colliding atoms). Makes me want to make a 2D protein-folding game, with a more simplified, comprehensible scoring system and a simpler, more direct interface. It would be a lot easier to create a progression of skill and difficulty with that, I'd imagine.
@ axcho
I thought I responded to this comment. I guess I didn't.
Yes, I remember Foldit well (especially after youtubing it real quick). It was pretty dense and complicated. Maybe the 3D would have been handled better with some handheld AR technology and some stereoscopic technology.
Who knows. But thinking in terms of 2D is never a bad thing. Too bad that in this case, protein folding is a strictly 3D coinsurance.