Game/Mental State is an article series that looks at how and why mental states can be interpreted from game states. In other words, we're looking into our minds by looking into the game. This concept holds immense potential for developing machines and software that can truly work with and for us. That can truly know us in a way that only the dedicated, patient, and insightful have known us previously. The very idea of being able to measure, express, and quantify parts of our mental process via software output may seem like science fiction, but as I've begun to explain in part
one and
two, it's possible, it's happening, and we've only begun to scratch the surface.
Furthering the discussion the next three entries into this series will cover the following topics; interpreting gameplay as a type of language that players can use to align their mental states. Exploring the science behind mental channels, a sub-facet of knowledge skill. And examining the limitations of short term memory particularly when we use external tools to extend our minds.
silhouettes from
Olly Moss merged with image from radiolab.com
What does it mean to truly know someone? To get what they're saying or to know how they feel? Perhaps sharing brain states gets us closest to achieving a shared experience. It seems like mankind has never found an ability to transfer emotions and experiences directly between people. So, we've always used intermediary tools like language to create a bridge. Verbal language, body language, pictorial images, and by extension playing a video game are behaviors coded with patterns designed to what's inside our heads to others. It comes at no surprise to me that you can interact and know someone purely through their interactions within a virtual system. That Game Company's Journey highlights this phenomenon by stripping away all verbal and text language options between online players, but all emergent games have potential for such self expression, identity, customisation, and therefore language. The more emergent, the more potential.
"You know the difference between explanation and experience is like the Grand Canyon... well, she found a way to quantify the gap." ~ Abumrad from Radiolab
To frame this idea, you must listen to this radio lab podcast titled
"Are You My Brain Double." If you're short on time, start at 5 minutes. The key takeaway is that two independent brains light up in the same areas at the same times when the two people are synced. Focus and emotional response are important for synchronization. In this case, Lauren1, the storyteller, had a vested interest in her personal story because it carried meaning and emotional significance for her. And Lauren2's, the most synced candidate, ability to become immersed with a super dream state like focus allowed her to sync up best with Lauren1. Clearly
engaging, challenging, and interactive design is very important here.
Understanding the interaction between a speaker’s brain and a listener’s brain in the context of real-world communication requires the development of new experimental paradigms. Using function Magnetic Resonance Imaging (fMRI), we measured neural signals from two brains (a speaker and a listener) during a complex everyday communication. We then built a simple, interpretable model that leverages the dynamics of fMRI and uses the speaker’s brain responses as a model for predicting the brain responses within the listener. Our model reveals that during successful communication, the speaker and listener’s brains exhibit joint, temporally coupled, response patterns. Such speaker-listener neural coupling vanishes when participants fail to communicate (such as with different languages). The temporal nature of this speaker-listener coupling suggests that an ability to evoke similar brain patterns in another individual via speech may gate our communication abilities. Moreover, while in most areas the listeners’ brain responses mirror the speaker’s responses with a delay, some areas in the listeners’ brain exhibit predictive anticipatory responses. Finally, we found that the extent of the anticipatory neuronal coupling between interlocutors is predictive of communicative success.
Here's the idea as it applies to video games. By reading the actions of a player as a type of virtual body language, one may be able to share an experience or share a mental state with another to a very high degree of synchronicity. This theory explains why yomi, the term for predicting an opponent's moves, was coined from the Japanese word for reading. Playing a video game intensely like in fighting game competitions sets the stage for both players to be completely focused on the game. The more focused players are the more of their will and minds they extend into the gameplay. The greater this extension, the more likely it is for a player to read an opponent's mind through the virtual body language. Of course, any externalization of our minds whether it be video games or language will never translate our mental states 100%. However, getting close is still a significant achievement. This is why it's not surprising to me that I can tell what my opponents are thinking when I compete in Smash Brothers. At times I can tell what they're focused on, what they're thinking, and anticipate their mental hiccups occur. Of course, getting to this level requires learning the language of Smash Brothers. But I'll venture to say that every activity holds potential to develop into a language of its own.
If your game design goals are to create games where players can share brain states, then you should privilege
shared screen and
clean design. If a picture says a thousand words and the majority of video game feedback is through visuals (because around 80% of human sensory impression is visual), then it should be clear why screen and feedback design are so important for our purposes. Language, as with any external or intermediary tool, works best between two people seeking to communicate when both parties use the same language and experience the same stimuli through their sense. If one party doesn't understand the words you use or if your words must be translated, the external bridge that connects your minds becomes less and less effective.
Immersion and focus are important for sharing brain states. Getting into "the zone" whether reading, playing sports, music, or video games is a common phenomenon. Read more about the flow zone as it applies to game difficulty here. You should understand this from first hand experience. If you've ever tried to carry on a conversation while playing a real-time action game (or while doing anything engaging and challenging) you know how much focus gameplay takes. The language centers of your brain can be blocked truncating all of your sentences and stunting your vocabulary. Your sense of direction can warped. And your logic impaired. No matter what we do, our minds are limited. So, to boost our cognitive abilities we've developed a range of techniques.
Restricting these actions in turn restricts cognitive ability which is reflected in test results. The truth is our minds and our ability to do complex functions would be pretty poor if it weren't for two things: our ability to process and coordinate different types of stimuli simultaneously and our ability to extend our minds using symbols, language, and other tools. The former deals with mental channels while the latter extended minds.
It's all coming up in parts 4 and 5. Calm your mind and stay with me.
Saturday, September 17, 2011 at 12:19PM 
Reader Comments (2)
Don't forget that some sort of sensory overload can be positive! I read about one of the people who got a world record in minesweeper and that zie got it while talking on the phone, surprised hirself over the result (having focused on talking and not playing).
@ Ava Avane Dawn
That scenario is quite complex. Yes, it is possible for your own thoughts to get in the way of your actions. Being conflicted and consciously overwhelmed can do this. So, some people have techniques where they occupy some parts of their minds so they don't get in their own way. That's what this minesweeper case sounds like to me, at least in part.
Good point though.