Game/Mental State is an article series that looks at how and why mental states can be interpreted from game states. In other words, we're looking into our minds by looking into the game. This concept holds immense potential for developing machines and software that can truly work with and for us. That can truly know us in a way that only the dedicated, patient, and insightful have known us previously. The very idea of being able to measure, express, and quantify parts of our mental process via software output may seem like science fiction, but as I've begun to explain in part one and two, it's possible, it's happening, and we've only begun to scratch the surface.  

 

Furthering the discussion the next three entries into this series will cover the following topics; interpreting gameplay as a type of language that players can use to align their mental states. Exploring the science behind mental channels, a sub-facet of knowledge skill. And examining the limitations of short term memory particularly when we use external tools to extend our minds.

  

  

silhouettes from Olly Moss merged with image from radiolab.com

What does it mean to truly know someone? To get what they're saying or to know how they feel? Perhaps sharing brain states gets us closest to achieving a shared experience. It seems like mankind has never found an ability to transfer emotions and experiences directly between people. So, we've always used intermediary tools like language to create a bridge. Verbal language, body language, pictorial images, and by extension playing a video game are behaviors coded with patterns designed to what's inside our heads to others. It comes at no surprise to me that you can interact and know someone purely through their interactions within a virtual system. That Game Company's Journey highlights this phenomenon by stripping away all verbal and text language options between online players, but all emergent games have potential for such self expression, identity, customisation, and therefore language. The more emergent, the more potential. 

  

  

 

 

To frame this idea, you must listen to this radio lab podcast titled "Are You My Brain Double." If you're short on time, start at 5 minutes. The key takeaway is that two independent brains light up in the same areas at the same times when the two people are synced. Focus and emotional response are important for synchronization. In this case, Lauren1, the storyteller, had a vested interest in her personal story because it carried meaning and emotional significance for her. And Lauren2's, the most synced candidate, ability to become immersed with a super dream state like focus allowed her to sync up best with Lauren1. Clearly engaging, challenging, and interactive design is very important here. 

Heres more on Lauren Silbert's research from hplussummit.com:

 

Understanding the interaction between a speaker’s brain and a listener’s brain in the context of real-world communication requires the development of new experimental paradigms. Using function Magnetic Resonance Imaging (fMRI), we measured neural signals from two brains (a speaker and a listener) during a complex everyday communication. We then built a simple, interpretable model that leverages the dynamics of fMRI and uses the speaker’s brain responses as a model for predicting the brain responses within the listener. Our model reveals that during successful communication, the speaker and listener’s brains exhibit joint, temporally coupled, response patterns. Such speaker-listener neural coupling vanishes when participants fail to communicate (such as with different languages). The temporal nature of this speaker-listener coupling suggests that an ability to evoke similar brain patterns in another individual via speech may gate our communication abilities. Moreover, while in most areas the listeners’ brain responses mirror the speaker’s responses with a delay, some areas in the listeners’ brain exhibit predictive anticipatory responses. Finally, we found that the extent of the anticipatory neuronal coupling between interlocutors is predictive of communicative success.

 

Here's the idea as it applies to video games. By reading the actions of a player as a type of virtual body language, one may be able to share an experience or share a mental state with another to a very high degree of synchronicity. This theory explains why yomi, the term for predicting an opponent's moves, was coined from the Japanese word for reading. Playing a video game intensely like in fighting game competitions sets the stage for both players to be completely focused on the game. The more focused players are the more of their will and minds they extend into the gameplay. The greater this extension, the more likely it is for a player to read an opponent's mind through the virtual body language. Of course, any externalization of our minds whether it be video games or language will never translate our mental states 100%. However, getting close is still a significant achievement. This is why it's not surprising to me that I can tell what my opponents are thinking when I compete in Smash Brothers. At times I can tell what they're focused on, what they're thinking, and anticipate their mental hiccups occur. Of course, getting to this level requires learning the language of Smash Brothers. But I'll venture to say that every activity holds potential to develop into a language of its own.

  

If your game design goals are to create games where players can share brain states, then you should privilege shared screen and clean design. If a picture says a thousand words and the majority of video game feedback is through visuals (because around 80% of human sensory impression is visual), then it should be clear why screen and feedback design are so important for our purposes. Language, as with any external or intermediary tool, works best between two people seeking to communicate when both parties use the same language and experience the same stimuli through their sense. If one party doesn't understand the words you use or if your words must be translated, the external bridge that connects your minds becomes less and less effective.

  

See website for complete article licensing information.