VOIP Wii U Solution

For the record, I think the voice chat system Microsoft has with Xbox Live on the Xbox360 is really good. Communicating online with the Xbox360 works, has loads of features, and is frequently expanded to use new technologies like the Kinect. When it comes to internet functionality, it seems like the west is generational leaps and bounds beyond the east. And I must admit, that Nintendo has done the bare minimum integrating online features into their games. However, there are still improvements to me made for everyone. Furthermore, I do not believe or want Sony and Nintendo to simply copy Microsoft. Each of these major players in the video game industry has their own guiding design philosophies and styles which permeate all aspects of their products. The following article presents a very Nintendo design solution for internet communication for the Nintendo Wii U that's a  generation beyond what is currently offered on consoles.

To set the stage, I want to cover a brief history of Nintendo's current internet features between the Wii, DS, and 3DS based on my own personal experience. 

• Mario Kart DS: Featuers a simple, free-for-all, ranked online versus mode. It was difficult to organize games with friends because you couldn't create groups or invite friends. You had to just hope that they made it into your group before the timers wound down and hope nobody else jumped in. With no means of communication, external tools were used.
• Metroid Prime Hunters DS: This game had a FFA, random matchmaking mode similar to Mario Kart DS. But it also allowed friends to form lobbies and boot people from the lobby. While in the lobby voice chat was enabled. At the time, there was a website for Prime Hunters where players could view their rank, rivals network, and other stats. It was a lite bungie.net before Halo 3. 
• Tetris DS: Streamlined random wifi battles seem to be the trend for the DS. In this game you can either select a FFA battle with items, 1v1 battle no-items, or play the new Push mode online 1v1. 
• Pokemon Diamond/Black: After exchanging friend codes, these games offer a lot of features. The hardest part is making sure that both players enter the wifi-club room at the same time. In the wifi-club room all other online players are displayed in the room like NPCS. Connecting is as simple as walking up to the a player and hitting A. From here you can select an option to battle, trade, chat, etc. While trading or battling you can voice chat. Pokemon Black has a web portal where players can asynchronously interact with other players from around the world, view stats, and share messages with friends.
• Advance Wars Days of Ruin: Because battles are turn-based, the network isn't heavily stress during online play. So playing friends online, the push-to-talk feature works well. 
• Mario Strikers Charged: Like the DS, this game only features 2 main modes for random online battles. The first is a 1v1 mode and the second is a 2v2 mode where two people play per Wii system. As is the trend, more options are available if you connect with friends. There are no communication options.  
• Super Smash Brothers Brawl. For a game with as many features, modes, and options as Brawl, the online mode is severely lacking. Aside from the relatively poor quality net coding, the random matchmaking only allowed for a 4-for-all or a team battle with a random teammate. There isn't even an option to team up with a friend on the same Wii and battle other teams. With no communication features, friend matches had to be managed via an external VOIP tool.
• Bomberman DSi. I've put many hours into this downloadable game. I've always like Bomberman multiplayer, and this was my first experience playing it competitively. The random matchmaking and the ranking system worked like Metroid Prime Hunters. It always tried to get a 4 player game, but would also work fine with 1v1s or 3-for-alls. Though players can't voice/text chat, they can cheer, jeer, and boo using the built in sound drops. You'd be surprised how expressive you can be with such a limited system. 

Clearly the Wii U needs a communication solution that maintains Nintendo's family friendliness for young players while offering everyone else an opportunity to communicate. So, I've come up with... 

 

I'm calling my design the Wii VU (pronounced Wii "view"). As the name suggests, this design focuses on visual communication. This contextual and communal "view" is only possible with the Wii U controller. Remember the days of sorting through voice messages recorded on small magnetic tapes? It was slow and quite cumbersome. Even when voice mail became digital, the way we had to navigate through it all wasn't easy. Now recall how much easier it all became when cell phones used their screens to offer visual voice mail. At a glance you get a history of all the incoming calls displaying a name, a picture when available, the phone number, and time/date the message was recorded, and the length of the message. With a tap of the screen more options are available. This leap in design has inspired Wii VU.  

The main feature of the Wii VU is the dynamic visual timeline. Using the touch screen on the Wii U controller voice conversations can be visually displayed. At a glance you can see the history of voiced conversation even with a group of people. Using Miis and speech bubbles, everything is organized onto a timeline. Keeping a history of all communication transferred, the Wii VU offers features and functionality whenever the user needs it, yet can be as automatic and hands free as any other voice chat system.

To explain how the features of the Wii VU work together as a next-generation online communication solution, let me explain my 4 part criteria.  

• Ubiquity. The Xbox 360 has a large number of its online gamers with functioning head sets. Though the PS3 can use any blue tooth headset, USB mic, or the mic in the Playstation Eye for voice chat, few players seem to chat when playing online. A great communication software solution is held back or held up by its ubiquity. 
• Clarity. When multiple people talk at the same time in person, clarity suffers. For many reasons, it's difficult to listen and talk at the same time. But when this happens over the internet where the voices of multiple people are projected from the same speaker, listeners can easily be overloaded. Audio quality is also a consideration. 
• Latency. Lag can be difficult to deal with in gameplay. But it can also plague communication. This issue is closely tied to the size and quality of the audio information being transferred. To keep the system running smoothly, audio is generally encoded at a lower quality. Though high speed connections work great, the greater the physical distance between players, the greater the latency problem. 
• Referencing. There are some things that audio communication isn't very good at. While pros develop special codes to reference specific parts of FPS levels (callouts), most gamers struggle with this type of communication. In Portal 2, co-opting players are given various demonstrative mechanics. Clearly visual communication tools would be useful. 

I've come up with a design that addresses all of these issues. In fact, I've been thinking about next-gen communication systems for a few years. What started with features to improve Skype conversations has carried over into my design for the Wii VU. Roughly arranged in order of importance, the following are Wii VU's features. 

Something like this 

Visual Conversation Timeline with Miis. This feature is key because it makes communication more effective and engaging. Using the touch screen, intuitive and direct options can be designed for the Wii VU. As you talk, you see the conversation sliding by in real-time. This is the kind of design that Nintendo has always supported and pioneered. It's the kind of design that Miyamoto would describe as being something that you understand right when you see it, and makes you want to walk up and try it.

Async shuffler with dead air sensor. Years ago I came up with a innovative design feature that dynamically breaks down and reassembles time in video games. I called it drebin.1 asychronous time (async for short). Traditionally video games have struggled with and designed around real-time gameplay. Because everything marches on in real-time more skills are stressed, clean games are harder to design, and we've faced a whole host of technical problems involving internet lag. Designing systems from the ground up using asycn potentially solves a lot of these problems. By redefining time rearranging, pausing, and bending it when necessary, all facets of a real-time system have more breathing room.

Currently, if 4 people all talk at once via voice chat, the result is a mess. This happens all the time on Xbox, Skype, and any other multi-person voice system. However, this problem never happens when text chatting. Each message in a text chat slots into a time line so you can read them one by one and even skip around if necessary. Using async the Wii VU can achieve the same effect with real-time voice conversation. Instead of simply transferring voice data and outputting it to all other parties automatically, the Wii VU records the audio and then puts it into the timeline on each party members queue. So instead of 4 people talking at once, you'll hear yourself talk, and when you're done the system will automatically play the next 3 messages one by one. The system will also be smart enough to know how far "behind time" you are and be able to catch you up if necessary. By listening to the dead air in the conversation, the system will know when there's room to speed up or slow down the timeline.

This simple ability to hold, shuffle, and reassemble layered time into a simple timeline changes everything. Even with a slower connection, the Wii VU system would be able to slot you in and catch you up without holding the whole party back. The awkward pauses due to latency would be a thing of the past.

Intuitive touch screen options. With the timeline on the Wii U controller touch screen, all the options you need are at your fingertips. If you want to only talk to an individual person in a party, simply tap, hold, and speak your message. You'll know only the intended person received the message because the visual indicators. And you'll never have to worry about interrupting the flow of someone else's conversation because the async flexible timeline will sort everything out. With either on screen buttons or touch gestures, you can have more options like mute or send friend invite. If you want a message repeated, instead of asking that person to repeat what they said, you can simply tap on the voice bubble in the timeline (before it gets pushed off screen). Again, you don't have to worry about getting behind in the conversation. Just let the intelligent async do its job.

Screen Share. As I described above, sometimes you just need to show someone what you're talking about instead of trying to explain it in words. A screen share, screenshot tool, or even taking images from the Wii U controller camera would be a great way extend the communication options. Just like the speech bubbles, the images would be displayed in the timeline. From here you can draw on the image communally much like how Pictochat works on the DS.

Contextual Wii VU finder. The Playstation 3 and the Xbox360 keep track of gamers you've recently interacted with. If you choose, you can send these gamers invites, messages, or choose other options. Likewise, the VU finder, a feature for the Wii VU, is a contextual list that you can reference at anytime on the Wii U controller screen. The lists automatically sorts based on friend availability, the history of players you've interacted with, and the current game state. So, when you're playing an online FPS with strangers and you're following someone around in game, instead of trying to get that person's attention by making a callout over the party chat, you can find that player on the top of the VU finder, tap+hold, and send a direct message. Once the player moves off your screen or out of your general area, he/she will no longer be at the top of your VU finder. The VU finder will also display who signs on or offline so you don't have to be bothered by the messages popping up on the main screen.

Mii face recognition. This technology already exists on the 3DS. Basically, your Miis in the timeline will reflect your facial expressions automatically. This is a vastly superior option to video chat for many reasons. You don't have to worry about lighting or camera position as much. Adjusting Mii expressions taxes the network connection much less. And it side steps privacy and shyness issues letting people be expressive while sitting comfortably behind a cartoony Mii face.

Speech to text. This feature is a big one, but its viability depends on the quality of the software and how much processing power it requires. The general idea is that storing even a small history of audio data will take up memory somewhere. So, at some point before the system holds too much, the audio data can be converted into text. The faster this can be done, the better. Imagine when you glance down at the timeline, the conversation can be read like an IM conversation.

This extra feature would go a long way in boosting parental controls. At the strictest allowance, no audio from strangers (non-friends) will ever be heard. Instead, their messages will be encoded in text, analyzed, and pass along as text if it is clean. The filter can censor language and other text strings it recognizes to be sensitive like addresses, credit card, and phone numbers. Attempting to fool the system with oddly spelled profanities won't work because the system won't display anything to the receiving player outside of approved words and phrases. This design would allow even kids to play and interact with strangers without fear of foul play.

Furthermore, knowing how sensitive and shy the Japanese and other cultures are with features like video chat, I believe a text based system would be more easily adopted. It's like texting, twitter, and email all in one seamless system.

 skip to 1:40

If the Wii VU sounds like something Nintendo would never do, allow me to give you a bit of hope. Nintendo's interface style from the DS, to the Wii, and now the 3DS is all about clear, simple, direct touch/pointer based interactions. The whole idea of browsing the Wii like TV channels is just the kind of idea Nintendo goes for. Looking closely at the 3DS internet features, players can see who's online and what games each other are playing by scrolling through a line up of their friends visually represented by Miis. And most importantly of all, Nintendo has already designed something like the Wii VU. Though most think of the Wii Speak microphone as odd and grossly underutilized, the accompanying Wii Speak Channel has many features similar to Wii VU. Users are represented by Miis. When you talk, sound waves radiate  from your Mii. Also, the Mii mouthes along with you when you talk (see here). You can invite others to your chat room and even share pictures.

You would think, with the Wii U Nintendo would incorporate such functionality into their core system.

Comments

[axcho]

This sounds like a great idea, especially the "async shuffler" and Mii facial recognition. But is there any chance that Nintendo could pick up on these ideas from your blog?

How might someone from Nintendo come across this blog post? And even if people think it's a good idea, don't most game companies explicitly avoid receiving or reading idea suggestions from people outside the company for annoying legal reasons?

I mean, it's great that you have these great ideas and are sharing them here, but it's kind of disappointing to think that there's no way they'll ever get used. Is there any way they could have an impact?

[Richard Terrell (KirbyKid)]

@ axcho

It is a little depressing. Even if nobody picks up the idea, I think it'll eventually do some good. Maybe people will see that I can generally think of such things. Maybe I can make some kind of working model or a video of the idea.

I never try to contact Nintendo directly. I have a hard enough time not upsetting smaller developers and indies when I sent them emails.

Man... depressing. But we have to have hope!

[VoIP providers]

I found very interesting stuff on your blog. Thanks for sharing.

[Amay]

Amazing blog I enjoy reading your blog I found very interesting stuff here with this blog