Controller Design: Touch Screen pt.3
Friday, August 12, 2011 at 9:24PM It may seem odd now to use a handheld gaming device that doesn't have a touch screen. The 3DS, Playstation Vita, the iPhone/iPad, and all droid phones now feature touch displays. Yet when Nintendo first announced the DS with its odd dual-screen-single-touch design the gaming industry had trouble seeing the point. It's been over six years since the launch of the DS and the legitimacy of the touch screen has been proven.
Reach Out And Touch
The touch screen is an analog 2D input and display combination where the visuals on the screen can have a direct, one-to-one relationship with the touch sensitive areas. As long as you have screen to touch you have potential interactivity. Before I go into detail, there are three types of touch controls that I want to focus on; touch pads, touch screens, and multi-touch screens.
Touch pads are touch sensitive areas that have no display. The degree of sensitivity varies. Sometimes styli are used. Other times, just fingers. The Guitar Hero 4 guitar controller features a touch pad. With it players could play by pressing on the sections like buttons or by sliding their fingers up and down. Most laptops feature a touch pad to control the pointer. And the Playstation Vita will innovate by featuring a rear touch pad that is the same size as the screen display on front.
Touch screens are displays that are sensitive to touch. In this case the screens are pressure sensitive so a stylus or a finger will work. However, the resistive screen technology is limited to accurately tracking one touch input at a time. The Nintendo DS, 3DS, and upcoming Wii U all feature touch screen with stylus control. Using the pencil like a stylus draws on one's fine motor skills from writing or drawing. With pixel fidelity stylus control uses the muscles of multiple fingers, your wrist, and your arm. Such controls would be quite a dexterity barrier if we weren't already trained. There is, without a doubt, no input device that gives players a higher level of 2D analog control and precision. One downside to stylus control is how the player's hand and stylus can obscure the view of the screen.
Popularized by the success of the iPhone, capacitive touch screens are commonly referred to as multi-touch screens. Capacitive touch screens react to anything that conducts electricity. So a plastic stylus doesn't work on these screens. Instead, special capacitive styli can be used though they are not widespread. With the norm being finger tip interactions a few issues emerge. A finger tends to obscure much more of the screen than a stylus. The finger has less fine control than a stylus. And it's more difficult to tell where exactly a finger touches the screen because of how large and soft it is. Though the multi-touch technology allows for more complex actions like pinch zoom, more fingers on the screen obscures even more of the view.
Regardless of the type of touch screen, the biggest limitation developers face is dealing with the lack of tactile feedback. Though one can feel when a stylus or a finger makes contact with the touch screen, when interacting with virtual elements there is no tactile differentiation if you miss. It all feels the same. Fortunately, good audio and visual feedback can help make up for the lacking tactile feedback. When interacting with or manipulating a touch screen element the player tends to focus on the visual object. After this point if the object moves or the context changes the player can react accordingly. It's when players are encouraged or required to look at a location other than where they are touching that the chances of missing the target increase. With traditional gaming controllers players quickly learn to feel their away around the controller so they can focus on the game screen. The more complex the game and inputs (like multi-touch) the more players will have to focus on inputting. Yet, if the virtual buttons are in the same place relative to the screen, muscle memory can help.
Regardless of input device limitations we can always design smarter. Design (the software part) is the most flexible part of game design. A few years ago I wrote a post on the great design features for the DS that utilize the touch screen. I suggest taking a look here. The following are a few examples I want to highlight.
- Kirby Canvas Curse. While Yoshi Touch & Go introduced all-touch-screen, indirect-control design, Kirby Canvas Curse perfected it (and at the beginning of the DS's life too). You can tap Kirby, obstacles, switches, and enemies to help you progress. To effectively move around players must draw rainbow pathways that have a built in treadmill effect. Depending on the curve of your lines you can make Kirby bounce off, crawl vertically, speed boost, or even travel upside down. This kind of gameplay is only possible with the fidelity, accuracy, and directness of touch screen controls.
- Zelda Phantom Hourglass and Spirit Tracks.These games (both on my GOTY lists) can be played entirely with the touch screen. Just with contextual touching players can read a sign, move Link, or attack an enemy. With a tap of the virtual button, Link can change weapons and prepare to use them. While at the ready, Link can aim but not move (stop and pop makes for a cleaner game). This design simplifies the gameplay, allows players to plant Link at a spot to focus on other elements, and keeps the controls from overlapping. Such a design is a perfect way to work around the limitations of touch. These controls held up well through the scrutiny of multiplayer competition too (read about the battle mode here)!
- Rhythm Heaven. With this near flawless music-rhythm game players hold their DS book style. To avoid the lack of tactile feedback issue for virtual buttons, the developers made the entire touch screen a single button. Not only does this design make inputting hard to mess up, but it also utilizes the edges of the screen as a tactile barrier. With this game you can feel the edges of the touch screen to help you stay on target even when you're not looking. The visual feedback on the touch screen also clear indication where you tap, where you hold, and in what direction you flick.
- Metroid Prime Hunters & Geometry Wars: Galaxies. These games offer a variety of control schemes. If you prefer the direct interactivity of touching exactly where and what you want to shoot, you can set the game view on the touch screen and tap away. If you'd rather not obscure your view, you can move the game view to the top screen and use the touch screen like a virtual analog stick or mouse. Personally, I prefer the latter option. In fact, I played Metroid competitively online using the thumb stylus. This small piece of plastic was fashioned over your thumb so you could slide it around the touch screen much in the same way you would an analog stick.
- Ninja Gaiden: Dragon Sword. Relativistic controls can become difficult to manage in games with frequently changing camera angles. This game solves the problem with by focusing on direct touch screen controls. It doesn't matter if you spot an enemy deep into the background, to the side of you, or closer to the camera. Just tap on them and Ryu Hayabusa will throw a shuriken at the target. To move Ryu through side scrolling, isometric, or overhead environments, just tap and hold where you want him to go and he'll path find there. Attack combos are also context senstive to the enemy, Ryu's position, and how the player swipes with the stylus.
- Eliss. Multi-touch can become quite complicated. Technically, like Street Fighter commands, we can design multi-touch commands to require specific multi-finger motions and timings. Generally, this is not a good idea. Even a pianist/violinist/artist like myself doesn't have too much simultaneous finger dexterity (using one hand). Keeping it simple and intuitive works best so far and Eliss does just that. Simply use your fingers to pull part or push together the colored circles to decrease or increase their size respectively (see video below).
Triggers Down
The trigger is basically an analog button. Or perhaps it's like a button and a lever in one. Still one dimensional, a trigger has a much wider range than just on and off. To clarify, a trigger must have a springy push back (the button like quality) and a wide range of sensitivity. While most console controllers have triggers the Wii Nunchuck and Wii U controller do not. More examples include the Wii Balance Board, which is made up of 4 triggers like a D-pad for your feet. The Guitar Hero/Rock Band drum pedals are also triggers. Similar examples that can't quite be categorized as triggers are the DJ Hero controller turn table, the Arcanoid paddel, some racing wheels, and the mouse wheel.
The hardest part about designing mechanics that utilize the analog trigger is coming up with fitting ideas. As I've explained, buttons are great for turning things on and off (binary 1D). And because countless mechanical and electronic devices are activated by buttons, in video games buttons cover a wide array of creative possibilities. Analog sticks are great for controlling actions like motion especially in 2D. Simply point in the direction you want to move and the game actions generally follow suit. But how many things in real life operate by a pedal (analog 1D)? The easy answer and possibly the only answer are motor vehicles. The bottom line is that a trigger is a specialized input device best used for controlling variable continuous actions that deactivate once you release the pressure. There aren't many functions and devices that need this.
After racking my brain and reading through a few NeoGaf threads, I couldn't come up with many mechanics that work best (or even work well) with an analog trigger outside of racing/acceleration mechanics. Here are the examples I found.
- Racing games use it for acceleration and break pedals (Forza. GT. GTA. Burnout. etc.).
- Grand Theft Auto 4. Apparently you can switch between free aiming and lock on based on how far you press and hold the trigger.
- Super Mario Sunshine, Luigi's Mansion, and Super Smash Brothers Melee each used the trigger to control the strength of the F.L.U.D.D. (water gun), Poltergeist 3000 (vacuum), and shields respectively.
- F-Zero GX: Players can side strafe at different speeds using the analog triggers.
- In Halo 3, the Spartan Laser requires a charge before firing. Players can control the length of charge up time with the analog trigger.
I believe most developers simply design trigger mechanics as if triggers are buttons. With such mechanics, developers have to be mindful of the "deadzone" design like analog stick mechanics. For me, the actions mapped to the triggers in Street Fighter 4 for the PS3 are too sensitive. And for Brawl, because light shielding was removed, players have to press all the way to the button at the bottom of the triggers to activate a shield or air dodge. This means when I hit the trigger like a button, my timing gets off. Adjusting to mismatches like these between software and hardware design takes a bit of time. After all, the mismatch exist because of a clash between long established learned and intuited behavior.
In part 4 we'll cover pointing devices from mice to Wiimotes. We'll also tackle microphones as inpute devices.
Reader Comments (4)
Well done, I think you hit the nail on the head regarding tactile feedback on touch screens!
@CrowbarSka
Thanks. Here comes part 5.
By merging touch-screen gaming with a video game system that is designed to live next to your TV set rather than be carried around in your pocket or purse, Nintendo is not merely acceding to the cultural tide. It is also trying valiantly, perhaps quixotically to stem it. After creating a world in which we are no longer bowling alone. Nintendo is seeking to invent display controllers way for us to commune with our screens. The company’s hope is that the Wii U will bring families together in their living rooms for touch-screen gaming rather than leave them isolated with their tablets and smartphones.
@Amethyst
Ummm. Sure... but that's kinda a dramatic way of putting it.