Homework suggestion: 'Hallway Tests'

[Z-Man]

We've had several complaints about things that are supposedly wrong with ArmagetronAd here in its friendliness to new users. The problem: those complaints come from people who stuck with it anyway, and not by the people who gave up because of it. For all we know, there may not be a problem at all, and before we randomly change stuff around, we should gather some data. So I'd suggest we have a look at what is called a 'Hallway Test' in the industry. It typically works in large corporate structures and is quite simple: you go out into the hallway and grab the nearest passerby who hasn't got anything to do with your game. You put him in front of your game in its freshly installed state and yell 'PLAY!'. Then you sit back and take notes, watching how your victim struggles with bad controls, bad instructions and bad game design, but not helping him in any way (unless maybe he gets stuck completely and explicitly asks for help).

So yeah. Don't start the kidnapping just yet Our resources in unexposed victims is finite. First, let's determine what would be useful to test. My suggestion would be:
- Grab a random person you know who hasn't been exposed to Arma yet. (S)He should be roughly the target audience, that is, probably not your grandma.
- Put them in front of a computer without arma installed. Mac, Windows or Linux PC, doesn't matter. It doesn't need to be the victim's computer, any system they're supposed to be familiar with would do.
- Point them to www.armagetronad.net.
- Say 'This is the website of a multiplayer game. Download, install and play. Pretend I'm not here.'
- Take notes what the user is doing, where they get frustrated or confused, what goes smoothly, what they try, where they gives up.
(Ideally, you film it. Both the screen and the user's face. Lacking two cameras, one camera and a mirror should do it.)
- When they ask you what they should to, make your answers as short and general as possible to get good coverage of an average players' first steps:
"Download the game."
"Install the game."
"Start the game."
"Play."
"Play against other humans."
"Don't die."
Take notes about the instructions you're giving, too.
- Make sure they don't feel pressured to do things quickly. Let them read any documentation they find. If they don't find the documentation they're looking for, note that.

The best time to start would be when 0.2.8.3 is out. That's basically our reference frame for the immediate future. It should also be after the layout changes to our websites (they're less about usability and more about giving stuff a consistent look).