Comments on: Researching Video Games the UX Way

By: austingovella

austingovella — Wed, 30 Nov -0001 00:00:00 +0000

I really enjoyed how you pull the Kimono back, even going so far as to post the sample video. The second video felt a little more like marketing, but it shows a lot of the rationale, and I love the shots of the observation room.

Did you have to get permission from EA to post these? Was it difficult?

I’d love to see more write ups and sample videos from other research.

By: boltron

boltron — Wed, 30 Nov -0001 00:00:00 +0000

Thanks Austin glad you liked it. We did have to get permission from both the individual gamers and the Spore team at EA. But the Spore folks were unbelievably cool about giving us permission to talk about the details, with two big caveats:

(1) It all had to be post-launch. While the game was still in development, when all the research took place, everything was super-duper top secret.
(2) We can not ever discuss the actual findings from the study, only the techniques.

The most difficult part was planning this before we even started testing by drafting really rigorous consent forms for participants and sending them info on how we would use their recordings. Some people opted-out, which of course was totally cool. Most were like “sure that’s hella funny lulz”

By: bayareagirlgeekdinners

bayareagirlgeekdinners — Wed, 30 Nov -0001 00:00:00 +0000

I love the videos! The gamers are hella funny, LULZ. Makes me think about my own game face

By: pomade

pomade — Wed, 30 Nov -0001 00:00:00 +0000

Good stuff Nate!! Spore did a really nice job with their Spore Creator user experience – big fan!
Methods you guys used sound similar to Listening Lab – http://www.goodexperience.com/2004/12/tips-on-moderating-listening-l.php

By: unthinkingly

unthinkingly — Wed, 30 Nov -0001 00:00:00 +0000

love that guy in the first reel: “THE DESIGNING NEVER STOPS, SON.”

By: andreszap

andreszap — Wed, 30 Nov -0001 00:00:00 +0000

Nate and Tony – AWESOME STUFF! I am trying to understand your technique a little bit better. Did you have two moderators, 6 subjects and 12+ clients all going at the same time? That sounds like mayhem! How did you control the situation – at least from the observation room? For example, I’ve been in many usability tests where it was just me moderating, one participant, and one colleague hanging out with three clients at the same time – and that was tough to control .. granted we have chatty clients, but that’s good… that means they are paying attention and they are engaged with what is being discovered. How did you guys do it? How did you know which subject to focus on at any given point? Write a part II on this! Thanks for sharing.

By: dan_soltzberg

dan_soltzberg — Wed, 30 Nov -0001 00:00:00 +0000

Nate and Tony, your writeup really illustrates the importance of being flexible and responsive with research methods. Thanks for sharing the story.

By: tonytula

tonytula — Wed, 30 Nov -0001 00:00:00 +0000

Hi Andres— Yeah, you’re absolutely right about the mayhem:

We had two moderators dividing up the six participants between each other; otherwise you run the risk of asking redundant questions.

We had the observers chat with us over IM whenever possible, or had them pass questions to us during the (very brief) lulls in testing; we also had a chat room in which the moderators and support staff could all coordinate with one another.

As for who to focus on, that was largely driven by observing what each player was doing. If they were performing a task the developers were interested in, we switched the focus to them; we also encouraged players to thinkaloud while playing, not necessarily anticipating a response. If there was a technical glitch or request, we just had the participants wave at the camera to catch our attention.

By: demetrius

demetrius — Wed, 30 Nov -0001 00:00:00 +0000

Good article. I’d been wondering for quite some time how research was performed on games considering the fun factor. It sets a great precedent for future research in games, especially how successful Spore has been. It was also great to hear that EA was willing to commit to such a costly research commitment.

By: copywryter

copywryter — Wed, 30 Nov -0001 00:00:00 +0000

Uhhh, too bad the game was almost universallly panned…and not just because of the Orwellian DRM, but because it was ridiculously oversimplified.

http://www.amazon.com/Spore-Pc/dp/B000FKBCX4

What does this say about the usability testing?