Be a Human (aka, "Outlandish Beta Suggestions")

Encarta would be really cool if came with a talking Robot toy to explain content!

There are a lot of strange ideas that come out of testers during Betas. Some ideas are very high caliber. They make us sit up and say “Why didn’t we think of that ourselves three years ago?” Other ideas aren’t so great. A Robot addition to Encarta wouldn’t make a whole lot of sense- and anyhow, Nintendo tried this years ago with their R.O.B.. It’s a cool folklore story, but I’m sure Nintendo wouldn’t have called it a success.

From a Tester’s perspective, we do see some suggestions that clearly aren’t practical. This isn’t any fault of the Beta Testers- it’s their job to submit bugs & suggestions, not to censor their ideas based upon practicality. When an impractical suggestion comes through the pipeline, there are really two ways to resolve the issue.

1. Take the quick and dirty route. Resolve the bug “Won’t Fix”, and move on.

2. Resolve “Won’t Fix”, but take a few moments to comment to the Beta user.

Depending on impossible time constraints, sometimes option 1 is all that is possible. This saves time for the STE, but it leaves the Beta Tester with no worthwhile feedback. Should they keep submitting similar suggestions, or should they devote their efforts in a different manner? The question goes unanswered, and they really won’t have any ideas on how to improve their bug submissions.

Personally, I’ve always tried to make time for Option 2. Writing a quick sentence to the Beta tester only takes an extra 15-30 seconds of time, and it can help motivate good beta testers to become great. I’d probably reply to the above suggestion with something like this: “Thanks for the suggestion, but a Robot Toy would add too much to the price of Encarta for most of our users to be worthwhile. Cheers, Gregor”

It’s a little over the top to say “Thanks!” on every single bug, but it really adds a human aspect to the people on the hidden side of the bug reports. This makes the Beta tester feel better represented, and they know that their ideas are being read, even if they’re not all passed on. I’ve heard recently that the Encarta Beta program is one of the most popular at Microsoft. Having a cool product is a big help with Beta satisfaction, but being personable when communicating with Beta testers goes a long way as well.

Greg

Comments

• Anonymous
  May 27, 2004
  Right on! Getting those small comments really helps me understand what the product team is looking for. In the past, there've been betas that were really opaque, and its hard to know what the testers should be doing.
• Anonymous
  May 27, 2004
  Oh come on, you mean you can't dig one of those old Actimates Barney dolls out of a closet somewhere and jury-rig it up for the purpose?
  
  
  No, wait. Forget I said that. The term "Scary beyond all reason" comes to mind...