Founder of the Serious Games At Work website, Tom Grant will be hosting an Innovation Games® workshop for Customer Understanding on May 29-30 in Washington DC. These games (originally outlined in Luke’s groundbreaking book, Innovation Games: Creating Breakthrough Products through Collaborative Play) enable you to work directly with your customers, eliciting unique insight into what they truly want from your product or service. Tom Grant has been using serious games for over two decades, in education, software innovation, and military affairs. He currently works as a senior consultant for the Cutter Consortium, and previously as a senior analyst at Forrester Research.
This two-day certification course will teach you how to use a variety of games with your customers to:
Research studies back up years of anecdotal evidence. Games really are a valid method for doing work.
If you’ve used Innovation Games® or Knowsy®, then you know our game platforms, well, just work. Over the past decade our customers have used Innovation Games and Knowsy to answer questions, solve problems, unearth serious insight and foresight, align their organizations, and a whole host of related work. We have years of anecdotal and experiential data, and there’s no question that serious games are becoming more common solutions in the business world. However, we feel it’s still critical for us to assess the effectiveness of games for solving problems. After all, we want to know if our gaming platforms are producing as high-impact results as other techniques–or if they are even better.
Fortunately, the preliminary research that I’m sharing confirms our years of practical experience: Our games are good. Really good.
Practical Experience Drives Research Design Parameters
For a number of years, we’ve been collecting the feedback from our customers on the business impact of our games. They’ve told us that the games generate a number of hard and soft benefits:
They improve the novelty of new product concepts. Let’s define “novelty” as an idea that your team or company had not yet identified or considered. Customers report that using our games creates more novel ideas.
Increase the number of novel ideas. Getting one novel idea is great. Getting ten is better. We’ve produced games that have generated hundreds of novel ideas.
Strengthen Intellectual Property portfolios. You don’t have to bring a new product to market to get value from a novel idea: Many organizations use the results of games to stay two moves ahead of their competition.
Reduce time to take decisions. While pundits tell us that we need to “move faster” in business, they often fail to give us better tools. Our prioritization games are especially effective at helping businesses move faster: Cisco, VeriSign, HP and others have told us that Buy a Feature alone has saved them months of time.
Increase engagement. Novel ideas and efficient decisions are enhanced when employees are actively engaged in their work. As you’ll see later in this post, one of the reasons Innovation Games® produces the previous benefits is that the games increase engagement.
Enhance strategic relationships. Executives and Strategic Account Managers know that strong personal relationships are the foundation of strong business relationships. Playing games like Knowsy® creates these foundations.
Strengthen corporate brands. More broadly, companies that demonstrate they’re understanding their customers and using this understanding to drive offerings create the strongest, most effective brands.
While this is an impressive list of benefits, it is by no means exhaustive. Quite often the highest impact result of a game is its ability to directly solve a specific problem. For example, reducing the time it takes to prioritize product features often pales in comparison to the hundreds of thousands to millions of dollars of direct savings from avoiding unnecessary or unwanted products or product features.
My experience in business suggests that for senior executives these benefits are typically sufficiently compelling to start leveraging the games. My academic training, though, motivates a desire for deeper explorations: To what degree and in what situations are the games better than traditional techniques? To what degree and in what situations are online games more effective than in-preson games? What kinds of players and facilitators produce the best results? And while we have more questions than answers, the answers we’ve got are pretty darn exciting.
Measuring Novelty and Feasibility
The benefits listed above provide a good starting point for research design. The first study I wish to share is from Hadi Ghanbari from the University of Oulu in Finland, who compared the online versions of Prune the Product Tree‘s effectiveness at generating novel, or previously unknown requirements, again traditional requirements gathering techniques and Buy a Feature‘s effectiveness at identifying the most important, most feasible requirements.
Hadi found that Prune the Product Tree was significantly more effective at identifying previously unknown requirements. Perhaps more importantly, the identified requirements were more clearly understood by the stakeholders precisely because the collaborative structure of the game enabled participants to share information clearly.
Hadi also found that Buy a Feature was also significantly more effective at prioritizing requirements, and that the requirements selected through the game were judged to be more feasible, because the game structure generates prioritization data, conditions of acceptance that shape the requirements, and deeper understanding of the motivations for the requirements which creates greater clarity on the problems these features are designed to solve.
In reviewing these results, I found that Hadi was testing a relatively small sample size compared to what we see in corporate implementations of our platforms. This suggests that the advantages that Hadi identified to our online games may be magnified as the number of features and players increase.
Unfortunately the paper is not yet cleared for publication, we will post it as soon as it is available!
I’d like to see this research extended to see if we could identify more fine-grained aspects or dimensions of “novelty” and which of the visual collaboration games are optimal for what aspect of novelty we’re trying to identify.
Our second research study comes from Karlsruhe Institute of Technology, which worked with Daimler Financial Services to explore the effectiveness of using Buy a Feature in prioritizing the ideas that employees submitted to an internal “idea catcher”. Historically, these systems excel at capturing “spur of the moment” thinking, but are typically weak on prioritization. After all, if all you can do is give a “thumbs up” or a “thumbs down” on an idea, you’re not going to be engaged in trying to select the best idea possible.
While the full results of the study have not yet been released, Daimler has approved sharing some key insights. These include the following:
The Daimler team found that preparing the ideas for inclusion in the game produced a much better result, because items in a Buy a Feature game must fairly state benefits. By “fairly”, I mean that a project with outlandish claims of benefits (for example, 1000% ROI) won’t be purchased, and projects with too few benefits won’t be purchased. Playing Buy a Feature results in more fairly defined projects.
Employees reported significantly higher levels of engagement, when prioritizing ideas using Buy a Feature.
For the reasons previously mentioned, the Daimler team also found that the selected projects were more feasible, and that the chat logs provided significant insight that made the proposals even better.
Like Hadi’s study, the Daimler research was based on a relatively small sample size. Increasing either the number of employees engaged in the study or the number of projects would likely show even greater impact.
Making Your Move
For those of you who have already experienced the incredible power that comes from playing our games, I’m sure the results from these studies are no surprise, and will only confirm what you know to be true. However, you may find that the results may sway others who are still skeptical about the role serious games can play.
If you’re new to our games, or perhaps still on the fence about whether games are really a valid method for solving business problems, I hope these studies provide you with a reason to make the move toward using serious games for solving business problems.
Finally, ff you’re a researcher who’d like to join us in assessing the effectiveness of our games, drop me a line. We’re eager to support you in your efforts to explore the effectiveness of our games.
The best part of putting together this newsletter is seeing howInnovation Games® are being adapted and put to work around the globe, in every industry imaginable. This month we feature a post froom Portland-based applied anthropologist Amy L. Santee. Amy adapted 20/20 Vision for use with wooden blocks to get feedback on her clients ideal customer experience.
Not just another boring focus group.
Focus group participants, Amy writes, “really appreciate it when they don’t have to sit […] for two hours in one, long drawn-out conversation.” Instead, Amy includes at least one or two hands-on activities to get people talking, thinking and using their imaginations.
Called 20/20 Vision, this game is a way to gain insight into customer priorities, Amy writes as she describes how she used the technique with a current client.
20/20 Vision Blocks
“First, I thought it would be better to involve the participants by having them do the actual “arranging” of the list of attributes, rather than having me doing it for them with index cards.
[Second], why use paper index cards? Why not use something more interesting and tactile that adds to the hands-on, collaborative feel? I couldn’t think of anything that would work better for this very purpose than big sturdy blocks of some sort.”
How did it work?
Amy writes that the 20/20 Vision games as blocks, “was a useful catalyst for revealing how people define the ideal customer experience. All of the talking, sharing of opinions, debating, and collaborating provided insights unattainable by more traditional market research approaches.”
Collaborative play for problem solving has hit the mainstream! Businessweek recently profiled our work with the city of San José, CA, highlighting how we and the Every Voice Engaged Foundation have been working with local governments and nonprofits to apply Innovation Games to the difficult problems many communities are facing.
The inspiration for the 2011 and 2012 Budget Games for San José, CA, emerged from our work with organizations like Cisco, Qualcomm, Yahoo!, Adobe and others. Tackling complex problems in prioritization, strategy, new product development is all stock and trade for us, and many of those techniques are applicable for communities as well.
“One Saturday morning last year, about 90 leaders of neighborhood associations in San José gathered in small groups to play a game. Each person had a roll of fake money, from which he or she could pay for city services—like beat cops or libraries. Each group lacked enough money to cover the city’s budget. “We intentionally, just like reality, gave them far less money to buy the things they wanted,” says Kip Harkness, San José’s senior project manager.
By morning’s end, all the groups had agreed to run the city’s fire trucks with one less fireman each to save money. City council members adopted that change in San José’s actual budget last summer. At the same meetup this year, residents agreed to eliminate paid overtime for city managers, and six of 10 groups were willing to raise their sales tax by 0.25 percentage points, which the city is now considering. “I really haven’t had anyone tell me this is a waste of time,” says Harkness. “That’s pretty incredible when you’re talking about budgets.”
Blue Cross Blue Shield Puts Innovation Games® to Work
Leon Sabarsky’s Scrum team has been using Innovation Games for a while in their work on Claim Automation.The Raleigh, NC-based team at Blue Cross Blue Shield has used Speed Boat for project retrospectives and Prune the Product Tree for backlog prioritization—both common ways agile teams put Innovation Games to work. But recently he introduced the games to the market research team at Blue Cross Blue Shield—with interesting results.
How did you discover Innovation Games?
Word of mouth. I heard about the games from colleagues and though sessions at conferences I attended. I started trying out the games at work, and eventually took the two-day class taught by [IGQI and Enthiosys President] Jason Tanner.
Can you tell me more about how are you using Innovation Games at Blue Cross Blue Shield?
I’m the manager of a Scrum team in Durham, NC. We work on claims automation, and I also serve on the Innovation Committee here at Blue Cross Blue Shield.
We use Innovation Games in our Scrum practice—Speed Boat for retrospectives and Prune the Product Tree for backlog prioritization—but have started using them in other ways as well, such as in our market research work on our insurance products.
How did the market research work come about?
It was Jason Tanner who first suggested we use Innovation Games for market research, instead of just internally with our Scrum team. I bought the book [Innovation Games for Understanding] and invited the market research lead to lunch. After some discussion, we decided to try it.
What was the project?
The target market was college-age consumers, and we wanted to determine what type of insurance products and benefits they would be interested in purchasing—and how much they would be willing to pay. In essence, we wanted them to produce a list of benefits and prioritize which ones were most valuable to them.
How did you structure the event to get those results?
We recruited 20 of our summer interns as the subjects for the market research project, and decided to do two phases in July 2012.
First, to get the benefits, we had the interns build Product Boxes outlining the insurance products they would most be interested in. This was face-to-face, of course. And each intern presented his or her box and then the group voted on the best one.
Second, we used Buy a Feature Online to prioritize the benefits that the interns had developed through their Product Boxes. The interns logged into the game from their different offices on the Blue Cross Blue Shield campus. We had priced the benefits and gave the interns 40% of the total budget to spend on the insurance products they most wanted. We were surprised and pleased with the quality of chat and negotiations during the game. At the end we got a prioritized list of insurance products that college age consumers would want to buy.
What did you learn from the games?
Price was an issue for many of them. They are buying insurance for the first time and wanted it to be reasonably priced.
Yes, we had some unexpected results. The market research team had done a series of focus groups on the same topic and they had got different results than our project using Innovation Games.
Also, during the Buy a Feature game I facilitated, the game didn’t stop when the interns had spent all their money. The chat and negotiation continued, and the interns decided to un-purchase an insurance product to buy one concerning healthy benefits. It took 5 minutes of chatting to decide, after I had thought the game was done.
I was impressed with the depth of thinking during the game. The interns were really serious; we thought half would not be engaged, since the game was online and they weren’t together, but they surprised us. The game results went beyond what we expected.
Are you planning on using Innovation Games again?
We have a “FedEx Day” coming up in October. We’re inspired by Daniel Pink’s Drive, and want to create and deliver something overnight. Basically, you drop whatever you’re doing for a day and present a product at the end.
I truly believe there’s a different dynamic when you get people moving. You get much more robust idea generation with activities like Innovation Games, than sitting down around a conference table. There’s just something about Innovation Games, the moving around and collaborating, that you don’t get from focus groups.
Our very own IGQI (that’s short for Innovation Games Qualified Instructor) Ant Clay presented at SHARE 2012 in Atlanta this past week. His presentation was on how to use Innovation Games in gathering SharePoint requirements and was a rousing success. Check out some of the pictures below and make sure to read Ant’s blog post.
One of our Innovation Games® Qualified Instructors Michael Sahota is presenting at Agile Games 2012. He will be discussing how to bring Games to Work, featuring Innovation Games® among others. Check out his presentation here: A Guided Tour to Games at Work.
Monica Zinchiak of Z. Research Services recently wrote to tell us about how she used Prune the Product Tree, while facilitating a series of focus groups for a client. The client was developing a new product resources website focused on the academic market; they had the concept and features, but needed input from their target audience to know which features to implement right away, which to postpone and which ones to abandon. She writes, “I just used my second collaborative game, Prune the Product Tree from Innovation Games® and it was a huge success. Slight modification of the game gave me the stand-out features, along with input on the not-so-appealing features, for a client’s in-development website.”
When I read “modification”, I knew I had to find out more, and Monica graciously agreed to tell us about the project.
Tell us about how you used Prune the Product Tree?
My client is building a new website aimed at teachers. It’s a new product for them, and they were really looking for a way to
understand what features they needed to start to develop and which ones they didn’t need to spend time on. Despite having a list of 18 possible features for the website, they had no idea which ones would be most valuable. They really needed a process to prioritize them.
We decided to conduct a series of focus groups using the game, though I called it “Shaping the Tree” since we were not working with existing features. Instead, we were working with feature concepts (ideas for features and functionality).
What were your goals for the focus groups?
Our overarching objective was to learn which features would be the most valuable to middle school and high school teachers. We wanted to know how this website would stack up against the plethora of other resources that teachers currently have at their disposal.
How did you structure the games?
I facilitated six games with three players each, and preprinted the 18 proposed features on Post-It notes. I also included three blank
Post-It notes of a different color for any new ideas or features. I also added a cloud, our “Cloud of Confusion,” created from a white 8.5 x 11 sheet of paper. This acted as a parking lot for features that the teachers found confusing, or didn’t know how they would use them.
The metaphor of the tree was typical. Core features went on the trunk or stronger internal branches. Features that weren’t as critical were placed on the canopy. Features that weren’t useful or wanted were placed on the ground.
Why did you limit the number of participants to three?
I purposely kept the number of participants small so that the teachers could interact with each other as peers as they worked through the features instead of having a teacher/researcher dialogue. The small group also made it easy for me to sit on the sidelines and better observe the discussion.
What I had hoped would happen did: The groups self-organized their thoughts about the features and could talk about the pro and cons more effectively. We were also able to complete each game in just under 90 minutes, with 20-30 minutes for the game and the rest of the time for debrief.
Would you use Innovation Games again?
I would definitely use Prune the Product Tree again in the same way. The client was tickled with the results and so was I. The game worked so much better than card sorting or driving through the potential features in a Q&A manner.
We’ll discuss the business problems of attendees, pick one or two, and as a group select the right Innovation Game, playing and presenting the results. This interactive, real world, case study-based approach gives you a chance to put the games into practice immediately, learning by doing.
What business problem are you struggling with?
Unsure if you’ve really got the requirements right for your next release? Do your really know what your customers want/need?
Does your product roadmap match up with customer expectations and market conditions?
Are your project/sprint retrospectives so dull, you’re not getting the insight needed to make the next iteration a success?
It’s no secret that Product Box is a great tool for uncovering what your customers value about your products and services. The act of creating your ideal product (with an avalanche of office and school supplies) — and then selling it to your peers — triggers insights that you just wouldn’t get any other way–and it’s seriously fun. It’s also incredibly adaptable, as is seen in these videos from Adobe System’s Guta Riberio.
Guta, a senior group program manager at Adobe Systems, recently took part in a strategic planning meeting that leveraged Innovation Games®. When we asked how the teams were using Innovation Games in their work as part of our followup, Guta forwarded three videos (in English, Spanish and Portuguese; see videos at end of post) of how she put Product Box to work to get know her niece and nephews better.
“In your last email to us,” Guta writes, “you asked us to send you great stories about how we’re leveraging the games in our work. Well, I was on vacation, so I decided to try one of your games with some members of my family. My purpose was to try to get to know them a little better.”
Guta continues, “I asked Felipe (10), Henrique (10) and Sarah (6), what they thought were the most exciting features of a product. Since I live so far away from them, and have missed most of their development, it was a good way to learn about them. And the videos are hilarious!”
Guta shared the three video’s of Henrique’s dream product, the “Constru-ball”. Henrique’s videos, despite his young age, show global awareness and marketing awareness. “Not only did Henrique give his product a price in US dollars,” Guta reports, “but he also made his product available in several regions. He even knew what the diverse countries’ flags looked like! Henrique also picked a sponsor for his product, FIFA, and expanded on the secondary benefits of buying his product, the development of motor skills and exercise.”
It looks like Product Box has helped to discover a budding entrepreneur.