Superfans and Lead Testers

This post was inspired by this tweet, part of a really interesting thread from Emma Larkins, who rocks and is awesome and you should all follow:

The Superfan is something I try to get very early on in a game. Of course, I design games I love, but as Emma said, I want to make sure I’m not the only one who does so. I talked before about designing pitch-first, to make sure you’re designing towards something appealing. Getting a Superfan is a great way to do make sure that that pitch is attractive to someone else, but also that through your process, you’re staying true to your original vision. Plus, if you’re lucky, they’ll keep you accountable: “hey, how is game X going? looking forward to trying out the newest version!”

On each of my games, I’ve managed to get a Superfan early on, and then, I’ve turned them into what I like to call Lead Testers. If a Superfan is proof that your game can attract someone other than you, a Lead Tester is a tool to make sure it stays this way. A Lead Tester is someone who rides the line between playtester and developer, who through repeated play becomes an expert, and who through repeated discussions about the game has a deep understanding of the game.

If you’re reading this, you hopefully know the oft-repeated advice “don’t just playtest with your friends and family”. Cast a wide net, get 50, 60, 80, 100, 200 people to try it out. You need to get multiple opinions, see if the game can survive multiple different groups, and also, get some people hyped about your game, and maybe even your other designs! But another part of it is also that you need people who haven’t played it before to try and see how well a first play can go. Can people understand the rules? Are there first-level strategies? How approachable is the game? How readable? All those questions can only be answered through testing with new players, and you can’t “pretend you don’t know it”. In this world of games which are only played once, that first impression is key.

That being said, the Lead Tester will also help you with something else: expert-level play. Can the game sustain interest through repeated plays? Can a strategy be exploited? Are all of these factions viable? Sometimes, what you need is not a pair of fresh eyes, but the keen, wise gaze of a veteran, someone who’s been there before, and knows the game as well as you do, without being as close to the content as you are. Sometimes, you need someone to play the game 10 times in a row, so you can try out a few different faction ideas you’ve had. When I first added what would become the Shadow cards to With A Smile & A Gun, my buddy James ordered me to go make some more: “we both have Thursday off, let’s go for coffee and try them all out!”

Of course, a Lead Tester cannot be your only source of testing, but I would still suggest you try and get one for each of your projects. But how? It’s a very simple 4-step process:

  1. Get an awesome pitch;
  2. When somebody responds positively to the pitch, playtest with them;
  3. If they respond positively to the playtest, playtest with them again;
  4. After each test with them, push the discussion a bit deeper.

Some people won’t want to be that involved. Some people won’t need to be pushed. At this point, it’s social skills: try to feel their interest level, and the relationship you have with them; make sure not to push on them roles they don’t want. If you’re lucky, a Lead Tester will take that role with pride, without you having to ask, but most of the time, you’ll either have to gauge it as you go, or, and I would suggest doing it this way, just asking them: “Hey, I think you’re exactly the kind of gamer I’m making this game for. Would you mind being making sure the game stays true to that throughout its development?”

Have you had the chance to get a Lead Tester for one of your designs? Or have you yourself taken that role, whether officially or not, for another designer?

Getting testers to your table at public events

Last weekend, I went to a local gaming event to playtest With A Smile & A Gun, and hopefully get a few mailing list subscribers. This article is meant to give a few tips and tricks to those of you who are looking to do something like this. I’m far from an expert on approaching potential testers, and maybe you disagree with some of these: let me know in the comments in that case.

Before I go into the tips, a quick presentation of the event: I went to the Bissextile Ludique (Bissextile is French for leap year, because it was February 29th, and Ludique means fun), organized by Longueuil Ludo, which organizes weekly game nights at a community center, and bigger events a few times a year. There were about 70 people, there were two merchant booths, a flea market, raffles and tournaments, and a Prototype zone for 3 designers—actually, it was just three adjacent tables in the main gaming area. That last part is important: it means that we had more traffic than if we were in a separate room, but it also means that we often had to explain why we were showing off this crappy game that looks like it was printed on our home HP (hint: it was!). In an event this small, a separate room would most likely have meant no traffic.

I’ll also assume your game is ready to be playtested with strangers. Don’t take your first drafts to strangers: you’ll burn their goodwill, it’s harder to get them to try it, and a buddy’s feedback would have been just as helpful. By the time you go to those events, your game’s core should work, and you want to have new perspectives on it. Your game should also look presentable: it doesn’t have to be full of art, but the graphic design should be clear and understandable. It should not look like homework, and you should have an intriguing pitch to hook them in.

Tip 1: Be approachable.

Please smile. Please don’t look like people are disturbing you. Please talk to those around you. Testing or demoing for someone is a lot easier if you know them, even a little bit.

In an event this small, there were times where I was without any testers, and everyone was sitting down in a game. Then, I’d get out my phone, or go for a quick browse of the flea market. But if there were people moving around, get up and invite people over. Even if they’re fascinated by your prototype, most people won’t disturb you if you look busy.

Tip 2: Build social capital.

Even if you’re approachable, that’s still not enough. At an event like this, people will have a choice between a game they actually know and love, learning a new game they’ve been yearning to get to the table, or this prototype you brought. Some people love helping out, love trying games in development and having some input on final products, but for most people, that’s not the case. Whether they’ve never tested a prototype before, or been burned too many times, or just don’t enjoy it, you’re often fighting an uphill battle.

The best way to turn it on its head is to make a connection with people, so that when you ask them for a playtest, they want to help you. If you have down time, play a game with them. Walk around and look at other people’s games: I spent a good 20 minutes observing a game of Marvel Champions last Saturday, which was both awesome, because that game rules, but also allowed me to connect with those players. I participated in a Just One tournament (well, actually, won a Just One tournament, thank you very much) with another 6 people. Later on, when all those folks came back from dinner, I could call them by their name and invite them over to play.

Of course, respecting people’s boundaries, adapting your approach to their particularities, reading the room, and other social guidelines very much apply here.

Tip 3: Be respectful of their time.

Once you get a tester to sit down and play your game, you’re not done: word of mouth is still a thing. If a tester gets up from your table and tells others your game was a waste of time, you might as well pack up for the rest of the day. On the other hand, if someone gets up and tells their buddies to come and try it out, they’re doing your job for you—and are probably much more efficient than you are.

This is not about delivering a good game experience, but it’s being respectful and grateful for their time. Set the game up before players sit down. Have your explanation prepared and rehearsed. Listen to their feedback. But also, if a tester wants to drop out before the game’s end, that’s fine: just sit in for them. If multiples do, don’t pressure them to stay. Actually, after the first round, you should ask everyone if they want to go through the rest of the game. That’s the difference between answering “it wasn’t for me” when asked, and actively telling everyone around them “I just wasted half an hour of my life”. You won’t get good feedback from frustrated people anyway.

Also related is “Know the crowd”: look up the event. When I saw on the Facebook group the kind of games people were playing on pictures, I went with my 20-30 minute dice game, not my 90-min engine building, area control Euro game.

Tip 4: Raffles are… meh?

So I thought about the “Give before you ask” mantra, and decided I’d raffle a game amongst those who tested my prototype and left me their email for the mailing list. I thought about raffling a pledge for the game, but decided maybe a prize that you’d get a year after you won it wouldn’t be that enticing. I therefore bought a copy of Jixia Academy, the newest edition of Hanamikoji, which is another 2-player majority game, thinking if they’d like one they’d like the other. Price wise, the FLGS copy of Jixia was about the same as the landed cost + shipping of my game. Tomato-tomato (wow does that not translate well to written text), I don’t feel strongly either way. If I were closer to the KS, I’d probably go for a pledge. Maybe they’ll be interested in adding more copies, since I’d already cover shipping?

I had the box on my table, next to my display of the game’s cover image and basic instruction. People were confused, so I added a sticker on it that said “Try my prototype for a chance to win this DIFFERENT, yet AWESOME game”. Which made people a bit less confused, but still… pretty confused. The pledge would probably have been a better option on that end.

And even if that were clear, of the 19 emails I got, 19 said they would have given me their emails even if it weren’t for the raffle. And no one was attracted to the table because of the raffle. At the end, I still drew the name and gave them the prize, and they couldn’t. care. less.

I’m still happy I did it, but I probably wouldn’t do a raffle again. I might organize a play-to-win tournament, where the winner gets a copy of the game, and maybe others get a coupon. However, I’d only do so with physical copies of the game after fulfillment, or by offering a pledge during the Kickstarter (or possibly, in the weeks right before it).

Where I’d do a raffle is for a Gleam campaign, “subscribe to my social media for a chance to win”, where that chance at a prize is the only thing you’re offering. In this case, I think at best it didn’t matter, and at worst is might have even taken away from the genuine “I want to go when this comes out” value of giving your email address.

So that’s that. Have you ever gone to such an event to playtest or demo your game? How did you get people to your table?  

Sen-Foong Lim on his Roadblock

So this is the first post in what will hopefully become a weekly series of interviews with more established designers. In each episode, an established designer will come and talk about a Roadblock they’ve run into with one of their designs, how it showed up, how they identified it, and the process of fixing it. The goal is to give you specific examples of the process of designing a game, and ideas of what you can do when faced with similar situations.

Today I’m sharing with you folks a discussion with Sen-Foong Lim, designer of so many games, such as Junk Art, Belfort and Akrotiri, just to name a few, and fellow Canadian! Sen is also the first ever recipient of the James Mathe Mentorship Award!

SFL: Hey there, thanks for having me! Jay and I have designed a ton of games together and we’ve got another one coming out soon!

JV: Awesome! And this one I’m really excited about! First, can you give us a bit of background on the game you want to tell us about today?

SFL: The game that’s been on my mind (no pun intended) recently is MIND MGMT. I co-designed the game with my long-time partner in crime, Jay Cormier, and the creator of the comic book of the same name. Matt Kindt is a New York Times’ bestselling author, comic book writer, and artist. He’s a force to be reckoned with!

The comic that the game is based on tells the tale of secret agents who work under the assumption that they are saving the world. They quickly find out that their organization, MIND MGMT, may not be everything it seems to be on the surface. A group of agents go rogue and try to bring down the organization from the outside. The game tells the take of these agents trying to stop MIND MGMT from recruiting enough agents to follow through on their diabolical plans. It’s a one-vs-many hidden movement game with a system that allows the game to adapt and change over repeated plays. Did I mention that these secret agents have psychic powers?

MIND MGMT is going to Kickstarter in early March 2020. This is the first game that Jay will be self-publishing under his “Off The Page Games” imprint, so we really needed to get it right!

JV: Secret agents with psychic powers sound like an incredible game right there! Now, what was the issue you ran into, and when did you first encounter it?

SFL: The real problem was how we had tested the game – we had primarily used the same 2 groups for multiple playtests. As they got better at the game, they wanted more challenges and so we complied! 

And this was a huge problem.

We poured our hearts and souls into this game which resulted in it being overdesigned. There were so many player powers and extra bits and stuff that we had built into the game that players who had never played it before felt overwhelmed by the sheer volume of options they had. They were paralyzed by all of the options.

We actually broke Matt Leacock’s brain during one of the playtests. He was just overwhelmed by all of the possibilities in the game. To have a designer the calibre of Matt give us that feedback was really powerful. That told us that we had swung too far in one direction, and made us take a step back to reflect upon what had happened to the game as we developed it.

JV: Was there something about the game that pushed you towards testing with the same group all over again? Or just happenstance?

SFL: We just got comfortable in our groove, with our testers, and we (looking back) probably didn’t want to teach the game to new players again and again. We were also likely in the brainspace of “more is better!” without realizing the detrimental effect too much content would have on new players who were just trying to grasp the basics.

Jay and I live in different provinces and we both have playtest groups. We thought that would be enough diversity but we failed to recognize that skill creep is a thing. To go along with this, we can turn to the psychology of meaningful engagement as demonstrated by Mihaly Csikszentmihalyi’s concept of flow.  Csikszentmihalyi states that as a persons’ skill in an area improves, they require the challenge to rise accordingly or they may disengage from the task. Applying this to game design, what was happening was that our players were getting more skilled in the logic and deduction part of the game and so they needed more moving pieces to keep the game interesting for them. You can see this in many games – the constant influx of new cards in Magic: The Gathering is a great example where it is phenomenally difficult to understand ALL of the cards EVER as a new player, but experienced players clamour for more because they like the game and new cards maintain their engagement level.

JV: Is there something about the game that made it more susceptible to that overdesign? 

SFL: In this particular case, logic and deduction games require skills that you can learn and improve upon with repeated plays, especially if you play with the same group. So as the Agents learn how to use their psychic powers to improve their chances of capturing the Recruiter, the Recruiter feels like they need more tools to combat them.
We didn’t think about the product in terms of a long-term thing over many plays. We just kind of “thought vomited” all of the ideas we had into the box and that was what was overwhelming.

JV: Did that overdesign lead to anything useful? Were you able to keep some of that content?

SFL: Oh yeah, we scrapped nothing. The overdesign wasn’t in the content, it was in the content delivery. In fact, the way we changed content delivery allowed us to develop even more content because it would have less risk of being overwhelming.

JV: Can you talk about the steps that you took between figuring out the problem and finding the right way? What did you try that didn’t work?

SFL: There was a lot of gnashing of teeth and lamentation. A few tears may have been shed. After we had our pity party, we stepped back and took a good hard look at the game. The game was awesome. It was just too much all at once. We really wanted to keep everything we had spent so much time working on, but knew that we couldn’t expose players to it all at once. We thought about expansions, but didn’t like the idea of people having to buy more things to experience the game as we had fully conceived it. We wanted to give people all of the content, but not all at the same time. It had to be doled out in the right amount, each time.

The massive amounts of content went from being a design bug to being a design feature once we figured out how to deliver it all in a way that made sense.

JV: What was the right way to deliver that content then?

SFL: Rob Daviau had just shown us the power of Legacy games with Risk: Legacy and Pandemic Legacy (co-designed with the aforementioned Matt Leacock). We didn’t want to make a Legacy game, so instead we took a different tact by asking a different question:
“What if the game changed between games based on who won or lost the last game but in a non-permanent and customizable way?”

We wanted players to be able to reset the game should they be playing with a new group. We also wanted players to have some control of their choices in terms of what to add to the game the next time they played.

So we came up with the Shift System!

The Shift System is a little bit like Rob’s Legacy System in that there are things that you unlock over the course of the game. It’s a system in which the team that lost the last game gets to pick advantages for them and disadvantages for their opponent. Players track their victories and losses, so the more losses a team has, the more advantages they can give themselves / disadvantages they can give their opponent in future games against an opponent that trounces them regularly. In that way, it can help balance games for players with differences in skill due to age or ability.

JV: What advice would you have for designers to identify these overdesigns before they go too far?

SFL: In general, it’s all about testing and finding the fat.

Test your game widely. Test it with people who’ve never played it. Have a new team learn your game without any input from you except the rulebook. What you’re testing for if you think you’ve overdesigned your game is looking to see if people can’t make a good decision because there is too much to process (though, sometimes, that is intrinsic in a design – see Space Alert).

Things that need to be cut often appear to me in these forms:

– Rules that are forgotten or played incorrectly by players

– Rules that take longer to explain than they’re worth

– Rules that drive the game procedurally but are not a decision point

– Conditions that are passively triggered versus actively engaged

– Components or rules that are underutilized or have low impact.

JV: And once the overdesign is identified, how would you advise a designer to deal with the problem?

SFL: Some of this is preparation, some of this is mindset, some of this is testing

In terms of preparation, keep versions of your past games as well as documentation about your changes. The changes themselves are important – the rationale as to why you initially made the change may be more important, though. Your rationale for change will guide you in how you might tackle the problem from a different angle – the problem likely still exists, it just needs a different solution. You may also, as you alluded to above, have to roll back to a simpler version because it was, in fact, better!

In terms of mindset, the old adage of “kill your darlings” is echoed by many because it’s true – you need to be flexible enough to see your way around problems to find new solutions and sometimes that means removing something entirely. To ease the pain of that loss, remember that you can often keep things you discard for later use!  If something does not support the primary hook of the game or the overall experience… cut it.

Remember: less is often more.

In terms of testing, you not only need to go back to the drawing board, but you’ll need to test your changes again. You cannot assume that things will work. You need to put the new implementation through its paces. Often times, you’ll also find that players are naturally efficient and will make new sense out of your game that will streamline some of the processes for you. If players almost always do B before A because it makes sense to them in practice, go with it!

JV: Well thank you Sen! I do feel like overdesign and designing in closed groups are a kind of problem new designers face a lot, and it’s interesting to see that it doesn’t seem to go away with experience!

And that was the first episode of Roadblock! Hopefully you found in it something that was helpful to you and your design process. Now imagine we had a cool 90’s out-tro with weird punk music: ROADBLOCK! ROADBLOCK! ROOOOOOOOOOAD BLOCK!

My go-to questions to playtesters

So during a playtest, I’m an Observer more than a Poller. I find 95% of the feedback I get from looking at players’ engagement, listening to the questions they ask, spotting the mistakes they make. Usually, at the end of a playtest, I’ll mostly share my observations so they can either be confirmed or nuanced by the testers.

That being said, I have 5 questions I love to ask in playtests, and thought to share them with you with a quick rationale. Overall, you’ll notice a pattern: I rarely ask the question I want the answer to. I was a Research Assistant in Psyc in college, and learned quickly that people get in their own way a lot, especially when you ask them to analyze their own experience. By asking related questions, they tend to overanalyze a part I’m less interested in, and share truer reactions about what I care for.

How long did you feel like the game lasted? If their impression differs from the actual length, it gives you a great amount of insight in how engaged they were. Of course, it’s important too ask this one before people look at their phones and watches, or you lose that subjectivity–which is exactly what you’re asking for!

I also like to follow it up with “how long would you like a game like this to last?” I’m not really asking for the number I should aim for, but there are two types of comments that can come out of this: (1) The game finished a turn too early/too late to feel satisfying; or (2) This has too little depth / too much complexity for its length. Asking follow-up questions is how you can make the difference between the two, although sometimes the vocabulary used is a hint on its own: “It could have ended a round early” vs “I feel like this is a 30 min game”.

How well do you think the final scores represent your performance? This is my favorite question because of how much can come out of it: perceived balance issues, frustrations, actions that players really enjoy doing but are not incentivized. I’ve before that balance is not as important as the feeling of balance, and that is exactly what this question addresses.

I’m trying to add variability to the game: do you have any suggestions for special powers or special goals? Hint: I rarely actually am thinking about those things, but it’s the best way I’ve found to get players to talk about other stuff they’d have liked to do, or do more of, or limits they found frustrating. And sometimes, you even get an idea for a little bit of variability! It’s how, for With A Smile & A Gun, I got a lot of players saying they wanted to move the police around more, send it out of their ways and into their opponents’, and that became a core part of the game.

Can you rank these in order of power? It can be actions, strategies, special abilities, goal cards. Usually, a table will be able to come to a consensus (sort of), because of social dynamics and the impact it had in that one game. That being said, you’re keeping that info handy to compare over multiple groups: if you see a consensus across groups, then there is a problem.

What do you think should happen? This is more of a question during games, but it still is one of my favorite tools: if players run into a corner case and ask “so what happens now?”, even if I know what the official answer is, I ask them what seems intuitive to them. If they come to the right conclusion, great! If not, that’s okay… unless it happens all the time. And if you didn’t have an answer yet, it gives you (1) a proposition, and (2) some time to think about it. In that case, you know you’ll have to figure it out, and you just want to make sure it doesn’t break the rest of the test.

So these are my go-to questions, and aside from asking for confirmation or explanations, they’re almost the only ones I ask. What are your go-to’s after a playtest?