Oh, I’m stepping in it now!
Recently a discussion came up about testing miniatures games. I thought I would give you guys some insights about how that actually goes. I’ll talk a little bit about my experience with Dust Warfare testing, but mostly this goes for any miniatures game (or any game as complex as a miniatures game). This was going to be a two part article, but now it’s 4! First I’m going to talk about the types of feedback you can get from testing. In the next articles, I’ll talk about what to look for during a play test, and some common balance decisions that many players might mistake for poor design, but which are actually quite healthy for a game. Don’t worry, I’ll also explain why!
There is also going to be a fair amount of jargon coming at you guys. I apologize in advance. So lets start off with some of the things that make testing really super hard.
Seriously, this can happen in video game testing. If ONLY I could get that sheer volume of feedback with computer tracking! I can’t even begin to tell you how much that would solve everything.
Game Logistics
A computer game “pvp match” usually takes anywhere from ten to fifteen minutes. It involves 4 or more players, and the computer can track all kinds of information that real humans can’t watch in real time, like most common death positions. It’s also deceptively a more controlled environment than a miniatures war game, with set maps, ect.
By contrast, a miniatures game takes 2 players at least an hour to play. The board changes drastically (and randomly) and it is extremely difficult to monitor all the things going on in a game. How often did a roll come up more than one standard deviation above the norm? Is the unit too efficient at it’s point cost, or did the player just roll well? These are all questions that can only be answered with extensive notes… which increases a games play time to well over three hours.
So where a video game can test a bunch of games simultaneously over the internet, or even locally, a miniatures game takes three hours to test just one game. A game like Star Craft, or Halo, for instance, can push out a public beta, and have the computer run literally millions of games through a processor to find what happened the most. That can’t be done with a miniatures game… due mainly too…
Feedback Dataset Characteristics
Doesn’t that sound nice and sciencey? Basically there are four types of feedback that a designer can garner from playtests. They are each useful, very useful, but for different reasons.
How about now?…. Now?… How about now?
Experiential: This one is pretty simple… is it fun! We are designing a game people want to play. The purpose is to have fun. If a player is excited about play testing again, and had a good time, that is extremely good information to have. If the players can pinpoint things that stopped them from having fun, that’s even better. It’s important, however, to note that this feedback usually only comes from new gamers. Experienced gamers tend to focus on the other types of feedback.
Mechanical: Also simple, the question here is about how clearly the rules are written. Wording can be made more clear from this feedback, or if it’s as clear as it needs to be FAQ questions can be compiled. Any game as complex as a miniatures game is going to need an FAQ. If the designer does this perfectly, there are still going to be questions that come up a lot, just because different people will read a sentence differently, or just misread something early on. An FAQ can help get everyone on the same page. The designer also needs to be on the look out for terms that might need explanation (like clockwise to younger players).
If you don’t think RPS has a metagame… you’re crazy. Check out the national RPS leagues. There are books written on it… and they should be read by any aspiring game designer.
Balance: Now we are getting into tougher territory. This one is tied in closely with Metagame, but it has subtle differences. Balance just aims to make sure that one option isn’t head and shoulders more efficient than the rest. In a FPS (First Person Shooter) this might involve making sure every weapon has a maximum DPS (Damage Per Second) and that qualities like Splash damage are properly valued in the final DPS equation. In a miniatures game, it’s about making sure that one unit in an army list isn’t the best choice. Players will perceive best choices (based on the metagame, coming in a moment). Situation balance is critical to a miniatures game. Every player must feel like they have a tool to use to answer a variety of situations.
Metagame: This is the toughest one to test, but arguably the most important to a competitive miniatures game. It’s closely tied to balance, but it involves a pair of concepts I’m going to talk about later called Lead Position Disadvantage and Perfect Imbalance. This is possibly the hardest thing to test for in a miniatures game.
Coming up, I’ll talk a bit about how we go about getting each of these types of feedback during play testing, with the extreme time issues!
Do you think balance on the army-to-army scale is more influenced by the metagame or pricing than design when it comes to miniatures games? I understand the relevance when it comes to individual units within an army’s options being better, but what about entire armies? GKs in 40k, for many people, were or are the best codex among codices. I feel that this is largely due to a couple of items that are not about balance, which are 1) the dollar price of a GK army versus the (probably better) IG or Necron army, and 2) the perception that GKs are the best army because they are the most “flashy” or are considered the best, even though IG can do almost anything better than GKs can for less points and with a more solid bell curve.
This leads me to another semi-related question. When making dust, how much did you focus on “hoser” units to disable any particular strategy/list, assuming that strategy/list got too powerful?
Ok two fold answer. “Outside of design” factors can certainly sway someones purchasing decisions. Unfortunately, those pricing decisions are usually not made by the designer. As such, he can’t really play for them. So yes, I think they are a factor (among many factors) in the grand scheme of things. How big a factor is probably unknowable.
Secondly, we tried to balance issues a bit more broader. Sure, Snipers are an option for handling heavy armor troops, but they aren’t the only option. If a unit was a big problem it’s probably just going to lead to “list rock paper scissors” if we do to much “this unit crushes that unit” design.
Hi
So far I have enjoyed your articles and hope to read more of them in the future. I’m very interested in Lead position Disadvantage and perfect imbalance (because I’ve never heard the terms, they sound fancy, and they have to do with metagame). I haven’t read up on a lot of designers blogs or articles; but I have had to muddle through the design of, and trying to get playtesters to work on, a ccg.
Most people can’t seem to believe that the way they play the game (ie their local meta) is not the same everywhere so I was curious how you deal with that, which amounts to very biased playtest feedback a lot of times.
I’ll be covering that in future reports. Basically, you just have to take everyone’s opinion into account, but you have to believe yourself when you think something is fine. Lead Position Disadvantage basically means that when you are in front, everyone changes their strategy to beat you. So a unit that is 5% more efficient than we would like (which feels bigger to players) can really just help predict, and pull, a health metagame.
Definately looking forward to hearing more. (And perhaps you might apply some of these to your new gig with Wyrd?)
Pingback: Testing a Game, Part 2 | Dice Like Thunder