Dev Update 4 : Survey Results Part 1

In today’s dev update I want to focus on some results of the survey we recently carried out. This will be a 2- or 3-part series, and in the first part we will concentrate on general ‘hard’ metrics, i.e., metrics that rely on the multiple-choice answers, rather than the written text ones.

The survey was carried out in a particular way, so that we could distinguish between two groups; whether a participant had seen it either on:
a) the Tapestry Digital Edition FB group, the Tapestry forum on BGG or the Tapestry Discord Server, or
b) the ‘General’ BGG forum, or anywhere else that Jamey had mentioned it.

This was done to roughly separate the participants into two groups; people generally interested in Stonemaier Games/Board Games, and people interested in Tapestry specifically. A lot of the later statistics/graphs distinguish between these two groups.

So, first of all, thanks to all of you! I didn’t expect that so many people would take part. After removing double submissions – probably due to technical issues – we still had more than 1400 people in the general group, and more than 200 in the Tapestry group. Thanks! Thanks! Thanks! This will definitely have an impact on the Tapestry development!

Furthermore, I think that a lot of these results will be interesting for other developers too, especially the feature importance data and the ‘reason to play’ data. So, if you know of any other digital board game that is currently being developed, do make them aware of this post.

Just before getting into the data, I wanted to let you know i’ve decided to enable the comments section on this webpage again, as I think discussions/comments about this should best take place here, as opposed to on Facebook, or Discord!

When looking at the data, we need to keep two things in mind –

First, the respondents only make up a sub-group of people who play digital board games. Due to where the surveys were posted, most participants are likely to be at least somewhat ‘into’ board games as a hobby. So, keep in mind that there are other gamers that play these digital adaptations, but who do not frequent BGG, or follow Stonemaier Games.

Second, even though we have a total of more than 1600 responses, to be sure about some results we would need to employ some statistical tests. However, as I don’t want to bore you with statistics, I’m not going to employ scientific scrutiny but just present the data as is, just keep it in mind, and look at all of it more like a set of anecdotes 😉

And now, without further ado…

Age

The following two charts show the age distribution amongst the groups ‘General’ vs ‘Tapestry’:

The average age of the General group (‘G’ from now on) is 39.1, vs 41.2 in the Tapestry group (‘T’ from now on). The alert reader might wonder how we arrived at that number, as people only responded in categories. We assumed a uniform distribution within those categories, and calculated the value from there. But even if the number is not 100% accurate, the above pie charts show that the T audience is a bit older. I won’t speculate why this might be the case, but I think it is noteworthy.

Have you played the tabletop version of Tapestry?

Be aware that this question is specifically about the physical version. People who answered ‘No’ here, may still have played Tapestry on Tabletopia, or on TableTopSimulator.

No surprises there. I don’t think anyone expected otherwise. But, if we now look how age influences this value, specifically if we look at the group who are 30 or younger:

The change in the G chart reflects what I would have expected; that the number of people having played the physical version goes down (45.4% -> 40.3%). For the T group, this surprised me, an increase from 81.6% to 88.9%. I am not sure how to interpret this, but it was the first ‘weird’ thing I noticed.

Tapestry Rating

Here we look at the distribution of the ratings that participants gave to Tapestry:

The first charts are, again, nothing unexpected. People interested in Tapestry gave, on average, a 0.67/10 higher rating.

The next charts are now not split along G and T, but rather combine all participants:

‘Owns Tapestry’ vs ‘Doesn’t Own Tapestry’:

With the average rating of 7.09/10 for ‘Doesn’t Own Tapestry’, I would argue that the 7.98/10 rating in the G group previously, implies that there are a lot of participants who own Tapestry, but are not in the T group (maybe due to lack of interest, or maybe because they don’t know about the fact that Tapestry Digital Edition is being developed).

‘Has Played Physical Version’ vs ‘Has Not Played Physical Version’:

Be aware that a ranking itself implies the person has played Tapestry, as the question was phrased ‘If you played it, …’. So, these charts basically compare the ranking of people having only played it on Tabletopia/TTS vs people having experienced the physical version.

I would have expected that people who had experienced the physical version would give a higher rating, but 0.84/10 higher is not what I expected. If you look at the charts, it looks to me not that the whole distribution shifted, but rather a lot of low rankings were added. Maybe this suggests that not everyone has had a worse experience with Tabletopia/TTS, but maybe that a sub-group within the participants just don’t enjoy Tabletopia/TTS.

How many games do you own?

In the base G vs T split, we see only a slight difference:

Participants from the T group seem to own slightly fewer games. Even though this appears as only a slight difference, due to this being at the base level – with still more than 200 participants in the T group – I’d argue that this is still statistically significant. Although in reality, it doesn’t really tell us much.

However, if we look at the distribution for participants that rated Tapestry an 8/10 or higher, it gets interesting:

The number of samples on the T side is 168, so this is not a sample size issue. Participants who rate Tapestry highly within the T group, own less games than the average participant in the T group. In the G group it is the opposite. People rating Tapestry highly, own more games than the average participant in the G group. This is another weird outcome where I’m not sure what to make of it, but one that I still find fascinating.

Do you mostly play …?

This is one of the questions I was really looking forward to. It has a lot of influence on which aspects of the digital version we will place a great deal of emphasis on.

First of all, I separated it into 2 x 3 columns: ‘against…’ and ‘only against…’ – where ‘only against…’ only selected this option, and ‘against’ potentially selected multiple options. To be honest, I suspected that more people play against friends. I’d always assumed I would be more of an outlier, as I personally prefer to play against the AI (keeps downtime low ;-)). But it seems like the majority of games are played against the AI. The differences between the G and T group seem very minor. ‘Against friends’ goes down from 50.4% (G) to 43.5% (T), and ‘only against the AI’ goes up from 42.4% (G) to 48.4% (G). This could also be due to age differences, which I will show in the next graph. If we combine groups G and T, and show the change in these 6 options per age group, the following chart emerges:

As you can see, the older people are, the less likely they are to play against friends, and the more likely they are to play against AI. I suspect this may be due to the reason that at a younger age it is easier to organise playing games together, and as one gets older this becomes more and more difficult. I don’t think it has to do with wanting to play less against humans, as we can see that ‘against strangers’ and ‘only against strangers’ do not change as much.

Finally, if we combine this question with the question of how many digital board games people have played, we can take a look at why people play their first digital board game:

It seems that friends are indeed a much more important factor for people playing their first game. As an anecdote, a lot of my friends always asked – before we played their first online board game – why should we not just play another video game (not a board game) online, aren’t board games quite restrictive?
I think there is a certain barrier to digital board games. People might think ‘board games’ = social games and don’t even consider playing them against an AI. What do you think?

Platforms

This is a very interesting section too. Developers often have to decide on the order of platforms which they release their games on. Whilst we need to keep in mind that the participants are a very biased group – likely deep into board games as a hobby – this is interesting data for any developer or publisher.

While the charts are pretty similar, the highest differences are (G -> T):

Nintendo Switch: 33.5% -> 21.1%
Android Phone: 40.1% -> 34.9%
IPhone: 37.0% -> 32.3%

Often people think that iOS (iPad and iPhone), as it makes up such a large chunk, should be the first platform to be released on – before Steam.
First, this dismisses the fact that even though iPhone and iPad share the same bytecode/underlying platform (iOS) they need drastically different UIs, and so from a development perspective you are developing two different versions.
Second, a lot of people who have an iPad also have an iPhone, and a majority of these people expect only to pay once on the iOS platform. This means from a business perspective, you can’t just ‘add’ them and count them independently.
Last, people want to pay significantly less for a game on iOS than on a computer (we will get to this in part 2 or 3 of the series). So even if the total number of iOS customers would be more than Steam customers – which it is not, don’t forget about Mac users on Steam (!) – it might not make sense from a business perspective.

One thing that surprised me, is that while the numbers for iPhone and Android Phone are pretty similar, the numbers for iPad / Android Tablet are very different. iPad is 43.8% (G) / 41.7% (T), but Android Tablet is 18.3% (G) / 19.2% (T). I wonder why that is (and yes, the platform I mainly play digital board games on is… Android Tablet ;-))

Now… the Nintendo Switch…

Generally, the group of Switch players is not that large. If our goal is to bring the game to as many people as possible, we need to first look at ‘How Switch players play’.

The first question analysis was:

If the main platform a player plays on is the Switch, do they still play on other platforms?:

So, even if one never releases on the Switch (don’t worry, this is not our intention! 😉 ), 90.7% of players can still be reached via a different platform.

What if we look at general Switch ownership?

For people who play on the Switch, no matter if this is their main platform or not, 99% still play on different platforms (no surprise here though).

What about: for the people who own a Switch, how many use this as their main platform?

This is similar to the first question, and thus a similar proportion was expected.

This all suggests that Switch should be a low priority for developers – from a business perspective. However, one thing we shouldn’t forget is that the participants of the survey are a very select group. When releasing a game on mobile (iOS or Android), developers must compete with millions of other titles, however the Switch ecosystem is still comparatively small. Therefore, developers can still expect people to buy the game simply by coming across it on the Nintendo Store. Additionally, pricing is again different.

Feature Importance

The last thing we will look at in this part of the series is feature importance.

Two things I did not expect:
Such a low importance for ‘Animations’.
Such a high importance for Automa.
This is a result that also will have a direct influence on the development of Tapestry – and at least one of them will definitely make some people happy 😉

If we look again at the age split we did before (<= 30 vs > 30), there are a couple of things that are different:

Younger people seem to care more about mixing AI and humans (not sure how to interpret this), and – what I actually did expect – care more about online rankings. Maybe competitiveness fades with age 😉

Next Part

In part 2 – which is now published here – I am addressing things like localisation, accessibility, pricing and perhaps others.

6 thoughts on “Dev Update 4 : Survey Results Part 1”

  1. ery glad to see automa importance being found to be so high.

    I feel like with Scythe, the developers probably assumed “meh who will want an automata when we can give them an entire AI opponent to play against instead?” And I am certain, if that was the thought, that it was a big mistake. People enjoy automa for very different reasons than they enjoy full AI’s. An automa does want to give you a similar feeling to playing against a human, but is not even sort of trying to pretend to _be_ a fully fledged player of the game. It’s its own thing. An AI in the meantime, is trying to pretend to be a fully fledged player–and because we just aren’t good at making fully-fledged artificial players, the experience inevitably disappoints.

    Automa promises a clearly defined interesting challenge. AI, if not done extremely well (as in, better than we are generally capable of doing for games like this), can be fun as well but tends to unfortunately promise not an interesting clear challenge but a fuzz of “why the heck would it do that, this AI is nonsense” feelings.

    1. Thanks for your thoughts!
      Have you never encountered an AI in any digital adaptation where you thought “that is a well done opponent”?
      I feel that there are sometimes AIs that are well done, and people immediately start to assume the AI is cheating (e.g. Star Realms)…

      1. Hive, Race for the Galaxy, and Onitama seem good.

        As to cheating, I’ve never believed a board game app AI secretly cheated or anything. I do have Star Realms, but haven’t played in quite some time and don’t remember what I thought of the AI.

        1. To clarify, I don’t find Star Realms impressive, as I still win vs the hard AI probably 60-70% of the time (but to be fair, I have played hundreds of games). But I do think it is a very challenging AI (up to a certain point).

          I wasn’t even aware of Hive having a digital version. I have to check that out!

  2. Another perspective on the Automa. I don’t consider AI games to be real plays. I don’t log them as plays. It has nothing to do with the moves they take; instead it’s because I can’t talk to them. Playing against an AI is not the same as playing a real board game. I *do* log digital plays against a *solo* system (automa or not) that is the same as the physical game. This makes the solo digital game a one-to-one match with the physical solo game, so it counts. It might be a weird psychological perspective, but I’m not the only one that feels this way.

Leave a Comment

Your email address will not be published. Required fields are marked *