The return trip

The evening you sign up for something, you are interested. That is the easy part, and it is the part every product measures. The harder question is what happens on a Tuesday eleven days later, when the interest has gone quiet and nothing in your day reminds you the thing exists.

We went looking for that Tuesday. What we found is that for most players, the reminder is a letter, and the letter gets read far more often than the site gets visited.

Who we could actually count

Everything below covers 151 distinct accounts created between 24 July and 13 August, belonging to people who reached the dashboard at least once. Automated traffic, our own testing, and a handful of browsers shared by more than one account are excluded throughout. That last condition is not a quality filter, it is an honest limit. Until you land on the dashboard we have no way of recognising you as the same person on a later visit, so a player who signed up and never got that far could come back every day and we would not know. We would rather count these 151 properly than a bigger number badly.

Every percentage on this page counts players, not visits and not letters. One player who opened six letters counts once. One player who came back four times counts once. And coming back means a visit at least a day after signing up, because the browsing you do in the ten minutes after making an account is still the first trip, not the return one.

The letter is doing the remembering

Take day seven, a week in. Of those 151 distinct people, 18 opened a letter from us that day and 10 came back to the site. Across the whole first week the same gap holds at a larger scale: 94 of the 151 opened at least one letter, while 50 of the 151 made a return visit. Two in three read something. One in three showed up.

It is worth being blunt about what that second number is not. An open is not a session. Nobody analysed a game by opening an email, and we do not get to count a delivery as a sign of life. What the gap says is narrower and more useful: for about a third of these players, the only contact we had in a given week was a letter they read on the way to somewhere else. If that letter had not gone out, the week would have been silent.

What the returners had in common

Of the 94 distinct people who opened a letter in their first week, 40 came back, which is 42.6 per cent. Of the 57 who opened none, 10 came back, which is 17.5 per cent. That is the sharpest split we found, and it is worth resisting the obvious reading of it. Opening a letter and returning to a site are both things an interested person does, so some of that gap is simply interest showing up twice. We cannot tell you from this data how much of it is the letter causing the visit.

The second split is smaller but points at something we can act on. People who linked a chess account came back at 36.0 per cent, 49 accounts of 136. People who never linked one came back at 6.7 per cent, which was 1 account out of 15. Fifteen people is not enough to build a theory on and we are not going to pretend otherwise, but the direction is not surprising. Without a linked account there is nothing of yours to come back to, and the letters have nothing to say on your play.

Reading more on the first evening helped less than we expected. Players who opened five or more pages on day one returned at 37.8 per cent, 37 of 98, against 24.5 per cent for those who read fewer, 13 of 53. Real, but a long way short of the letter split. A long first session predicts a return trip much less well than a letter opened on the following Thursday.

The cohort that looked like it moved

We came into this expecting to write about a sign-up week that fell off a cliff. On the first pass it looked dramatic: one recent week returned at half the rate of the weeks around it. Then we looked at who was in the denominator.

That week had far fewer players reaching the dashboard at all, so far fewer of them were players we could recognise on a later visit. We were not measuring a collapse in returning. We were measuring our own blind spot getting wider. Once the comparison is put on the same footing, by asking the question only of players we can actually follow, the drama goes away.

Three players. That is the entire distance between a week that looks like a crisis and a week that looks ordinary. When a group is this small, the honest move is to print the count next to the percentage every time and let the reader see how thin the ice is.

The three days our instruments were asleep

One more thing had to be checked before any of the above could be trusted, and it is the reason we are wary of week-over-week comparisons in general. A measurement can change without the world changing.

For three days at the start of August our open tracking recorded nothing, while the letters themselves went out normally and every other signal kept flowing. No audience behaves like that. If we had drawn a chart of open rates through that window we would have published a cliff that never happened, and then gone looking for a cause that did not exist.

Because those days sit inside the first week of some of the players counted here, every open figure on this page is a floor rather than an estimate. The real gap between reading and returning is at least as wide as the charts show, and probably a little wider.

What we are doing about it

If two in three players read the letter and one in three comes back, then for a great many weeks the letter is the whole relationship. It should be written that way: worth the two minutes on its own, with the position in the letter rather than behind a link. That is why the letter below arrives with the board drawn into it, and a move you can think about before you decide whether to click anything.

The other half of the work is the arrival. A player who has been away for nine days does not want the same page they saw on day one, they want to know what changed while they were gone. So the dashboard now opens on the games you played since your last visit, and stays quiet when there is nothing to report, because a page that invents news for an empty week teaches you to stop trusting it.

We will run these numbers again in a month with more players in them. If the letter split holds up as the cohorts grow, it settles what the letters are for.