Surfacing design tenets to propel a premium brand

Your product design can propel your subscription efforts, or it can be an impediment requiring gobs of time and effort to overcome. The purpose of this study was to help us accomplish the former.

Advance Local is a hub for a collective of outstanding regional news organizations. Their digital products had always been free and ad-supported. In 2018, they were planning to sell digital subscriptions for the first time. My design team and I wanted to contribute meaningfully to the mission.

We immersed ourselves in design strategies appropriate for different stages of a customer's journey. Our research in behavioral science surfaced an opportunity for design to have a great impact on the discovery stage of the journey.

In a nutshell: A person's automatic thinking process forms first impressions in a fraction of a second. This is accomplished through pre-established shortcuts in our brains. Once formed, these first impressions become anchors, which are difficult and costly to move. A poor perception of value at the start will make it more difficult to convert later.

Design drives first impressions.

Study: Is it free or premium?

We wanted to establish design tenets that would increase the value perception of our experiences. To do that, we conducted a study with end users, led by talented researcher and designer Luwin Changco.

Objectives:

• Validate the notion that design could affect automatic value perceptions of articles and homepages. The research that inspired us did not pertain to digital news products.
• Identify design strategies and tactics that promote high perceptions of value.

Methodology

We collected examples of homepages and articles from eight news websites (alphabetically): Asbury Park Press (regional), Globe and Mail (national, Canada), The New York Times (national), NJ.com (regional, Advance Local affiliated publication), San Francisco Chronicle (regional), Seattle Times (regional), Washington Post (national), and WWLP (local TV station).

We captured four experiences from each publication—two articles (one mobile, one desktop) and two homepages (one mobile, one desktop). Logos and other identifying brand assets were removed.

The format for the test was 10-second testing. Each experience was shown to 60 frequent news readers for 10 seconds. After 10 seconds, we asked them: Based only on what you saw, do you think this website is free or premium (requires paid subscription)? In total, our study comprised more than 1,000 individual sessions.

We also collected open-text feedback but assigned little weight to it during our analysis. Asking participants to explain Automatic System assessments with Reflective System reasoning would have resulted in false insights.

Some panelists (and colleagues) objected to the format. “How can I be expected to say something intelligent after seeing the thing for five seconds?!” Results, however, confirmed that the science and the method are sound. If I could get a do-over, I would reduce the tests to five seconds.

Analysis

After capturing feedback from our tests, we tallied the “free” and “premium” responses. Then we grouped websites by category and studied commonalities among them.

Results

First impressions of premium—the expectation that an experience viewed was associated with a paid subscription—varied from as low as 6% to as high as 46%.

Without the power of the logos to separate the brands, the differences in the scores were surprising to some colleagues. “Isn’t an article an article?,” a senior executive asked. From an Econ perspective, perhaps. The experiences were, at a wireframe level, very similar. Article templates are so similar—headline, hero image, body text, ads—that they are a heuristic. Likewise, homepage ingredients—topic label, thumbnail image, headline, description.

What about content?

Subject matter and quality of content mattered, but not as much as you might expect.

• We carefully selected articles of similar topical weight for the study. The article chosen from the New York Times, for example, was a feature. We avoided weighty opinion pieces about health care or world affairs.
• Our open-text feedback confirmed the fallacy of supply and demand. Local and regional news is more scarce than ubiquitous national political stories, so the local content should be more valuable, right? Although local news often has greater relevancy and more direct impact on people, our study data and open-text feedback make it clear that local news topics are considered less important and therefore expected to be free. “I don’t think premium news would be about raccoons having rabies,” said one participant.

You might assume that national and larger publications dominated the regionals in value perception. That wasn't necessarily the case. Differences that moved the needle for study participants were presentational quality, thoughtful curation, and brand expression. When the experience of the national publication checked the boxes for premium quality, they performed well. When they didn’t, they performed poorly.

Don't try to fight me next time you see me; I'm not saying design is more important than content. I am saying, however, that design has at least as much effect on forming positive first impressions of quality and value perception. We can grapple about that if you want to.

Signals of Premium

Commonalities among top performers:

• Saying "special" with space: Among desktop experiences, the greatest positive correlation with “premium” was generous, effective use of space. Setting select content apart, such as a hero package on a homepage, or giving an article plenty of room to breathe, projects specialness. Setting something apart implies that it’s worthy of being set apart.
• I incorrectly predicted that homepages presenting lesser volume—fewer, more preciously presented content recommendations—would fare better than homepages with more stuff. Think of a shoe store display with three well-lit shoes placed on a spacious table versus a crammed shelf with shoeboxes haphazardly stacked. You'd probably assume the well-presented shoes are more valuable. Regarding homepages, however, organization seemed to outweigh preciousness.
• User-focused curation patterns: Signals of thoughtful, concierge treatment propelled homepages to the top of the spreadsheet. In addition to space, these signals included the use of scale to differentiate levels of importance, thematic grouping of content, organization paradigms, topic labels, and other framing devices, such as description fields under a headline.
• In open-text feedback, it was stunning to see participants, after seeing an article for the 10-second test, voice appreciation for small details, such as a timestamp update.
• Elevation through aesthetics: Aesthetics was a driver of “premium” for all experiences. “It just looked premium.”
• Typography: Higher-quality fonts (subjective appraisal) implemented skillfully were common among top performers. Publications with relatively larger body text took the top six positions among desktop articles and five of the top six on mobile. Serif headlines and body text outperformed sans serif, but this requires more study. In subsequent research, we were able to identify fonts that projected brand attributes that support subscription—quality, value, trust, cognitive authority.
• Color: We wanted to validate research that suggested higher contrast colors and bolder brand colors would be seen as more premium. Our test results were unclear, possibly because the publications in our test shared similar color choices and intensity. A follow-up test of a subscription offer CTA, however, confirmed that use of black and bolder, distinct brand colors with clear contrast were more likely to prompt first impressions of “premium.” Blues and low-contrast colors created an expectation of “free.”

Signals of Free

Commonalities among bottom performers:

• Noise level: The clearest predictor of "free" was noise. Websites with high ad frequency, aggressive ad types, intrusive CTAs, and multiple content recommendations (in articles) were always assumed to be free.
• Sloppiness: Bad line breaks, awkward spacing, clumsy ad placements, and other clear design flaws always resulted in the expectation of “free.”
• Clickbait patterns: Study participants responded negatively to websites that included familiar clickbait design patterns and clickbaity language.
• Prominent social share bars: Prominent share bars always prompted expectations of “free.”

More study needed

We know the results but not the whys:

• Mobile screens: Mobile versions underperformed desktop versions in 13 out of 16 experiences tested. Perhaps it's more difficult to project quality with fewer devices at a designer's disposal—white space, multiple layout columns, broader range of scale. Maybe it's more about the nature of mobile users. More study is needed.
• Article vs. homepages: As you can see, articles were much more likely to signal "free" than homepages. I can't confidently say why, but I can make a couple of observations:
• Articles are less noisy and make greater use of space. When there is less noise, presentational quality and content have greater effect.
• Homepage content recommendations follow the same pattern—topic label, thumbnail image, headline, description—as third-party recommendations described in open-text feedbck as click bait. As insiders, we're quick to cite the differences, but perhaps the differences are less distinct among users. In fact, I know from other research I've conducted that users do not categorize our content as precisely as we'd like to believe.