With Qualtrics Value Advisory: How to Optimize Your Research to Navigate Business Uncertainty

2022-08-31

This blog post was originally published on the Qualtrics blog on August 17th, 2022. 

 

Around the world, business uncertainty is up. The global economy has shifted and leadership teams are adjusting the way they make investments. This means that the traditional approach of prioritizing projects and securing budget may no longer be sufficient. Just as customer and employee experience professionals need to respond to the financial uncertainty in their organizations, researchers need to do the same. With emerging markets in uncertain times, getting clarity is vital.

 

Read now: Making the case for CX in times of uncertainty

 

Does business uncertainty have a cost?

 

With perfect information, decisions are easy. But when you are working with variables like the size of your target audience, how much customers will spend on your products, and the cost to serve them, perfect information does not exist. Even in that one sentence, we touched on several noisy but key variables like media costs, the salience of your advertising, the loyalty of your customers, competitor behavior, interest rates, consumer demand, and more.

 

If customers are less loyal than expected, you may overpay on acquisition and erode profit margins. If your advertising is less salient than previous campaigns, you may reach fewer customers. Alternatively, if your customers are more loyal than projected, or your advertising strikes a nerve, you may regret not investing more in a campaign.

 

Regrettable decisions are the cost of business uncertainty.

 

"Many businesses choose to deal with uncertainty through inaction and cost cutting. Organizations that “save” money through cutting research and insight budgets still end up paying a price.

For this group, however, it’s the price of making less informed decisions."

Tom Vladeck
Managing Partner, Gradient Metrics

 

Is it worth spending money to reduce business uncertainty?

 

AND it’s actually more worth it in times of higher uncertainty. When uncertainty is high, organizations should take advantage of the greater payoff of market research efforts.

 

Let us share some general principles for why this is the case:

 

  1. During times of uncertainty, markets are often more unforgiving. Pets.com’s speedy decline to insolvency is a particularly public and impressive example. This compounds with the general trend of customers becoming more demanding over time. A Qualtrics XM Institute study found that 72% of executives foresee their customers becoming more demanding over the next three to five years. In good times, many organizations have the luxury of launching things they aren’t certain will land, take feedback, and pivot. These organizations now carry a heavier burden of getting things right the first time, with less flexibility to learn from failed launches.
  2. Economic disruption often results in conflicting internal metrics. For example, customers may be saying they are happier than ever with a given product at the same time the organization is seeing a drop in customer demand and sales. This Harvard Business Review article shares a common example of conflicting metrics - whether someone who says they will recommend a product actually recommends the product (and vice versa). Research allows organizations to dive into these inconsistencies to understand the underlying factors at play.
  3. Significant market changes often result in significant consumer behavior changes, which contribute to business uncertainty. Organizations may need to reinvent themselves (products, messaging, GTM strategy, etc.) in order to continue delivering value to their customers. We have seen good examples and bad examples of this. While we think “Flights to nowhere” has a nice ring to it, it demonstrates the power of research in preventing bad investments. On the flip side, Trinity Health gives a great example of redesigning experiences based on market changes. They found that hesitancy and lack of access were keeping many people in Black, LatinX, and Asian American communities from receiving the COVID-19 vaccine. In response, Trinity designed a life-saving program that brought access and education to vulnerable communities. Research allows organizations to make the right redesigns.

 

There are also statistically validated reasons why research is more valuable in times of uncertainty than otherwise. The price an organization pays for uncertainty is directly related to the level of uncertainty. As the price of uncertainty increases, the potential value of reductions in uncertainty also increases. Information has a computable value when it reduces uncertainty, even when it is imperfect (Hubbard).

 

How much is it worth spending to decrease business uncertainty?

 

Because it costs resources to reduce uncertainty, smart organizations use analytical rigor to determine the best research investments. And let us be clear that there are times when it SEEMS like research would be super valuable but actually isn’t.

Let us give you two extreme, but not uncommon examples of where it helps to be able to articulate the right value of research:

 

  • A family member of an appliance manufacturing executive has a strong opinion that the manufacturer should make products in a certain color. In a strategic planning meeting, the executive gives the research team a budget to validate this opinion. In this case, the uncertainty of which colors to manufacture (and the regret from choosing incorrectly) may not be sufficient to justify the research expense to validate the opinion.
  • The design team of the same manufacturer doesn't want their product designs to be held up or critiqued by research. They just want them to be manufactured and taken to market. However, a daily used but poorly designed product exposes the organization to significant business risk. In this case, the uncertainty on the design (and regret of making the wrong decision), may be enough to justify the research expense.

To avoid misallocating investments, we must utilize the intersection of financial and statistical models. This approach uses tools from statistics, survey science, and finance to help you allocate the right level of resources to reduce uncertainty. It will enable you to save money where the results won’t move the needle, and to justify larger research expenses when uncertainty is very expensive.

 

1. Create a financial model of the decision

  • In order to evaluate the quality of a decision based on its financial implications, you need a financial model to understand the decision you are making. It doesn’t have to be highly detailed (although it can be!) and it can use shortcuts.
  • For our example, let’s imagine that we are deciding whether to launch a brand campaign. A simple representation of the payoff of this decision might look like this:
    • Campaign Payoff = Addressable Audience x Portion of Audience that Converts x CLV - Campaign Spend

Allow us to define two important terms we are using here to help us make the uncertainty around possible future outcomes more concrete.

  • Payoff: When we say payoff, we are referring to the financial return of a certain decision under certain circumstances. For example, if we decide to invest $1M in the campaign, and there happens to be an audience of 150k people, a conversion rate of 10%, and a CLV of $100, our payoff would be $500k.
  • Regret: Whenever we make a suboptimal decision, given a certain set of circumstances, we experience regret. That regret is quantified as the delta between the payoff of your decision and the ideal decision. For example, if rather than launching the campaign in the circumstances above, we decided to hold. Our payoff would have been $0, with a regret of $500k ($500k-$0).

2. Populate the model with values and scenarios

  • Let’s say we have a reasonable level of certainty about our conversion rate at 10%, CLV at $100, and campaign spend at $1M. But we aren’t sure about the size of the audience that we will reach. We’ll provide the two ends of a broad range where we think this will be as scenarios
    • Scenario 1: 150k people
    • Scenario 2: 75k people

 

3. Compute the payoff and regret for each scenario

  • Scenario 1: 150k x 10% x $100 - $1M = $500k (regrets if held)
  • Scenario 2: 75k x 10% x $100 - $1M = -$250k (regrets if launched)
  • Assuming we had no more information, we would choose to launch. But let’s turn these payoff figures into regret figures. If the audience is 150k, we make $500k. And if the audience is 75k, we only lose $250k. Given the available information, if we held, we would have missed out on $500k in a 150k scenario, but only saved $250k in a 75k scenario. In this scenario, we would experience the least regret by choosing to launch ($250k regret vs. $500k regret).

4. For each uncertain input, calculate the expected value of perfect information

  • Now let’s imagine an oracle can eliminate the uncertainty for us around the audience variable. How valuable would it be to know whether we were in a 150k world or a 75k world? Let’s run the numbers.
    • If we know 150k: We will launch and experience $0 regret
    • If we know 75k: We will hold and experience $0 regret
  • We have reduced our expected regret by $250k. If you are able to pay an oracle any amount less than $250k to eliminate your uncertainty around this variable, pay it!

 

By doing research, you narrow down your scenarios to some subset of the original scenarios. But you don’t know which set you’ll narrow down to. Outlining the payoffs of the possible scenarios allows you to identify the reduction in regret from doing the research. The difference between your original regret and the new regret is the value of that research.

 

What does this mean for your organization?

 

The real world is far more complex than this example we highlighted. There is often more than one uncertain variable. The uncertain times we are in currently are a great example of that. The variables and set of possible outcomes are often continuous as opposed to binary. And research doesn’t always provide 100% certainty for the variable (nor does it have to only reduce uncertainty for a single variable).

If you want to see more examples, check out this guide, which talks about how to establish probabilities as part of your scenario planning and provides calculators to plug in your own possible outcomes.

However, this framework can build on itself in order to address the complex scenarios. This approach can be used to:

 

  1. Identify the most important questions to do research on: Let’s say we had uncertainty around the conversion rate variable. We can use it to determine which variable would have been a more valuable topic for our research.
  2. Determine how much to spend on research: We can use this method to determine the delta in regret and set a value on the research to reduce that regret.
  3. Measure the impact research is driving for your organization: As researchers look to demonstrate the value they are delivering their organizations, they can apply this same approach to put a dollar value on their impact.

 

And this simple model teaches us how much more valuable research is in times of uncertainty. As the expected ranges of uncertain values increase, the amount of least regret increases, as does the value of a reduction in uncertainty.

 

The ability to navigate highly uncertain environments is a competitive advantage. Research functions that are intentional about research efforts, as prescribed in this article, deliver substantial value to their organizations. And the positive impact of managing uncertainty becomes particularly apparent in times of economic disruption as we're currently in.

 

  1. Make a list of the biggest questions your organization is trying to answer. Plug in some broad ranges for your key assumptions and estimate how valuable it would be to reduce uncertainty for each.
  2. Identify the costs for each of the priority projects and compare them to the benefits of the research.
  3. Make a case to your organization to ensure you are appropriately allocating resources to the highest return projects.
  4. Check out some calculators you can use.
  5. Learn more about managing your brand in times of change.

 

Is your organization facing uncertainty? We can help reduce that. 

REACH OUT HERE