The Hierarchy of Marketing Measurement Needs

The marketing effectiveness measurement landscape is confusing, and it doesn’t appear to be getting any simpler.

There are entire academies dedicated to studying the intricacies and dynamics behind whether all that spend actually worked, so if you are struggling, know that you are not alone. There are multiple methodologies available and sometimes it is hard to know where to start, when you just want to do the right thing and measure the impact of your marketing spend.

A major challenge you may face is that there is not just one solution that will give you everything you need, and this means that making that first step is really hard. How do you decide which solution is right for you? How do you know what your measurement priorities are?

As a Marketing Science enthusiast (yes that is a thing), I wanted to focus on this problem from a more user-friendly angle, as I often do when I see complexity being a barrier to measurement excellence. So I have designed what I hope is a useful framework for marketers to use as guidance for solution selection, one that meets them where they are – wherever that is – when it comes to measuring impact.

Every marketer has marketing effectiveness needs, but not everyone is at the same point in their measurement journey. Some are just starting out, some have engaged with a few solutions but aren’t getting the full picture and others have all the bells and whistles but are not sure how everything fits together. So this gave me an idea.

What if we could create a hierarchy of measurement needs? Based on the universally understood Maslow’s hierarchy of more broad human needs. This way, marketers can identify where they are in terms of level of progress, and then follow guidance on what they should be thinking about at that point, and what to do next. So this is what I have done.

Let’s start from the bottom. What is the most basic, foundational element of measurement needs?

Level 1: Coverage

Your first concern has to be coverage. If you aren’t measuring anything, then you can’t make informed decisions; if you are investing blind then you are out in the cold in effectiveness terms. My advice here is not to overthink this stage. Set a KPI for your activity and select a solution that can measure it. There can be different KPIs across campaigns but try and keep cross-channel goals per campaign consistent. These can be brand metrics and/or performance metrics, but they need to be important and therefore linked to your business goal.

Recommendation: MMMs can be expensive and will likely take months to set up (no matter what your potential provider tells you – data is complex and needs to be right for your models to work properly). They will give you coverage but they will take time. I would recommend you put this solution in motion and you have three options (1) kick off conversations with specialist providers (2) build your own measurement team and models or (3) Build out a hybrid solution that is a mix of both.

However, in the meantime – what do you do? There are a number of options you can use to fill the void, and also some of those aren’t just a filler because they are actually complementary to an MMM. These solutions can use a similar methodology to MMMs – that of Bayesian regression modeling, but they don’t rely on first party data. This approach has three benefits – (1) you can get started straight away (2) you are not tied to traditional MMM frameworks so can start to understand alternative KPIs to ROI, those closer to the upper end of the marketing funnel and (3) you can widen your scope to learning from your competitive set because 3rd Party data can be obtained in the same way for you and your competitors.

When selecting potential partners, make sure they are experts at what you really want to know. Do you want to understand how your brand works? Are they checking in with you on a regular basis? Have they been open about the limitations they face in terms of what they can and can’t do? Are you going to be left with a self-serve platform that is not flexible to your actual measurement needs?

Experimentation can be initiated, especially if at a low cost, but it will not get you coverage of multiple channels unless you are setting up geo-tests (which is challenging if you are just starting out). Start to build up your partnerships with the platforms who can help you do this, but don’t rely solely on these methodologies because the minute you have platform specific solutions, they can leave you with significant blindspots.

Watchout: Once you have a certain amount of coverage that you are comfortable with, don’t sacrifice innovation and context for just more coverage. You will get trapped in a loop of measurement for measurement’s sake and insights will be limited. It would be like building more and more shelter within your home at the cost of having someone to share it with.

Level 2: Trust

When you have started to engage with more flexible solutions and are putting foundations in place for MMMs, you need to think about how much you trust them. It is no good if you have full coverage of your marketing spend, but don’t feel comfortable with the data being used or the insights being delivered.

Recommendation: There are three things I would recommend you look at when assessing how much you can trust a solution…

(1) How transparent is the solution? If you don’t know or understand the methodology, then you are unlikely to trust it, especially when you need to put it under pressure. There will always be a little bit of IP held back by a third party provider, that is normal, but if you don’t understand their entire modeling approach, make sure you are asking questions and demanding they open the black box.

(2) Is your provider open about data sources and flaws? Data is always going to be a thorn in the side of marketing effectiveness. All solutions rely so heavily on data quality and availability, that you have to understand the caveats that come with any data that solutions run on. Nothing will be perfect, but you need to understand how wrong it can be and what impact that has on your results.

(3) Are they unbiased and neutral? If you are relying on a platform or agency to provide measurement of their own work, it is definitely worth poking holes in methodology and asking for transparency. For example are they going to feel comfortable telling you something that they own or bought failed?

Watchout: Don’t mistake ‘yes’ people for trustworthy. Your best partners are those that tell you the truth about your approach. If your data is not great, you want to know that.. not just be told everything is fine and just to carry on regardless. Real safety comes through real partnership… which is defined by honesty, not compliance.

Level 3: Understanding

Many measurement methodologies are designed by data scientists who aren’t necessarily that passionate about consumer behavior. That is ok, because they can still build good models that tell you what happened to your investment through a mathematically robust approach. However, we know that at the center of every decision made behind a sale is a consumer – an actual human being. So even if you discover that investment in a certain channel drove sales, you don’t really understand why, and I think that is not good enough in a world with so much data depth and insight available to put the ‘why’ beside the ‘what’.

Recommendation: Do not lean too heavily on either brand measurement or ROI. To truly understand what is happening to your marketing investment and why it is working or not, you will need a combination of both.

The more traditional approach to covering both sides of this performance coin would be MMMs for sales uplift and Brand Tracking for brand equity. However, there are now quite sophisticated modeling solutions that can combine both brand and sales insights into the same frequency design. Brand metrics can be collected more often and more widely through digital platforms and can then be connected to sales impact through modeling methodologies. You also have to remember that brand tracking surveys are based on claimed behavior, and not actual behavior. You can get behavioral insights from brand lift surveys, but again this is limited to channel level findings, so doesn’t really give you the breadth of discovery you need.

Brand metrics can really give you an insight into something going wrong before it hits sales, so it is worth investing in getting this measurement approach right.

Watchout: Don’t be too conservative in your approach to understanding consumer behavior. This is an area where I would recommend innovating and taking some risks with new partnerships and products.

Level 4: Position

So you have enough coverage, you trust your solutions and the data powering them, have layered behavioral insights on top of structural learning, what next? You’ve remembered it is not just you that your consumers are connecting with, it is also your competitors. I am always surprised how brands have such high expectations of their measurement methodologies in terms of granularity, but they don’t take a step back and realize that they are getting close to no competitive insights. So focused on the detail that they miss the bigger picture.

Recommendation: Don’t just think about brand vs performance measurement…think about what your competitors are doing right and wrong as well. Any methodology based on first party data is obviously more accurate in reflecting your activity and the impact of that, but it is also flawed by its very nature. First party data is just that, it only belongs to you, so you have no comparable set of competitive data to run alongside it. There are measurement options out there that need no data input from you because they run on 3rd party data sets that are consistent across your competitive sets. Yes, use measurement solutions based on your own data for detailed insights, but alongside these you must lift your head up and see what everyone else is up to and you can do that through these new solutions. What are competitors doing that may impact your sales, and also very importantly, what can you learn from what they are getting wrong? They can make expensive mistakes against your target market so you don’t have to!

Watchout: Don’t stop at simple competitive reports. You may have noticed the AI revolution upon us, so look for a solution that you can interact with and build answers around your actual area of interest. I find the best approach to understanding competitive context is hybrid, strong AI capabilities with the support of human expertise.

Level 5: Peak

And this is where we all want to be… peak performance, aka self-actualization. Are you being the best you can possibly be at this? When you have a suite of measurement solutions that (1) covers basic measurement needs, (2) you trust to hold up against the scrutiny of a CFO, (3) helps you understand the parts of your marketing funnel that are or are not working, and (4) gives insight into what competitors are getting right and wrong – where do you go from there? Make sure it is upwards.

Recommendation: Whatever methodologies you have selected, they must talk to one another. Having one solution might feel simple, but it is fundamentally wrong. You need to triangulate, calibrate and layer your solutions so that (1) you have at least one solution that can answer your business questions and (2) if you have multiple, they work together or at least you can explain the differences.

Watchout: Not only do you want to consider how your solutions fit together in terms of insights, but also in terms of time dynamics. For example, brand lift will focus on the short term, MMM on the medium term with potential to pull long-term effects from the base. Ensure you are also extracting insights from your solutions as quickly as you can because people change and if you want to stay ahead of competitors, you will need to respond fast.

So my five rules of thumb when looking at whether you are measuring marketing effectiveness right are as follows:

1. Don’t just stick to traditional methods. Try out the new solutions based on tried and tested modeling approaches. Think about what you are going to need in the long-term for marketing budgets, will you get all of your answers from MMM?
2. Make sure you understand the strengths and weaknesses of the data and solutions you are using.
3. Combine methods that give you insight into the ‘what’ of return on investment but also the ‘why’ behind the behaviors.
4. Learn from your competitors as well as yourselves. This needs third party data solutions.
5. Understand how your solutions speak to one another. Remember there is not one solution that can answer all of your business questions on one time horizon.

I realize it is dangerous to bring a simplification tool to a community of analysts, and I am sure there will be a lot of ‘feedback’ I will receive on this model of thought. However, I hope it does what it is supposed to do….widen the community of marketing science enthusiasts and remove the barrier of entry to the more cautious marketer.