Looking For Value in Monthly Reports

6632168217_bdc32df214_b.jpg

Monthly reporting is something that everyone in marketing has dealt with before, and finding value in every month’s report can be difficult.

It is easy to copy the outline of the previous month’s report and update with the current month’s numbers. So, to make sure your monthly report isn’t, ask yourself, is your report adding value? The best way to add value to monthly reports is to add context to the data. I share the opinion with Nate Silver that data inherently needs context to be worthwhile. Without context, data is just data; it is just a grouping of numbers that have no real purpose.

Segmented Data

As I have previously said, there are limitations to data, but they mostly come from the human element of data analysis. There is nothing wrong with the numbers (as long as the collection and application was correct) so the errors and disagreements arise during the analysis stage.

It is virtually impossible to come up with a consensus when analyzing intricate data sets similar to those collected with tools like Google Analytics. Many of the issues associated with data can be found in how we apply context to data. Throughout this post, I will use an example website to show how context influences the numbers. Look at the following graph. Based on the data presented, it appears June was a terrific month for website traffic.

Here, I used a month-to-month view that segments the data to provide context, a technique frequently used in monthly reports. Segmenting data is a great way to add context to data, but it is possible to segment down too much. Now, take a look at the 13-month trend for this same website:

The month-to-month data is the same as before, but the historical trend shows a much larger and concerning trend that you can take immediate action on. Month-to-month numbers help put the current month in comparison with the previous, but they don’t possess any value beyond saying whether the numbers were up or down.

The same can also be said for year-to-year numbers. By only looking at two months, we can’t see anything outside of this small sample size. There are simply too many confounding elements when working with segments such as month-to-month and year-to-year to trust their numbers alone. If the green and red arrows of a report don’t appear to contain any value, they probably don’t. View data the same way you view any other marketing decision. If it is a waste of time and money, demand an overhaul of the reporting methods.  

Signal v. Noise

It can be tough to figure out which data points are signals worth tracking and which are just noise. Have you ever received a report that has an impressive amount of data but also has a severe lack of value? Using the same graph from above, we can show how merely presenting the number of monthly users may be a noisier trend than we think. In the graph below we can see the conversion rate for contact form submissions.

Presenting raw conversion numbers is subject to the traffic data. The trend would appear similar to the monthly user trend presented earlier, but this wouldn’t actually serve as a reliable method for understanding the current state of conversions.

Conversion rate, on the other hand, normalizes the data for sample size (in this case, number of users). We can see that, although user traffic is down, the conversion rate is actually experiencing an upward trend. The real signal is conversion rate and not total conversions in this example. Deciding what to include in a monthly report is a conscious decision. Sometimes the raw numbers look better than the rate numbers, but rate numbers tend to provide more value.

Once we look at the long-term trends and rate numbers, we can see what needs to be improved. Content may be missing the target market when we notice long-term drops in traffic. An SEO or content update might help cure this issue. Further, we can run usability tests to gain insight into how the website is performing. Perhaps, a few years ago the site was very sleek and modern, but over time the site has aged and users no longer perceive your business as a legitimate option. The data in the reports won’t hold magical answers, but the reports should indicate what you need to improve. From there, you can build a strategy to figure out exactly what is going wrong and how to fix it in order to get your online presence to where it should be.

Outside Context

The two examples we have gone over so far have provided context as to what the current state of the website is. The next step to looking for value in a monthly report is to try to explain why the numbers are trending the way they are. Again, context is essential to finding answers to these questions.

Let’s continue with our example website numbers. It has become fairly obvious that something is causing the traffic to decrease while the conversion rate has actually been increasing. After some discussions with the client, we find out they were running some out-of-home ads from July to October. This helps explains the influx of user traffic, but it doesn’t fully explain the increased conversion rate after the campaign stopped running.

Again, we look for more context as to what potentially caused the increased conversion rates. After the out-of-home campaign failed to bring in conversions, several SEO changes were made in December that most likely had a positive impact. Adjusting the on-page content on multiple service pages has improved the targeting. While these strong correlations don’t prove causation, we can assume they played major roles in the trends we observed. Your reports should contain this type of valuable information. The reports you receive aren’t providing actionable data if they don’t include this sort of information.

Actionable Steps

Now that we have all this data, we need to use it as more than a status update. There’s no point to have a monthly report if no changes are being made between updates. We have already taken into account the outside context to try to find out why the numbers are they way they are.

The next step is to take some time to think about the recommendations and goals from the report in an attempt to improve the numbers. It is fairly obvious to concur that you need to get traffic trending in the opposite direction. If you can improve traffic, raw conversions should increase along with it. The recommendations to move forward with should be a consensus decision between agency and client, and a timeline should be set to reconvene to update how the recommendations are doing. An optimal report would include reasons as to why the recommendations are made. If it does not include them, don’t carry on before addressing the “why” behind the proposed changes. This includes the scheduling of the reports. These do not have to be every month. Monthly reporting should only be used if it makes sense for the situation. Reports should be scheduled around obtaining adequate sample sizes, not some arbitrary time frame that seems convenient. If it feels like a cookie-cutter report, it probably is. Ideally, actionable reporting should be an on-going process in order to provide long-term, sustainable improvement. There should be limited pushback, if at all, when you suggest making changes to the reports.  

What An Actionable Report Includes

Much like an ad campaign, monthly reports should be targeted for a specific audience. The target will determine which data to include. For the most part though, the main rule to follow is include data that portrays the current state of the business.

Examples:

  • Overall Traffic
    • Segment paid, organic, direct, referral
  • Conversions
  • Conversion Rate
  • Downloads
  • Recommendations

Conclusion

Reading monthly reports can seem tedious and wasteful because it often is. Recognizing the difference between an actionable report and a status report is very important to improving your business. Search for context within the report. Without it, the numbers don’t really have any value. Data does not exist in a vacuum, so be very cautious if is presented this way. In the end, never stop evaluating the cost and value of the reports you receive. If the cost outweighs the value, it may be time to make a change.