{"id":2664,"date":"2025-03-12T20:40:23","date_gmt":"2025-03-12T13:40:23","guid":{"rendered":"https:\/\/mintea.blog\/?p=2664"},"modified":"2025-03-12T20:40:50","modified_gmt":"2025-03-12T13:40:50","slug":"2664","status":"publish","type":"post","link":"https:\/\/mintea.blog\/?p=2664","title":{"rendered":"How to investigate a spike in your data"},"content":{"rendered":"

How to investigate a spike in your data<\/h2>\n

So, you\u2019ve just noticed a spike in your data. Maybe your trendline looks something like this?<\/p>\n

\"A<\/p>\n

Or if you\u2019ve caught it early, it might even look something like this:<\/p>\n

\"A<\/p>\n

Now, obviously, I have no way of knowing if your y-axis represents something really good (like new customers) or something really bad (like system errors). And depending on which it is, you may be feeling a strong urge to pop the champagne or hit the panic button.<\/p>\n

But as we know, data can be deceptive, so don\u2019t jump to conclusions just yet. Instead, here are some simple steps to work through when you notice a spike in your data.<\/p>\n

1. Is this metric important?<\/p>\n

Before investigating any data anomaly, it\u2019s important to triage the issue. This makes sure we respond to the most important issues quickly, without unnecessarily distracting ourselves from our current priorities. Ask yourself: if this spike was genuine and represented a real-world change:<\/p>\n

    \n
  1. Would the consequences be significant?<\/li>\n
  2. How urgently would they require attention?<\/li>\n<\/ol>\n

    Scenario A: It\u2019s significant and needs urgent attention.<\/strong>
    \nIt\u2019s potentially a critical issue, so you may need to begin a parallel process. Respond to the issue as if it was genuine, but also begin the process of investigation. Remember to clearly communicate to others the extent to which you have been able to verify the data.<\/p>\n

    Scenario B: It\u2019s significant, but doesn\u2019t need urgent attention.<\/strong>
    \nIf you responded to every blip in your data immediately, you would never get anything else done. But that\u2019s okay. Schedule a time to investigate the issue (and make sure you follow through.)<\/p>\n

    Scenario C: There wouldn\u2019t be any significant consequences.<\/strong>
    \nRemember that\u00a0<\/strong>not every anomaly requires detailed investigation. Of course, curiosity is a valuable quality for anyone working with data \u2013 but so is efficiency. And living in the \u2018era of big data\u2019 doesn’t mean you have to be across every data point. Smart leaders choose to focus on the metrics that matter. Prioritize your investigations, or give them an appropriate time frame.<\/p>\n

    2. Is the spike just a natural variation?<\/p>\n

    Sometimes we see irregularities in data that are, in fact, completely normal. Ask yourself: how does the metric usually behave? Is it usually steady, or does it carry variance? (In other words, does it jump around a lot?)<\/p>\n

    A simple way to check this is by looking at how the trend changes over time. Have there ever been spikes this extreme before? Is the current spike consistent with any wider trends?<\/p>\n

    In the example below, a Customer Success Manager is looking at the average first response time over the past week. They\u2019ve noticed what looks like a spike in their data.<\/p>\n

    \"Looking<\/p>\n

    But that\u2019s because this metric carries a lot of variance, especially when calculated each day. By looking at the trendline over time (in this case, over the past month), the manager can see that this spike is completely normal.<\/p>\n

    \"Looking<\/p>\n

    Often, like with the example above, a quick look at the shape of the data over time will tell you if your spike is normal. However, if you want to be more rigorous, you can employ statistical process controls.<\/p>\n

    Statistical process controls (SPCs)<\/p>\n

    Statistical process controls are a more scientific way of determining whether a metric is behaving normally. Typically, the control would involve a series of tests; for example, do any of the data points fall outside of a defined \u2018normal\u2019 range. SPCs are also useful because they can be automated.<\/p>\n

    A common example of an SPC is an XmR chart \u2013 we\u2019d recommend this guide to\u00a0building your own XmR chart<\/a>\u00a0by Stacey Barr.<\/p>\n

    3. Consider seasonal trends and natural cycles<\/p>\n

    Some metrics vary according to natural cycles. Many sales patterns, for example, are tied to the calendar year. Think about consumer purchase behaviour around Black Friday and in the run up to Christmas, new gym memberships in January or ice-cream sales in summer.<\/p>\n

    \"Homer<\/p>\n

    When we correctly determine natural cycles, we can use this to add valuable context to our data. For example, this ecommerce site has added a yearly comparison to their trendline. It helps you to determine whether the spikes in December and May are normal, or the result of a seasonal trend.<\/p>\n

    \"trendline<\/p>\n

    Similarly, awareness of cycles can help us discover whether irregularities are happening that would otherwise appear normal. In the example below \u2013 this product would usually experience an uptick in sales in the run up to Christmas, but this hasn\u2019t happened this year. Without the comparison, we might incorrectly interpret the trendline as \u2018normal\u2019, with no cause for alarm.<\/p>\n

    \"seasonal<\/p>\n

    Be careful when making cyclical or seasonal comparisons. Particularly as they can be prone to our own biases. Find a way to test or verify these assumptions.<\/p>\n

    4. \u00a0Is there a data quality issue?<\/p>\n

    Your spike could be caused by a data quality issue, as opposed to a real-world change.<\/p>\n

    Data quality issues occur when there is a problem in the way data is collected or recorded, or because of non-genuine inputs. An example of a non-genuine input would be a member of your team testing a feature multiple times, without disabling the analytics, which then causes a usage spike.<\/p>\n

    Consider the obvious causes of data quality issues, and try to rule them out. Many common issues include:<\/p>\n