Resampling - Why and How

We don’t always need to forecast in the same time scale as our historical data. You may be more interested in an overarching pattern or a predicted total than in seeing a value for every single step in the forecast window. This is where resampling comes in. It’s the perfect technique to reduce information overload and forecast broader trends and seasonality.

How resampling works

Resampling is the process of aggregating your data’s current time scale to a broader frequency, such as converting daily data to weeks or months. Datamago does this by taking the sum of the original values that fall within each new time period.

When to use it

Use it when you care more about the big picture than a detailed forecast. Some advantages are:

1. The technique is easy to use.
2. Forecasts are created much faster due to fewer rows.
3. Forecasts reflect larger scale trends and seasonality since the finer details are aggregated.

How to resample your data

1. Open the forecast configuration sidebar either by clicking the ‘New’ button on the home page and selecting a file, or selecting the ‘configuration’ menu option of a pre-existing forecast.
2. Scroll down and click on ‘advanced options’.
3. Select a new time scale from the resampling dropdown.

Before vs. after resampling

We’ll create a 1 year forecast for bicycle crossings on the Fremont Bridge in Seattle with and without resampling. Let’s assume that the bridge will be under construction during a part of the forecast window and we’re interested in the number of crossings that will be affected. The original forecast is in days, but given this scenario, we’re not as interested in a 365 day forecast since we just want a number for each month.

Before resampling

As you can see, Datamago captures the weekly, monthly, and yearly pattern quite well. It’s worth noting the forecasted year-over-year increase of 4%. However, the forecast took a couple minutes to complete and the status of its performance on the validation set is ‘needs improvement’ (a blog post about validation accuracy is coming soon).

After resampling (to months)

Aggregating the data into months allows the forecast to focus on capturing yearly seasonality instead of also trying to account for smaller scale patterns. As a result, the validation accuracy is substantially better and the forecast took a fraction of the time to create. The forecasted year-over-year increase is also higher at 10%.

The resampled forecast could be further improved with smoothing which is also fast and easy to implement. Learn about the technique here.