Transformer Based Time Series Forecasting

A Deep Dive into Popular Transformer-Based Time Series Forecasting Models

Here, we discuss statistical time series forecasting models like ARIMA, SARIMAX, Exponential Smoothing, and Prophet. Later, deep dive into transformer-based time series forecasting models like Autoformer, Informer, and Temporal Fusion Transformer (TFT). Finally, we discuss the factors to be considered when selecting the right model for your use case. In the next articles, we walk through code implementation using TFT and later compare Statistical and Transformer-based time series forecasting on a retail dataset.

What is a Transformer?

A Transformer is a deep neural network that learns the relationship between input sequences to generate output sequences.

Input sequences can be any text, time-series data, or images. The transformer converts the input sequences into vectors using an Encoder. The Encoder converts input sequences into embedding and positional encodings. The Decoder later converts these encodings back to output sequences.

The most important concept of the Transformer is the “Attention mechanism”. The Attention mechanism identifies which portions of the input sequences are more important than other input sequences and hence should be paid most attention by using the magnitude of attention weights.