LSTM

When working with time series data, usually the first models are moving average models, autoregressive or a combination such as ARIMA. I tried using ARIMA to predict the foreign exchange for Australia to the US dollar. After many attempts and trying to figure out why this wasn’t working I found out that since the data is not seasonal moving average and autoregressive models are not the best option. I have spent too much time on this project to leave it at that. So I started looking into other time series models and Long short term memory (LSTM) models popped up.

Before I dive into my project with it I wanted to learn more about it so I had a firm grasp about what is going on. LSTM is great for predicting time series that is dependent on the previous information. This is because it stores the past information because it is a recurrent model. Feedforward models can not store past information to be used later on.

There are different types of LSTM models: univariate, multivariate, multi-step and finally multivariate multi-step.

Univariate

Univariate are problems of single observations and a model is required to learn from the series of past observations to predict the next value in the sequence. This sounds like what I will be using for my foreign exchange rate project. I am only working with the exchange rate and it can learn from the past rates to predict the future ones.

Multivariate

Multivariate are similar to univariate but they have multiple observations that are used. This would be a good predictor for weather where you have multiple observations at one time.

Multi-step and Multivariate Multi-step

Multi-step is used to get outputs that are more than one step away, so think of it as needing to predict the exchange rate everyday for the next month. Then the multivariate is the same thing but you have more than one observation.