Cryptocurrency Forecasting with Deep Learning Chaotic Neural Networks

Background 

In 2008 Satoshi Nakamoto (whose real identity is still very much debated), answered a very challenging question which offers an alternative to traditional centralised payment systems. Instead of each payment having to go through the central server to ensure no double-payments occur, each payment is sent for validation to all peers in the network (Nakamoto, 2008). The authenticity of each payment is done through ‘miners’, key players in the system who use their computers power to mine puzzles and by solving them get rewarded by validating the transaction and receiving a Bitcoin payment. This method keeps the system from corruption and prevents and fraudulent transactions from happening, thus keeping the the peer-to-peer payment system from breaking (Blockgeeks, 2018).

Bitcoin and other digital currencies are also seen as a lucrative opportunity to invest and get good return rates, and therefore have gained increased popularity over the last three years, but these complex and unregulated currencies have also sparked many debates. Some Members of the UK Parliament are seeking financial regulation challenging cryptocurrencies’ volatility (Monaghan, 2018).

In the field of Data Science, there is a growing demand for examining digital currencies’ general dynamics. Gullapalli (2016), created a model utilizing Time-delay neural networks (TDNN) and recurrent neural networks (RNN) on Bitcoin prices collected over the last few years.

Traditional neural networks assume all inputs are independent of each other, which is not the case in many situations and is seen as a challenge when predicting more complex data. On the other hand, RNN are recurrent because they perform the same task for each sequence element, and the output is dependent on the previous computations. In essence, RNN has a “memory” of previous calculations and can make use of information in arbitrarily long sequences. In practice however, RNNs are most successful looking back only a few steps. TDNN on the other hand, is a multilayer artificial network architecture which classifies patterns with shift-invariance, and models’ context at network layer.

Nakano, Takahashi and Takahashi (2018) modelled a seven-layered neural network structure to Bitcoin future return trend prediction. Moreover, the researchers’ approach majorly boosts the buy-and-hold strategy performance. In another work, the researchers presented a deep learning-based system which predicts Bitcoin price fluctuation and transactions based on online forums user opinions sentiment analysis with 81.37% accuracy rate (Kim et al., 2017).

Artificial neural networks were widely employed to predict financial markets via the use of lags as inputs (Adhikari and Agrawal, 2013). However, generating accurate predictions in a complex and fast analytical framework still pertains as definitely a challenging problem.

Aim of Study

The researchers utilized deep learning techniques to forecast the price of Bitcoin, Digital Cash and Ripple, three most widely traded digital currencies. This is the first work to make use of deep learning in cryptocurrency prediction which examines nonlinear dynamics. The aim of the study is fundamentally twofold; firstly, to assess the most active digital currencies’ predictability by analysing their inherent nonlinear dynamics (Lahmiri and Bekiros, 2018).

Secondly, this project implements deep learning as the underlying dynamical system topology to extract hidden patterns uncovering the nonlinear dynamics of their time series. Consequently, this paper is also an objective appraisal of the nonlinear statistical properties of the most active digital currencies, as literature in this research field is not yet published.

By examining whether chaoticity is inherent, short-term predictability of cryptocurrencies would become more successful. In addition, the extraction and exploitation of information hidden in linear or nonlinear patterns within the raw data, in order to build up a complex neural network could simultaneously make more accurate and faster predictions, with time being of utmost importance, according to modern trading practice. Overall, this new understanding of the hidden patters could also lead to better awareness of cryptocurrencies and new regulation being introduced.

Moreover, it is time-consuming and expensive to manually extract domain-specified patterns required for the training of a sophisticated and complex predictive system. The model presented in this paper can be used as a benchmark for future research and for a similar investigation of other currencies or stock.

In sum, the results stemming from a nonlinear-dynamics perspective would indicate whether cryptocurrencies are predictable or not in the short-term, depending on measured fractality and chaoticity, while deep learning results would demonstrate the consistency and accuracy of their forecasting.

Data Acquisition

The dataset used in our study comprises daily prices in US dollars for the following digital currencies: Bitcoin, Digital Cash and Ripple due to their high liquidity and data availability of at least more than one thousand observations for each one, these three cryptocurrencies were selected (See Table 1). Bitcoin has the largest sample of just over 3000 datapoints, generated over 8 years, Digital Cash sample was accumulated over 8 years and Ripple over 3 years.

Table 1: Study Sample

Data Preparation

Cryptocurrency data is available to download for free. The dataset used in this report was generated from multiple online sources on digital currency stock price and transactions. In R and Python, there are different functions which can be used to merge datasets, rename columns to create a new dataset with information for all three digital currencies with a unique point connecting the datasets being the time-stamp. Similar methods were used in other projects (Gullapalli, 2016).

Since the available number of sample observations is limited because digital currencies are new cryptocurrencies or digital assets, 90% of the observations are used for training purposes and the remaining 10% most recent ones are used for testing and out-of-sample forecasting. Moreover, the LLE estimation and the DFA-based HE calculation are used both on learning and testing sub-samples, to examine inherent chaoticity, fractality and any other nonlinear features throughout all time periods. Finally, the forecasting performance is evaluated by using the root mean squared error (RMSE) metric as widely employed in signal processing and prediction literature.

Data Analysis

The study utilized the largest Lyapunov exponent (LLE) and a detrended fluctuation analysis (DFA) based on the extracted Hurst exponent of the time series to detect chaos and/or fractal characteristics of the underlying digital currencies. Hurst exponent allows nonlinear deterministic map checks, whilst the later measures reveal the presence of long memory with no assumptions regarding stationarity. Finally, an intelligent signal data mining and forecasting system is utilised. The system is based on deep learning through Long-Short Term Memory (LSTM) networks. To validate and test the robustness, LSTM model is compared to a benchmark well-known generalized regression neural networks (GRNN).

The Lyapunov exponent is used to understand the hidden properties of standing balance. It is a nonlinear parameter used to quantify the sensitivity to the initial conditions. It indicates the average rate of divergence of two neighbouring attractor trajectories. It is usually hard to calculate LLE of a system or a time series exactly. For a chaotic system, LLE is positive. Therefore, estimating the largest Lyapunov exponent is an important for a nonlinear system.

The Hurst Exponent (H) is a dimensionless estimator for the self-similarity of a time series. Presence of scaling exponents can point to an inner fractal structure of the series. The power law exponent H, is the Hurst exponent. Fractal analysis or moving average estimates this power law exponent H, characteristic for time series. To compare two time series is a difficult task. For digital currencies, usually, H is time dependent. The Hurst exponent can be used to compare time series.

In recent years, the DFA method has become a used technique to determine the fractual scaling properties and the detection of long-range correlations in noisy and non-stationary time series. DFA is a simple mathematical method but very efficient to investigate the power-law of long-term correlations of non-stationary time series (Márton et al., 2014).

The estimated values of the LLE and DFA-based HE for the learning and testing sub-samples of all three cryptocurrencies can be seen in Table 2. The LLE associated with training and testing is positive and negative, respectively. Accordingly, the price sample used in the learning phase exhibits chaotic dynamics, while the testing one reveals convergence to classic attractors.

Table 2: Estimated LLE and HE valuables

The LTSM networks offer a robust expansion of the recurrent neural network (RNN) topology in terms of nonlinear modeling and forecasting. Deep learning LTSM neural networks systems store adjacent temporal information in a spontaneous manner, and control long-term (LT) information, which means the LSTM can keep information from the past, which can significantly help improving its signal sequences and inherent nonlinear patterns learning ability.

In addition, LSTM neural networks offer solution to (RNN) problems such as vanishing gradients, by replacing nodes in the RNN with memory cells and gating mechanism. Therefore, LSTM it is an appealing architecture, also thanks to its’ memorizing power for long and short-term temporal information (Lahmiri and Bekiros, 2018).

The key upgrade of the LTSM is the introduction of a filtering ‘gated’ process. Depending on the inputs, the LSTM memory cell could either remember or forget any cell state. Each cell is supported by three gates (or filters): input, forget and output. The input gate determines how much current information should be treated as input and the forget filter extracts how much information can be kept from the last prior state. Finally, the output gate filters the information which is significant and processes the output, in this context- a forecast (Lahmiri and Bekiros, 2018).

In order to fully appreciate the forecasting power of this model, its’ results are compared to a traditional generalized regression neural networks (GRNN) model. The GRNN architecture usually includes four layers: input (past cryptocurrency price), pattern, summation, and an output layer (future price), which provides the predicted value or forecast. The GRNN is a parallel and memory-based system that estimates the regression surface of a continuous variable while providing fast learning and convergence to the optimal regression surface, as the sample number becomes very large.

Results

The findings from the analysis are presented in the line graph below (Figure 1). Time horizon is represented by x-axis and the y-axis displays price values. The true and forecasted values derived from DLNN and RNNs are exhibited for each currency. It is clear, that the DLNN-based values followed a similar pattern to the observed values, especially when compared to the GRNN forecasts. It could be suggested this is due to the hidden patterns in the raw data. 

Figure 1: Forecasted versus true (observed) values for Bitcoin, Digital Cash and Ripple

Accordingly, the computed Root-mean-square deviation (RMSEs) for DLNN and GRNN are presented in Table 3. RMSE is a commonly used measure of the difference between observed and predicted values by an estimator or a model.

The RMSE scores confirm that LSTM neural networks perform better than GRNN in predicting the future price values of the chosen digital currencies. Deep learning LSTM systems were trained to predict chaotic and self-similar patterns much better than GRNN and robustly forecast future fluctuations. Therefore, the effectiveness of LSTM to model and forecast chaotic financial data structures in case of digital currency markets is confirmed. The implications of the results are discussed in the next section.

Table 3: RMSE scores for Bitcoin, Digital Cash and Ripple

Conclusion

Over the last two decades, there has been an increasing interest in using deep learning computational intelligence systems, particularly in the case of highly nonlinear forecasting problems. There is a growing consensus that deep learning neural networks are more effective than conventional methods used so far in forecasting and analysis of complex patterns in the financial markets (Lahmiri and Bekiros, 2018).

Initial results from investigating the nonlinear structure of the signals in question, showed that digital currencies exhibit chaotic characteristics, yet depending on the sample time period examined. Furthermore, all active cryptocurrencies revealed the existence of strong self-similarity in both training and testing sub-samples.

The deployment of the advanced and conventional neural network architectures found that the LSTMs outperformed significantly the generalized regression neural networks, in terms of the root mean squared error (RMSE). This could be explained by the fact that deep learning neural systems perform well when memorising short- as well as longer-term temporal information simultaneously. This enables DLNN to accurately mine hidden patterns from raw data sequences. Moreover, LSTMs achieved higher scores in learning fractal patterns for the 10% test sample (or prediction period).

The GRNN failed to approximate global patterns including chaotic features, as they are based on Gaussian kernels to locally approximate non-stationary signals with or without a high level of contamination with noise. Kernel methods are a class of algorithms for pattern analysis and the Gaussian kernel separates any sort of nonlinear data exceptionally well. This result could be verified by the fact that LSTMs  took between five and ten minutes of estimation time during the learning phase whilst GRNN concluded training in less than a second.

It is important to note that LSTM networks outperformed the GRNNs in this particular study, which should not lead to assumptions of their predictive ability over GRNNs, as the latter provide improved scores in very large signal lengths and sample observations. Overall, deep learning was found to be highly efficient in forecasting inherent chaotic patterns for the most widely traded cryptocurrencies.

Critical Analysis

This study’s novelty is derived from the analysis on non-linear dynamics. This brings a new set of challenges but also greater awareness of those chaotic patterns within the currency data. One of the reservations I have is the 90/10 ratio of training and testing data. It could lead to overfitting and usually a 70/30 ratio is recommended or 80/20 in a computationally intense model (Babyak, 2004). The issue with overfitting is that when examining the results, it might look as if they are significant, however that is only because the model was trained on the majority of the data and the remainder is insufficient when tested, to indicate otherwise.

In addition, the researchers are not very detailed when explaining the cryptocurrency price used in the prediction model as there are many different KPIs they could have used such as lowest and highest closing price for the day, or the true observed price at a certain time of the day. This makes is very hard to replicate the study, but an assumption can be made that the highest closing price for the day was used.

Examining the future applications, I believe the researchers have started examining a field long overdue, given the fact that cryptocurrencies such as Bitcoin receive major criticisms over their volatility. This effort to better understanding of the chaotic patterns within the data, can lead to a better understanding of how small differences in the environment can influence cryptocurrency.

Nonetheless, one cannot fail to mention the wider applications of studying and analyzing nonlinear dynamics of other complex fields. LSTM seemed to have handled the chaotic prediction factor very well, compared to traditional methods. Which could suggest complex topics such as voting polls, cancer research and climate change could really benefit from a LSTM network-based prediction models.

All in all, the topic and research approach have been receiving increased interest and for a good reason. The prediction accuracy rates really show the potential of the model and this research field would gain greatly from future research on cryptocurrency forecasting using a larger dataset. 

References

1. Nakamoto, S. (2008). Bitcoin: A Peer-to-Peer Electronic Cash System. [online] pp.1-6. Available at: http://Bitcoin: A Peer-to-Peer Electronic Cash System [Accessed 6 Dec. 2018].
2. Blockgeeks. (2018). What is Cryptocurrency: Everything You Must Need To Know!. [online] Available at: https://blockgeeks.com/guides/what-is-cryptocurrency/ [Accessed 10 Dec. 2018].
3. Monaghan, A. (2018). Time to regulate bitcoin, says Treasury committee report. [online] the Guardian. Available at: https://www.theguardian.com/technology/2018/sep/19/time-to-regulate-bitcoin-says-treasury-committee-report [Accessed 9 Dec. 2018].
4. Gullapalli, S. (2016). Learning to predict cryptocurrency price using artificial neural network models of time series. India: Jawaharlal Nehru Technological University.
5. Nakano, M., Takahashi, A. and Takahashi, S. (2018). Bitcoin Technical Trading With Artificial Neural Network. SSRN Electronic Journal.
6. Kim, Y., Lee, J., Park, N., Choo, J., Kim, J. and Kim, C. (2017). When Bitcoin encounters information in an online forum: Using text mining to analyse user opinions and predict value fluctuation. PLOS ONE, 12(5), p.e0177630.
7. Adhikari, R. and Agrawal, R. (2013). A combination of artificial neural network and random walk models for financial time series forecasting. Neural Computing and Applications, 24(6), pp.1441-1449.
8. Lahmiri, S. and Bekiros, S. (2018). Cryptocurrency forecasting with deep learning chaotic neural networks. Chaos, Solitons & Fractals, 118, pp.35-40.
9. Márton, L., Brassai, S., Bakó, L. and Losonczi, L. (2014). Detrended Fluctuation Analysis of EEG Signals. Procedia Technology, 12, pp.125-132.
10. Babyak, M. (2004). What You See May Not Be What You Get: A Brief, Nontechnical Introduction to Overfitting in Regression-Type Models. Psychosomatic Medicine, 66(3), pp.411-421.