IoT Traffic Prediction Using Multi-step Ahead Prediction with Neural Network

IoT traffic prediction using multi-step ahead prediction with neural network

Abstract-The Internet of Things (IoT) is basically a network of interconnected devices, like sensors and smart devices, that have processing, sensing, and communication capabilities and pass on the information to each other and a supreme console via the internet. Network traffic prediction is an operational and management function that is critical for any data network so, it has a significant role for today’s increasingly complex and diverse networks. Also, the network traffic prediction is more important for the IoT networks given the number of connected elements and the real-time nature of many connections. The artificial neural network (ANN) has been successfully applied to traffic prediction. In this paper, we perform the IoT traffic time series prediction using a multistep ahead prediction with Time Series NARX Feedback Neural Networks. The estimation error of a prediction approach has been evaluated using the performance functions MSE, SSE, and MAE, besides, another measure of prediction accuracy the mean absolute percent of error (MAPE).

 

Keywords: Prediction; IoT; Traffic; Artificial neural networks; AI

1. Introduction

IoT device is any kind of device that has processing, sensing, and communication capabilities. It’s composed of billions of these devices that connected to the Internet and forming dynamically changing ad-hoc connections among them in any possible communication pattern. The range includes devices of any conceivable size, functionality, and applicability.

The Internet of Things (IoT) is one of the priorities of the development of the info-communication system, which construction concept is reflected in the recommendation ITU-T E.800 [1]. The development of IoT is an extremely important step, as it affects almost all areas of human activity. The penetration of the Internet of Things will contribute to the availability of more and more information, the growth of its analysis capabilities, the formation of decisions and actions based on its results.

Network traffic prediction [2-4] is one of the main application areas of artificial intelligent (AI) to data networking. Traffic volume prediction can be defined as the forecasting of incoming and outgoing bytes-count at different connections levels in the network hierarchy (device, link, routing…). Traffic prediction is a critical element in network operations and management: congestion control, routing, resource allocation and management of service level agreements (SLA), among many other network responsibilities and functions.

According to [5-7], the traffic generated by the Internet of Thing devices can be divided into three characteristic types: deterministic – produced by devices operating on a fixed schedule; deterministic technological – necessary to maintain the functioning of the system; mediated i.e. generated as a reaction to some external events. The traffic generated by the Internet of Things devices can be served together with the traffic of other communication services, for example, base stations, wireless access points, and other network nodes.

In this work we present the multistep ahead prediction with Time Series NARX Feedback Neural Networks for IoT traffic we predict the packet loss rate with time series prediction we use make the prediction in two cases when the number of packets sent 2 pckts/s and 10 pckts/s. The estimation error of a prediction approach is evaluated using MSE, SSE, MAE and MAPE.   Table.1 shows the list of used abbreviations in paper.

The outline of this paper is as follows: section (1) Introduction; section (2) discusses Prediction using neural networks; section (3) discusses recurrent neural networks; section (4) discusses Multi-Step Prediction; section (5) discusses IoT model simulation; section (6) gives our experimental result; section (7) conclusion.

Table. 1 List of abbreviation

MSE	Mean square error
MAE	Mean absolute of error
SSE	Sum square of error
MAPE	Mean absolute percent of error
RNN	Recurrent Neural Network
ANN	Artificial Neural Network
MLP	Multilayer perceptron
NARXNET	Nonlinear auto-associative neural  network with external input
SLA	service level agreements
IoT	The Internet of Things
AI	The artificial intelligent

2. Prediction using neural networks

Artificial neural networks (ANNs) [8-9] can be applied for the prediction with various levels of success. The advantage of ANNs includes automatic learning only from the measured data dependencies without any need to add more information (such as a kind of dependency like with the regression), in addition they have the ability to learn by examples only and after their learning is finished, they can catch hidden and strongly non-linear dependencies, even when there is significant noise in the training set. 

ANN is trained using the historical data with the hope of discovering hidden dependencies and that it will be able to use them for prediction of the future. In other words, the neural network is not represented with an explicitly given model. Neural network has been described as more a black box that can learn something.

It is possible to predict [10] several types of data with time series. the time series displays the development of value in time and this value can be influenced by other factors than time. Time series represents a discrete history of value and from a continuous function, it can be acquired by sampling.

3.       Recurrent Neural Networks(RNN)

Today’s recurrent neural networks (RNNs) have been proving themselves as powerful predictive engines. It has been successfully applied to time series prediction. In RNN, the temporal relationship of the time series is explicitly modeled using feedback connections to the internal nodes (known as hidden units). Recurrent means the output at the current time step becomes the input to the next time step Fig. 1 shows overview for RNN architecture. At each element of the sequence, the model considers not just the current input, but what it remembers about the preceding elements [11].

An RNN model is trained by presenting the past values of the time series to the input layer and The weights of the network are then adjusted based on the error between the true output and the output predicted by the network until the algorithm converges. Before the network is trained, the user must specify the number of hidden units in the network and the stopping criteria of the learning algorithm.

 time series problem, predicting the future values of a time series y(t) from past values of that time series and past values of a second-time series x(t). This kind of prediction is called nonlinear autoregressive with exogenous (external) input, or NARX Network (narxnet, closed loop), and can be written as follows [12]:

$\mathbf{y}\left(\mathbf{t}\right)\mathbf{=}\mathbf{f}\mathbf{(}\mathbf{y}\left(\mathbf{t}\mathbf{–}\mathbf{1}\right)\mathbf{,}\mathbf{.}\mathbf{.}\mathbf{,}\mathbf{y}\left(\mathbf{t}\mathbf{–}\mathbf{d}\right)\mathbf{,}\mathbf{ x}\left(\mathbf{t}\mathbf{–}\mathbf{1}\right)\mathbf{,}\mathbf{.}\mathbf{.}\mathbf{,}\left(\mathbf{t}\mathbf{–}\mathbf{d}\right)\mathbf{)}$

     (1)

This model could be used to predict future values of a stock or bond, based on such economic variables as unemployment rates, wireless traffic variables, etc. It could also be used for system identification, where the models are developed to represent dynamic systems, such as chemical processes, manufacturing systems, robotics, aerospace vehicles, etc.

 

Fig.1 Recurrent Neural Networks

4. Multi-step Prediction

Multi-step prediction predicts the future values of a time series in a step by step manner. We first predict x_t+1 using the previous p values, x_t+1-p,…, x_t-1, x_t we then predict x_t+2 based on its previous p values, which includes the predicted value for x_t+1. The procedure is repeated until the last value, x_t+h, has been estimated. In this approach, it is sufficient to construct a single model for making the prediction [11].

5. IoT model simulation

the model shown in Figure 1 was chosen. The model consists of an IoT traffic generator that simulates the operation of one or a group of IoT devices, Traffic generator of traditional communication services and TI traffic, designated as H2H + TI (H2H – Human to Human, TI – tactile Internet). The produced incoming traffic streams to communication node, the model of which is presented as queuing system with Combined Service Discipline (with delay-basis and failure-basis system). The average service time of a packet (message) is equal to ̅.

The Internet of things traffic arrival rate is denoted by ${}_{\mathit{IoT}}$

, H2H traffic – ${}_{h2h}$

, aggregated stream ${={}_{h2h}+}_{\mathit{IoT}}$

. With probability p, a packet arrives at the input of the system where all positions in the queue are occupied and get a service denial (losses occur). The aggregated traffic stream at the system output has a total intensity of λ. The properties of the aggregated traffic stream at the system input are determined by the properties of both streams, therefore, in general, it differs from the properties of both traditional traffic and Internet of Things traffic.

To build the IoT model, the Anylogic simulation system was chosen, which allows creating discrete event simulation models.

To simulate a self-similar stream, a generator of a sequence of independent events was used, the time intervals between which are random and have a Pareto distribution.

Fig.2 Service model of aggregated traffic

6. Simulation results

In this work we perform IoT traffic prediction approaches using multistep ahead prediction with NARX neural network.   The estimation of error prediction was evaluated using the performance functions MSE, MAE, SSE and another measure is the mean absolute percent of error (MAPE).

Input-output time series problems depend on the prediction of the next value of one time-series given another time-series. The past values of both series (for best accuracy), or only one of the series (for a simpler system) may be used to predict the target series.

 The dataset can be used to demonstrate how a neural network can be trained to make predictions. The datasets are obtained from IoT traffic generator the IoT model was simulated using Anylogic simulator, after collecting and preparing the dataset, it was split randomly into 75%, 15% and 15% for training, validation testing, respectively. The feedback neural network was implemented to predict the performance accuracy of IoT traffic.

Table.2 shows the prediction accuracy for IoT packet loss rate using above mentioned performance functions and the another measure of performance accuracy MAPE.

Table. 2 the accuracy measure for the predicted model validation

Early Predict Performance
No of packets/s	MSE	SSE	MAE	MAPE
2 pckts/s	5.8208e-06	5.2387e-05	0.0015	00.18%
10 pckts/s	1.1145e-05	1.0030e-04	0.0015	6.18%

Table.2 displays the performance prediction of IoT traffic in case of the number of packets 2 pckts/s and 10 pckts/s in order to estimate the error of prediction we use the traditional performance functions MSE, MAE, SSE and another measure for performance accuracy MAPE.

From the tabulate results, the performance predicted based on the MSE performance function has the best performance in the case of 2 pckts/s and 10 pckts/s in comparison to its peers. Also the SSE performance has performance which is approximately equal to MSE performance functions. While performance of prediction using MAE performance function is worse than MSE and SSE.

On other hand, the MAPE has the best prediction accuracy in case of number of packet 2 pckts/s with percent 0.18% while in case of 10 pckts/s it has the least prediction accuracy with percent 6.18%.

Fig. 3 shows two curves the first curve observe the multistep ahead prediction for IoT traffic with time and the prediction time 12 which with the aim of verifying the ability of the ANN in predicting for IoT traffic load in the cases of number of packet sent 2 pckts/s. it illustrates the result, where it is clearly seen that, the packet loss rate with time for the observed and predicted models we notice from that the predicted packet loss rate increase starting from time 1 until time 4 then decrease until the time 10 which give the best prediction accuracy.

The second curve shows plot of the estimated error (difference between desired output and predicted output) of prediction with time

Fig. 3 The response of output element for time series in case of number of packets 2 pckts/s.

Fig. 4 shows two curves the first curve observe the multistep ahead prediction for IoT traffic with time and the prediction time 12 which with the aim of verifying the ability of the ANN in predicting for IoT traffic load in the cases of number of packet sent 10 pckts/s. it illustrates the result, where it is clearly seen that, the packet loss rate with time for the observed and predicted models we notice from that the predicted packet loss rate increase starting from time 1 until time 4 then little bit decrease until time 6   become constant until the time 10 which give the best prediction accuracy.

The second curve shows plot of the estimated error of prediction with time

Fig. 4 The response of output element for time series in case of number of packets 10 pckts/s.

7. Conclusion

In this paper, we proposed the ANN to predict the IoT traffic, we proposed the prediction approach multistep ahead prediction time series with feedback neural network for prediction IoT packet loss in order to promote IoT traffic prediction accuracy. The prediction accuracy of neural network learning process has been estimated in terms MSE, MAE and SSE in addition, another measure for accuracy is MAPE. Intensive analysis and simulation results show that the MSE performance function has the best prediction accuracy in comparison to its peers and MAPE has the best prediction accuracy in case of number of packets 2pckts/s.

References

[1]      ITU-T, ―E.800: Terms and definitions related to quality of service and network performance including dependability, 2008.

[2]      Raouf Boutaba, Mohammad A. Salahuddin, Noura Limam, Sara Ayoubi, Nashid Shahriar, Felipe Estrada-Solano, and Oscar M. Caicedo,” A comprehensive survey on machine learning for networking: evolution, applications and research opportunities” Journal of Internet Services and Applications, https://doi.org/10.1186/s13174-018-0087-2, Jun 2018.

[3]      M.S. Mahdavinejad, et al., Machine learning for internet of things dataanalysis: A survey, Digit. Commun. Netw. 4 (3), pp. 161–175, 2018. https://doi.org/10.1016/j.dcan.2017.10.002.

[4]      Manuel Lopez-Martin, Belen Carro, Antonio Sanchez-Esguevillas “Neural network architecture based on gradient boosting for IoT traffic prediction”, Elsevier, Vol. 100, Nov 2019, pp. 656-673, https://doi.org/10.1016/j.future.2019.05.060.

[5]      A. Muthanna, A. Khakimov, A. Ateya, A. Paramonov, and A. Koucheryavy,” Enabling M2M Communication Through MEC and SDN”, In International Conference on Distributed Computer and Communication Networks, Springer, Cham, pp. 95-105, 2018.

[6]      Paramonov A., Koucheryavy A. “M2M Traffic Models and Flow Types in Case of Mass Event Detection”, NEW2AN/ruSMART, Lecture Notes in Computer Science. Vol. 8638. pp. 294-300, 2014.

[7]      A Muthanna, A.; A. Ateya, A.; Khakimov, A.; Gudkova, I.; Abuarqoub, A.; Samouylov, K.; Koucheryavy, A. “Secure and Reliable IoT Networks Using Fog Computing with Software-Defined Networking and Blockchain”. J. Sens. Actuator Netw. 2019, 8, 15.

[8]      Ali R. Abdellah, Ammar Muthanna, Andrey Koucheryavy “Robust Estimation of VANET Performance-based Robust Neural Networks Learning”, 19th International Conference on Next Generation Wired/Wireless Advanced Networks and Systems (NEW2AN’2019).

[9]      Ali R. Abdellah, Ammar Muthanna, Andrey Koucheryavy “Energy Estimation for VANET Performance Based Robust Neural Networks Learning”, XXII International Conference on Distributed Computer and Communication Networks: Control, Computation, Communications (DCCN’2019).

[10]   https://www.obitko.com/tutorials/neural-network-prediction/prediction.html

[11]   Haibin Cheng, Pang-Ning Tan, Jing Gao, and Jerry Scripps, “Multistep-Ahead Time Series Prediction”, 10th Pacific-Asia Conference, PAKDD 2006, Singapore, pp. 765 – 774, April 9-12, 2006, Proceedings.

[12]   Zina Boussaada, Octavian Curea, Ahmed Remaci, Haritza Camblong, and Najiba Mrabet Bellaaj,” A Nonlinear Autoregressive Exogenous (NARX) Neural Network Model for the Prediction of the Daily Direct Solar Radiation” Energies, doi:10.3390/en11030620, 10 March 2018.