Data Processing in Big Data Centres Cost Reduction Approach

ABSTRACT

The tremendous development in cloud data processing leads to the high load on computation, storage and communication in the data storage centers, which influence the data center providers to spend a considerable expenditure in data processing. There are three features leading to this increased expenditure, ie., job allotment, data positioning and data movement. In this paper, these three features are taken into consideration and an approach for cost reduction for cloud data processing is proposed. I propose a Markov Chain Model to analyze the task completion considering the data transmission and its computation.

Keywords: Markov Chain Model, Data Center, Cloud data, Data Positioning, Data Processing.

INTRODUCTION

In recent years, the outburst of data all over the world has led to the demand of data processing in the data storage centers. This demand further leads to the increase in the cost incurred in the computation and the communication resources. As predicted by Gartner, by 2015, 71% of the data storage center hardware utilization would be from the cloud data processing which will cross around $126.2 billion. So, it is of vital importance to analyze the cost reduction problem in cloud data processing in the data storage centers.

Data Center resizing (DCR) has been proposed to reduce the cost involved in data processing by adjusting the number of activated servers through task placement^[1].

The Cloud Data Service Architecture mainly consists of distributed file systems which is helpful in distributing the data and its copies all over the data centers for an efficient load balancing and high performance. Some studies focused on reducing the communication cost by taking steps to place data on the servers where the input data exist to solve the remote data loading problem.

Get Help With Your Essay

If you need assistance with writing your essay, our professional essay writing service is here to help!
Find out more about our Essay Writing Service

Even though there were many solutions proposed to solve the above issues, none of the solutions were helpful in providing a cost efficient big data processing due to few disadvantages. First one, being the wastage of resources for the data that is not often accessed. Second, being the transmission costs involved depending on the distances and the type of communication used between the data centers. Not all the data could be stored on the same server because of its high volume; it is a mandatory one to store few data into remote servers that would incur transmission cost. Transmission costs get increased proportionally with the number of communication links involved.

To get rid of the above disadvantages, I consider the cost reduction for cloud data processing through a joint optimization approach of task placement and data positioning in the data centers. Every server may have only a few resources needed for each piece of data residing on it. The data will need more resources to carry out with its big data processing tasks. The main aim of this paper is to optimize the data positioning, task allocation, routing and DCR to minimize the overall computation cost involved. The contributions are briefed as follows,

1. This paper considers the cost reduction problem involved with the cloud data processing in the data centers by the joint optimization of data positioning, task allocation and routing. To explain the computation and the transmission involved with the data centers, the Markov Chain model has been used and the task completion time has been derived.
2. For cost reduction, three factors are taken into consideration. The first one is how to place data in servers and the second one is how to distribute the data and the third one is how to resize the data centers to achieve minimum cost operation.

OTHER RELATED WORKS

Cost Minimization in Server

The data centers are distributed throughout the world to store huge volumes of data that are accessible to thousands of users. A data center consists of a large number of servers that consume much power. Few Million dollars were to be spent on electricity cost that is a rising problem leading to the increased operation cost. The best known mechanisms proposed that grabbed attention was the DCR that focused on energy management by the data centers. Liu et al^.[2] examined the same issue by considering the delay with the network. Fan et. al ^[3] analyses on how much computing equipments can be hosted within a fixed power budget in a safer and an efficient manner.

Data Management

The main aspect of data management is the reliability and effective data positioning. Sathiamoorthy et al. ^[4] proposed a solution based on erasure codes that offered high reliability in comparison with the Reed-Solomon codes. Yazd et al^[5] proposed a scheduling algorithm to improve energy efficiency in data centers considering the data locality properties.

Data Placement

Agarwal et al^[6] gave a data placement approach for the geographically distributed cloud services by considering the bandwidth cost, data center capacity, etc. It analyzes the logs based on the data access types and the client locations.

All the existing works either focus on the task allotment or on the data placement or on the data management. But this paper takes into consideration, the data positioning, the task allotment and the routing of data systematically.

SYSTEM MODEL

The geographically distributed data center topology is shown in Fig. 1. with all the data centers containing the same data are connected via switches. There are a set of data centers(D), and each data center d ϵ D that consists of a set of servers S_d connected to the switch m_d ϵ M having a local transmission cost of C_{l .}

The local transmission cost C_l will be less than the data center transmission cost C_r. Le the whole system be modeled as a Graph denoted by G=(N,E) where,

N is the vertex set that includes all the switches(M) and the servers(S_d)

E is the edge set.

The weight involved with the edges are represented as,

w(u,v)= C_{r ,} if u,v ϵ M

C_l, otherwise

The data stored in geographically distributed data centers are divided into a set of chunks C. Each data chunk c ϵ C has a size and its is normalized to the server storage capacity. For each chunk of data, there will be P copies available in the distributed system for the fault tolerance. λ_c be the average task arrival rate requesting for chunk c.

Fig. 1. Data Center topology

The task arrival in each server is considered as a Poisson Process. If the task is distributed to a data center where the data chunk does not reside, it will take some amount of time till the data chunk gets transferred to that data center. Each task should be replied with a response time of R.

PROBLEM FORMULATION

Data Placement and Task allocation constraints

The binary variable y_sc is used to refer to whether the data chunk c is placed on the server s.

y_sc takes the value 1 if the chunk c is placed in the server s and it takes the value 0 if the

chunk c is not placed in the server s.

In any distributed file system for each data, there are P copies of data chunks stored and the data stored in each server cannot go beyond the storage capacity.

Any server is termed as an activated one(a_s), only if there are data chunks stored onto it or else tasks assigned to it.

Data Loading Constraints

For every data chunk c required by the server s, there are few external or internal data transmissions involved for which a routing procedure is devised.

The Graph containing the servers and the switches is divided into three categories,

1. Source Nodes: These are the servers consisting of the data chunks
2. Relay Nodes : These nodes receive data from the source nodes and forward them to the other nodes based on some routing technique.
3. Destination Nodes: These are the nodes that are receiving the data chunks.

Each and every destination node will receive the data chunks only if does not have a copy of it.

Cost Reduction

The cost involved with the transmission of the data chunks could be minimized by

choosing the parameters such as the y_{sc ,}a_s , λ_c etc.

PERFORMANCE EVALUATION

The performance analysis of the joint optimization approach describes that the communication costs decreased if more tasks and data chunks were placed in the same data center. Further increase in the number of servers will not affect the data chunk distribution among them. Increased requests lead to more activated servers and more computation resources and the joint optimization approach tries to lower the server cost. This approach balances between the server cost and the communication cost. When the delay requirement is very small, many servers are activated to provide quality of service. And the server costs decrease as the delay constraints increases.

CONCLUSION

This paper explains the joint optimization approach of data positioning, task allotment and routing of data to reduce the overall operational cost involved with the data centers that are geographically distributed.

This approach reduced the computational complexity considerably.

REFERENCES

[1] L. Rao, X. Liu, L. Xie, and W. Liu , “Minimizing Electricity Cost: Optimization of Distributed Internet Data Centers in a Multi-Electricity –Market Environment,” in Proceedings of the 29^th International Conference on Computer Communications (INFOCOM).IEEE,2010, pp. 1-9.

[2] Z. Liu, M. Lin, A. Wierman, S.H. Low, and L.L. Andrew, “Greening Geographical Load Balancing ,”in Proceedings of International Conference on Measurement an Modeling of Computer Systems(SIGMETRICS. ACM, 2011,pp.233-244.

[3] X. Fan, W. D. Weber, and L. A. Barroso, “Power Provisioning for a Warehouse-sized Computer,” in Proceedings of the 34^th Annual International Symposium on Computer Architecture (ICA).ACM, 2007, pp.13-23.

[4] M. Sathiamoorthy, M. Asteris, D. Papailiopoulos, A. G. Dimakis, R. Vadali, S. Chen, and D. Borthakur, “Xoring elephants: novel erasure codes for big data,” in Proceedings of the 39^th International Conference on Very Large Data Bases, ser. PVLDB’13. VLDB Endowment, 2013, pp.325-336.

[5] S. A. Yazd, S.Venkatesan, and N. Mittal, “Boosting energy efficiency with mirrored data block replication policy and energy scheduler,” SIGOPS Oper. Syst. Rev., vol.47, no.2, pp.33-40, 2013.

[6] S. Agarwal, J. Dunagan, N. Jain, S. Saroiu, A. Wolman, and H. Bhogan, “Volley: Automated Data Placement for Geo-Distributed Cloud Services,” in the 7^th USENIX Symposium on Networked Systems Design and Implementation (NSDI), 2010,pp.17-32.

[7] S. Govindan, A. Sivasubramaniam, and B. Urgaonkar, “Benefits and Limitations of Tapping Into Stored Energy for Datacenters,” in Proceedings of the 38^th Annual International Symposium on Computer Architecture (ISCA). ACM.,pp.341-352.

[8] P. X. Gao, A. R. Curtis, B. Wong, and S. Keshav, “It’s Not Easy Being Green, “ in Proceedings of the ACM Special Interest Group on Data Communication(SIGCOMM), ACM,2012.pp.211-222.

[9] J. Cohen, B. Dolan, M. Dunlap, J. M. Hellerstein, and C. Welton, “Mad Skills : new analysis practices for big data,” Proc. VLDB Endow. Vol.2, no.2, pp. 1481-1492, 2009.

[10] H. Sachnai, G. Tamir, and T. Tamir, “Minimal cost reconfiguration of data placement in a storage area network, “Theoretical Computer Science, vol. 460.pp.42-53, 2012.