Cluster Computing: History, Applications and Benefits

Abstract

This report will provide a detailed review of the cluster computing. In this report we look at the birth of cluster computing till the present and the future direction the technology is headed. After the literature review, we move on to the explanation of theories involved from the author’s point of view.

The final section of the report covers the current trends and future evolution of the technology as perceived from the author’s point of view. The essence of the report would be a better understanding of the cluster computing and its many uses in today’s world.

Introduction

A computer cluster consists of a set of loosely connected or tightly connected computers that work together so that in many respects they can be viewed as a single system.

The components of a cluster are usually connected to each other through fast local area networks (“LAN”), with each node (computer used as a server) running its own instance of an operating system. Computer clusters emerged as a result of convergence of a number of computing trends including the availability of low cost microprocessors, high speed networks, and software for high performance distributed computing.

Clusters are usually deployed to improve performance and availability over that of a single computer, while typically being much more cost-effective than single computers of comparable speed or availability.[1]

Computer clusters have a wide range of applicability and deployment, ranging from small business clusters with a handful of nodes to some of the fastest supercomputers in the world such as IBM’s Sequoia. [2]

Literature review

In 1967 a paper published by Gene Amdahl of IBM, formally invented the basis of cluster computing as a way of doing parallel work. It is now known as Amdahl’s Law. It is a model for correlation between the expected speedup of parallelized implementations of an algorithm relative to the serial algorithm, assuming the problem size remains the same. [3]

Types of Clusters

Computer clusters are used in many organizations to increase processing time, faster data storing and retrieval time, etc. These computer clusters can be classified in three main types of clusters but these can be mixed to achieve higher performance or reliability.

1. High performance clusters
2. High availability clusters
3. Load Balancing clusters

High Performance Cluster:

High performance computing sometimes refer to as high performance computing are used for computation-intensive applications, rather than handling IO-oriented applications such as web service or databases.[4] examples of HPCs can include the computational simulations of vehicle crashes or weather. Very tightly coupled computer clusters are designed for work that may approach “supercomputing”. The world’s fastest machine in 2011 was the K computer which has a distributed memory, cluster architecture.[5]

High Availability Cluster:

High availability clusters are commonly known as failover clusters. They are used to improve the availability of the cluster approach. In high availability clusters, redundant nodes are used which take over in case of component failure. It is used to eliminate single point of failure by having redundant cluster components.[6] High Availability clusters are often used for critical databases, file sharing on a network, business applications, and customer services such as electronic commerce websites.

Load Balancing Cluster:

Load balancing clusters, as the name suggests are the cluster configurations where the computational workload is shared between the nodes for better overall performance. One of the best examples of load balancing cluster is a web server cluster. It may use a round robin method to assign each new request to a different node for overall increase in performance. [7]

Benefits of Clusters:

There are numerous advantages to using cluster computing. Some of these are detailed below.

Cost:

Cluster technique is cost effective compared to other techniques in terms of the amount of power and processing speed being produced due to the fact that it used off the shelf hardware and software components as compare to the mainframe computers, which use custom build proprietary hardware and software components.

Processing speed:

In a cluster, multiple computers work together to provide unified processing, which in turn provides faster processing.

Flexibility:

In contrast to a mainframe computer, a computer cluster can be upgraded to a higher specification or expanded by adding extra nodes.

Higher availability:

Single component failure is mitigated by redundant machines taking over the processing uninterrupted. This type of redundancy is lacking in mainframe systems.

Cluster Management:

Message passing and communication

The two most often used approaches for cluster communications are PVM and MPI.

PVM stands for parallel virtual machine. It was developed around 1989 by Oak Ridge National Laboratory. It is directly installed on every node and it provides a set of libraries that make the node a “parallel virtual machine”. It provides a run time environment for resource & task management, fault notification and message passing. User programs written in C, C++ or Fortran can use PVM.[8][9]

MPI stands for message passing interface. It emerged in 1990s and supersedes PVM. MPI design is based on various commercially available systems of the time. Its implementation typically uses TCP/IP and socket connection. Currently it’s the most widely used communication system enabling parallel programming in C, Fortran, Python etc.[9][10]

Task scheduling

Task scheduling becomes a challenge, when a large multiuser cluster needs access to huge amounts of data. In a heterogeneous CPU-GPU cluster, mapping tasks onto CPU cores and GPU devices provide quite a challenge because it’s a complex application environment and the performance of each job depends on the abilities of the underlying technologies. Task scheduling is an active area of ongoing research and there have been proposals to build an algorithms which combine and extend MapReduce and Hadoop. [11]

Node failure management

Node failure management is a technique used to handle a failed node in a cluster using strategies such as “fencing”. It isolates the node or a shared resource when it detects a malfunction. There are two types of fencing. First is to disable the node and the second is to prevent access to resources like shared disks. [12]

The first method uses STONITH. Which stands for “Shoot The Other Node In The Head”. This method disables or power off the malfunctioning node. For example, power fencing uses a power controller to turn off the faulty node. [12]

The second method uses the resource fencing approach, which prevents access to resources rather than to turn off the node. For example, fiber channel fencing can be used to disable the fiber channel port. [12]

1137395 Muhammad Khurram Shehzad

Trends

The demand for powerful computers that can be used for simulation and prediction are of great interest to both the public and private sector. Last decade, was the most exciting periods in computer development. As a result of Moore’s law, microprocessors have become smaller, denser, and more powerful. The result is that microprocessor based supercomputing is rapidly becoming the technology of preference in attacking some of the most important problems of science and engineering.

A recent report from Intersect360 Research highlighted some interesting trends in HPC. Below are a few of the highlights. [13]

More Memory with Multi-core:

While memory usage per core is nearly constant in years past, the broader adoption of multi-core systems is creating a demand for more memory. This can be expected but the report also warns of additional system costs as the need for more memory rises.[13]

Processors per Nodes:

According to summary of the report, two-processors per node is still the preferred configuration with 60% of the market, with just 14% opting for four-processor nodes. These ratios have stayed about the same over the past five years.[13]

Future Outlook

High Performance Computing (HPC) is expected to increase with which Big Data is analyzed to address variety of scientific, environmental and social challenges, especially on very large and small scales. In order of magnitude more powerful than laptop computers, HPC processes information using parallel computing, allowing for many simultaneous computations to occur concurrently. These integrated machines are measured in “flops” which stands for “floating point operations per second.” [14] As of June 2013, Tianhe-2 (or, Milky Way-2), a supercomputer developed by China’s National University of Defense Technology, is the world’s fastest system with a performance of 33.86 petaflop/s. [15]

HPC is expected to move into “exascale” capacity by 2020, developing computing capacities 50 times greater than today’s most advanced supercomputers. Exascale feasibility rests on the rise of energy efficient technology: the processing power exists but the energy to run it, and cool it, does not. Currently, the American supercomputer MIRA, [16] while not the fastest, is the most energy efficient thanks to circulating water-chilled air around the processors inside the machine rather than merely using fans.

Applications of the technology

High-performance computing (HPC) is a broad term that in essence represents compute intensive applications that need acceleration. Users of application acceleration systems range from medical imaging, financial trading, oil and gas expiration, to bioscience, data warehousing, data security, and many more. In the information age, the need for acceleration of data processing is growing exponentially and the markets deploying HPC for their applications are growing every day. The HPC expansion is being fueled by the coprocessor, which is fundamental to the future of HPC.

1137784 Samra Mohammad

8Future trends, outlook and applications of cluster computing:

Computer plays an important role in the information age. Different countries have undertaken thorough studies on computing to improve the information level. I may observe some current trends and speculate a bit about the future of parallel programming models. As far as we can foresee today, the future of computing is parallel computing, dictated by physical and technical necessity.

8.1New trends in cluster computing:

These days, there is a new computing paradigm as computer networks called the Grid. It becomes very cheap and very fast .What is a Grid? It is a big system of computing resources that provides to users a single point of access and performs tasks. It is based on the WWW (World Wide Web) interface, to these distributed resources[17]

The Grid technology is currently in progress intensive development. The Grid is also the first tools which are already available for developers. In this type of application, we can use a high-speed network in regarding the interconnection between the parts of the grid via internet.

Nowadays , the Grid is agree to enable for scientific collaborations to share resources on an unprecedented level and geographically distributed groups to collaborate together in a manner that were previously impossible by using scalable, secure, high performance mechanisms for discovering and negotiating access to remote resources.

8.2Future:

In the future, the increase of industry support for low latency clusters will help in availability and performance, but restrictions may require a departure from the current multicast-oriented data distribution strategies.

However, latency and bandwidth performance will continue to improve in the Ethernet with a very low cluster with multicast support.

8.3Application and outlook:

Obviously, cluster computing is quickly becoming the architecture of choice. One of the categories of applications is called Grand Challenge Applications (GCA). It are defined as fundamental problems in science and engineering with broad economic and scientific impact whose solution can be advanced by applying high performance computing and communication technologies. The high scale of complexity in GCAs demands enormous amount of resources needs, such as processing time, memory space and communication bandwidth. A common characteristic of GCAs is that they involve simulations that are computationally intensive. Examples of GCAs are applied fluid dynamics, environmental modeling, ecosystem simulation, biomedical imaging, biomechanics, molecular biology, and computational sciences. [17]

Other than GCAs, cluster computing is also being applied in other applications that demand high availability, scalability and performance. Clusters are being used as replicated storage and backup servers that provide the essential fault tolerance and reliability for critical applications. For example, the internet, search engine, Google uses cluster computing to provide reliable and efficient internet search services.

Conclusion

Cluster computing offers a comparatively cheap, alternative to large server or mainframe computer solutions. New trends in hardware and software technologies are likely to make clusters more promising.

Statement of contribution

Member 1: Muhammad Khurram Shehzad (1137395), Abstract, Introduction, Literature review & Conclusion as a Group and Trends, Future Outlook & applications as individual.

Member 2: Samra Mohammad (1137784), Abstract, Introduction, Literature review & Conclusion as a Group and Trends, Future Outlook & applications as individual.

Member 3: Muhammad Faheem Abbas (1137391), Abstract, Introduction, Literature review & Conclusion as a Group and Trends, Future Outlook & applications as individual.

References:

[1] Bader, David; Robert Pennington (June 1996). “Cluster Computing: Applications”. Georgia Tech College of Computing. Retrieved 2007-07-13.

[2] “Nuclear weapons supercomputer reclaims world speed record for US”. The Telegraph. 18 Jun 2012. Retrieved 18 Jun 2012.

[3] Amdahl, Gene M. (1967).“Validity of the Single Processor Approach to Achieving Large-Scale Computing Capabilities”.AFIPS Conference Proceedings(30): 483–485.doi:10.1145/1465482.1465560

[4] High Performance Computing for Computational Science – VECPAR 2004 by Michel Daydé, Jack Dongarra 2005 ISBN 3-540-25424-2 pages 120-121

[5] M. Yokokawa et al The K Computer, in “International Symposium on Low Power Electronics and Design” (ISLPED) 1-3 Aug. 2011, pages 371-372

[6] Evan Marcus, Hal Stern: Blueprints for High Availability: Designing Resilient Distributed Systems, John Wiley & Sons, ISBN 0-471-35601-8

[7] High Performance Linux Clusters by Joseph D. Sloan 2004 ISBN 0-596-00570-9 page

[8] Distributed services with OpenAFS: for enterprise and education by Franco Milicchio, Wolfgang Alexander Gehrke 2007, ISBN pages 339-341

[9] Grid and Cluster Computing by Prabhu 2008 8120334280 pages 109-112

[10] Gropp, William; Lusk, Ewing; Skjellum, Anthony (1996). “A High-Performance, Portable Implementation of the MPI Message Passing Interface”. Parallel Computing. CiteSeerX: 10.1.1.102.9485

[11] K. Shirahata, et al Hybrid Map Task Scheduling for GPU-Based Heterogeneous Clusters in: Cloud Computing Technology and Science (CloudCom), 2010 Nov. 30 2010-Dec. 3 2010 pages 733 – 740 ISBN 978-1-4244-9405-7

[12] Alan Robertson Resource fencing using STONITH. IBM Linux Research Center, 2010

[13] http://www.intersect360.com/industry/reports.php?id=67 (Accessed 12/05/2014)

[14] http://ec.europa.eu/digital-agenda/futurium/en/content/future-high-performance-computing-supercomputers-rescue (Accessed 12/05/2014)

[15] http://www.top500.org/system/177999#.U3ORpPmSzDs (Accessed 12/05/2014)

[16] http://en.wikipedia.org/wiki/IBM_Mira (Accessed 12/05/2014)

[17] http://www.slideshare.net/shivakrishnashekar/computer-cluster (Accessed 14th May 2014)