Analyzing of Economic Data Using Big Data

• N.Rajanikumar, Dr.A.Suresh babu, Mr.G.Murali

Abstract: Big data can help at the e commerce data. The big-picture problems, the economic indicator many investors, business fortunate and judges are rely on are just too outdated by the time they’re out. People “pitch to the number,” but the world has often moved since it was considered and they won’t know it until the next report comes out. Take, for example, the case of increasing food prices in India and China that are pouring up price rises for a major percentage of the world’s residents. But principle claims to have been seeing the movement shaping up for weeks. Premise is able to capture economic data in close to real-time in some cases or at least much closer to it in others thanks to the technology trifecta of e-commerce, cloud computing and Smartphone’s. However, while e-commerce data is supportive for gauging the prices of certain goods in certain economies, it doesn’t really touch emerging economies where the vast popular of transactions are still local and cash-based. If groceries prices are rising across Asia, for example, that likely income, along with other things, inferior health and less money to spend on non-essential end user goods. That’s where mobile devices come into play in the form of Premise’s Android host. The company has more than 700 contributors in 25 cities, mostly in Asia and Latin America, who go into stores and markets and capture data about exact items on which Premise desires data. “We use them as sort of detection agents. The contributors take a picture of the item either on the shelf or in a market stall; it syncs with Premise’s servers in the blur; and Premise’s system is then able to extract information from the photos. It can verify information such as price, brand and quality of the items, and even ecological information such as how clean the store is and how stocked the shelves are. Interestingly, but not without warning, the app that contributors use is only for Android phones.

Keywords: Apache Hadoop API Using HDFS, Mapreduce, Pig, Hive, Linux-Unix, windows,Eclips.

1. INTRODUCTION

This paper mainly focuses on how to manage huge amount of data and how to analyse the data. The technology used for this is hadoop technology . In this project the data taken is Economic data from various E-commerce websites. Then the data is stored into HDFS( hadoop distributed file system) format in the form of clusters. After the storage is done, then the processing of data can be done based on the user requirements. The processing of data can be done using many modes. Hadoop basically contains many ecosystems which provide different ways of processing or analyzing the data in different environments. There are two basic methods of Hadoop are HDFS and MapReduce. HDFS is used to stock up the data and MapReduce is used to progression the data. In MapReduce we write codes in java to analyze the data in whatever way we want to. The ecosystems in Hadoop are also for processing and analyzing the data. The different ecosystems of hadoop are pig, hive, chukwa, HBase, ZooKeeper, sqoop etc. Here pig, hive and sqoop have been implemented. So the first ecosystem implemented is pig. Pig is scripting language. It can process both structured and unstructured data. In this pig scripts are written on the data to get results. Then hive is a query language, it can handle only structured data. In this queries are written on to the data to analyze it. Then finally sqoop, it is actually a support for hadoop rather than an ecosystem. It is used to transfer data from one data base to other. And after the processing of data the results are displayed.

2. What Is Big Data?

Big Data refers to the data sets whose size makes it difficult for commonly used data capturing software tools to interpret, manage, and process them within a reasonable time frame. Big data sizes are a continually moving target, as of 2012 ranging from a few dozen TERABYTES to many PETABYTES of data in a single data set. With this difficulty, new platforms of “big data” tools are being developed to handle various aspects of big quantities of data.

BIG DATA concept means a datasets which continues to grow so much it difficult to manage it using existing database management concept and tools. The difficulty can be related to retrieve the capturing of data, storage, searching and virtualization, etc.

The challenges associated with Big Data are the “4 V’s”:

Volume, velocity, Variety, and value.

The Volume challenges exist because most businesses generate much more then what their system were designed to handle.

The velocity challenge exists if company’s data analysis or data storage runs slower than its data generation.

The variety challenge exists because of the need to process difference types of data to produce the desired insights.

The value challenge applies to deriving valuable insights from data, which is the most important of all V’s in my view.

Fig1. 4V’s of Big Data

3. What is E-Commerce?

A type of trade model, or part of a larger business model, that enables a firm or individual to perform business over an electronic network, typically the internet. Electronic commerce operates in all four of the major market segments: business to business, business to consumer, consumer to consumer and consumer to business. It can be thought of as a more advanced form of mail-order purchasing through a catalogue. Almost any product or service can be offered via ecommerce, from books and music to financial services and plane tickets. Investopedia explains ‘Electronic Commerce: e-commerce’

E-commerce has approved firms to set up a market existence, or to improve an active market spot, by providing a cheaper and more capable distribution chain for their products or services.

4. Why Big Data is a must in ecommerce

The buzz nearby Big Data is far away from being needless. Not only does it permit merchants to gain deeper insights into customer behavior and industry trends, but it also lets them make more precise decisions to improve just about every feature of the business, from selling and publicity, to merchandising, operations, and even customer maintenance.

Below are a few more points that deeper explain the impacts of Big data in the Ecommerce empire. From improving customer familiarity to developing better products or marketing campaigns, it’s no question that Big Data is the next big thing for online businesses.

5. Characteristics of Big Data

A Big data proposal can give a solution which is planned specifically with the needs of the venture.

The following are the basic characters of the Big data:

• Comprehensive – It should offer a broad platform, and address all three dimensions velocity, volume and variety.
• Enterprise Ready – It should include the performance, reliability, performance and security features.
• Integrated – It should enable integration with information supply chain including databases, data warehouses and business intelligence applications.
• Open Source Based — It should be open source technology with enterprise class functionality.
• Low latency.
• Robust and reliability.
• Scalability.
• Extensibility.
• Allows adhoc queries.
• Minimal Maintenance.

6. BIG DATA OFFERS

There are many vendors offering BIG DATA Analytics are IBM, KOGNITO, etc. Here in this paper I have discussed about the IBM Platform.

Fig -2: IBM Platform of BIG DATA

7. Big Data Challenges

There are focal challenges of BIG DATA are data variety, velocity, volume and analytical workload intricacy

More number of organizations is belligerent to compact with many problems with the large amount of data. In order to solve this problem, the organizations need to ease the amount of data being stored and develop new storage techniques which can improve storage use.

8. Uses of Big Data for Online Retailers

Most minute merchants’ think that Big Data analysis is for well-built companies. In fact, it is essential for minute businesses, too, as they attempt to partake with the larger ones. This becomes even more important as online retailers proceed together with their customers in real time. Note, however, that management large sets of data can increase a site’s load time. A slow site troubles every aspect of the shopping procedure.

Here are six uses of Big Data for online retailers.

Personalization, Dynamic pricing, Customer service, Managing fraud, Supply chain visibility,Predictive analytics.

‘Big Data’ and e-commerce

Tuesday 25 September 2012

9. Conclusion

The expansion of information particularly of unstructured dataposes a special challengeas the volumeand diversity ofdata. One of the most promise technologies is the Apache Hadoop and Map Reduce structure for dealing with this big data problem.

Big Data is a popular trend in business and in marketing. The concept can indicate different things to different businesses. For ecommerce, retailers should seek to use Big Data to collect big information, if you will, that may be used to make better marketing decisions,.

10. REFERENCES

[1] Ecommerce.about.com

[2] bloomreach.com/2012/05/ecommerce-challenges-that-can-be-solved-by-hadoop-and-big-data-apps/

[2] Ziff Davis, “E-Commerce.” Software World, 2003, vol. 30,pp. 207-212.

[3] X. J. Tong, W. Jiang, “Research of Secure System of Electronic Commerce Based on Mix Encryption,” Microprocessors, 2006, vol. 4, pp. 44-47.

[4] S. H. Qing, Cryptography and Computer Network Security. Beijing: Tsinghua University Press, 2001.

[5] Y. P. Hu, Y. Q. Zhang, Symmetric Cryptography. Beijing: Machinery Industry Press, 2002.

[6] S. Z. Guan. Public Key Infrastructure PKI and Certification Authority. Beijing: Publishing House of Electronics Industry, 2002.