Shopping Assistance And Personalized Advertising Computer Science Essay

Published: Last Edited:

This essay has been submitted by a student. This is not an example of the work written by our professional essay writers.

This paper presents an in-store e-commerce system that provides shopping assistance and personalized advertising through the use of a new concept in context aware computing, dynamic contextualization. This system, Promo Pad, utilizes augmented reality technologies on a hand-held Tablet PC to provide for dynamic modification of the contextual settings of products on store shelves through the use of see-through vision with augmentations. Augmented reality enhances the human perception of reality in this application by contextualizing individual objects that are encountered in the real world with virtual complements so as to make the real objects more meaningful and appealing. This real-time modification of the perception of context, dynamic contextualization, moves beyond the traditional concept of context-aware computing into context modification. The technical requirements for realizing dynamic contextualization using augmented reality technologies are described in detail. The target design of the Promo Pad is a consumer friendly shopping assistant that requires minimum user effort and is practical in a public environment such as a shopping mall or a grocery store. This paper will also describe the technical implementation of the video see-through augmentation system and how this technology makes possible the concept of dynamic contextualization, the modification of context to direct the interest flow of users.

Keywords: Augmented reality, dynamic contextualization, context-aware, pervasive computing, e-commerce


This paper describes the PromoPad design, a computer device and a e-commerce system that performs shopping assistance and a personalised in store advertising. The PromoPad is a handheld prototype device that provides context sensitive shopping assistance. This assistance will be in the form of augmented imagery which uses the augmented reality technology and the content of this is built around the concept of dynamic contextualisation. Dynamic contextualization is nothing but an extension of the conventional advertising concept of contextualization to include real-time modification of perceived context which is based on the interaction among the user, the environment and focal objects using augmented reality technologies. Augmented reality(AR) differs from virtual reality(VR) in such a way that there is no attempt to replace the real world so the users can still interact with the real world and at the same time they can perceive enhanced views with augmentations. Augmented reality will enhance the human perception of reality in this application by contextualizing the individual objects which may encounter in the real world with virtual complements, so as to make the real objects more meaningful and appealing. The PromoPad is a tablet PC with a camera mounted on the back. The display captured tablet PC provides a modified version of the camera image. This image can be modified by using augmented reality techniques that add new imagery relative to a focal product or remove elements of the image that may distract from the focal product. Thus augmented reality techniques offer the capabilities necessary for realizing dynamic contextualization. In traditional context-aware computing, context is static and represents the situation of the user, providing only input to the computing system. But in dynamic contextualization, the context can be both input as well as output and is modified to be more meaningful for the focal objects and creates more interest to the users. Pervasive retail system was developed by Kourouthanassis and Roussos developed MyGrocer that can manage the shopping lists, also monitors the total cost of cart contents, navigate consumers and popup promotion information within the store.

The main aim of a good e-commerce system is not only to provide passive information but also be able to provide trigger impulse purchase decision. Dynamic contextualization can be made possible by augmented reality (AR) techniques that modify the perception of the real world in real time. Several studies on augmented reality provide evidences that tells us augmented reality technologies can improve the human interaction performance. In archaeology, an outdoor augmented reality technique called archeoguide system used to provide personalized tours of archaeological sites and also to improve the information presentation, simulation of ancient environment, and the recovery of destroyed sites. The research in augmented reality explores the technical feasibility and benefits for advertising and consumer experiences.

The remainder of this paper is organized in the following way. In section 2 the concept of Dynamic contextualization will be introduced and is provided with the theoretical basis for its application. Section 3 will describe the concept of PromoPad system and the technical details that make the augmented shelf view work. Section 4 discusses the implementation of dynamic contextualization on the PromoPad. Section 5 the results of this paper are summarised and discusses the future issues of research.

2. Dynamic Contextualization Overview:

From the perspective of human psychology, dynamic contextualization with the PromoPad using augmented reality technology can simulate an enhanced product experience. "Enhanced", here implies a combination of both direct experience and virtual experience. Traditionally, product experiences can be categorised as direct or indirect. Direct product experience is a direct interaction between consumers and products in a full sensory capacity which includes visual, auditory, taste and smell, haptic and orienting. Indirect product experience is the experience gained through secondary sources such as advertising. When the direct and indirect product experiences are compared, the direct product is much richer for several reasons. Direct experience product is considered as most trustworthy because the direct method the information can be self-generated by the consumer and also the direct examination of the shopper is allowed to feel and touch a product and get input from multiple sensory channels. The shopper can inspect a product in a sequence and pace of their choice and can customize the information depends on their needs. The consumer can examine the product, but not necessarily remove it from packaging and rarely place it into its intended context. Inorder to overcome these disadvantages, virtual experiences as simulated in 3-D visualization.

The real beauty of augmented reality (AR) is that it enables a shopper to inspect a product personally and at the same time they can also view some of the additional objects in 3-D visualization on the Tablet PC display of the PromoPad. The objects found in 3-D visualization are able to generate a new form of mediated experience - virtual experience. The virtual experience is one form of indirect product experience, because both are mediated experience. But the virtual experience seems to be richer than indirect experience rendered by printed ads, television commercials, or even 2-D images on the Web etc. Li, Daugherty, and Biocca indicate that virtual experience, as simulated in 3-D visualization, consists of more active cognitive and affective activities than 2-D marketing messages . They attribute to the psychological and emotional effects to the interface properties of 3-D advertising, as well as to the psychological sensation of presence.

3. The PromoPad system:

The PromoPad is a mediated device that provides in-store virtual experience with 3-D product visualization. The system consists of two server components, the front-end client component and back-end server component in which the front-end component is a light-weight display device that slips into a cradle in the shopping cart. This can implement by using Tablet PC technology in which a camera is attached to the back of the Tablet PC, the client device is aware of the position and orientation of the shopper relative to the shopping cart and store shelves. It is also capable of providing the shopper a see-through view of the shelves and also the additional information that is related to the items in the view by the shoppers. The back-end components mainly consist of one or more servers that contain inventory data, customer profiles and business logic, from which information in the databases is filtered and returned to the front-end component. The PromoPad employs augmented reality technologies and passes an augmented camera image from the rear of the Tablet PC to the display.

The main goal of using this technology is that it is an intelligent shopping aid which provides the shoppers automatic and meaningful help when needed and also to minimize human interference and effort. With wireless communication technology, a Tablet PC can have different modes for shoppers in different shopping orientations. Those entering the store with specific purchase in mind (the planned shoppers), may use a tablet PC to optimize their shopping routes in a store to quickly find items they plan to buy. Those seeking sales and clearance items (bargain shoppers) may use the tablet to find sale items that they are interested in with ease. Those entering the store simply to browse (recreational shoppers) can use a Tablet PC to obtain product information that is not on the packaging. Let's consider an example; an augmented display of a bottle of wine might include images of the winery and/or wine ratings or reviews. By using the Tablet PC the Content customization and personalization can be greatly facilitate the convenience of all types of shoppers by enhancing them with the shopping experience. The most important thing is recognize that the vast majority of grocery and convenience store purchases are impulse purchases. Slight improvement in marketing performance can even be resulted in massive increase in sales.

To achieve this goal of the Tablet PC as a see-though augmentation device, several technical issues have to be mentioned. First, the ways of tracking the location context needs to be robust, stable and scalable. Second, the real image in the Tablet PC display should be adjusted in such a way that it should offer a true see-through view as if the Tablet PC display was transparent so that the device is well integrated with the environment and also in harmony with real products. Third, the real image should be accurately registered with the virtual object. Finally, the system should be able to deal with different virtual and real composition methods which can include overlays, occlusion, and diminishment.

3.1 In-store tracking:

The location of a shopper as a 3-D position and orientation relative to product and store shelves is acquired by an in-store tracking system. When the shopper is using the PromoPad, it is reasonable to assume that the position and the orientation of the Table PC are a good approximation of the position and orientation of the shopper. Some of the existing technologies can be scaled for this application to store-size volumes with large quantities of PromoPads. Location information required for the PromoPad is considerably more rigorous than that required by traditional context-aware computing systems. Dynamic contextualization and augmented reality require modification of the camera image. In order to achieve pixel-resolution registration of virtual elements with store shelf contents, the PromoPad system requires high-accuracy knowledge of the location and orientation of the client device.

3.2 Video see-through systems:

The view as seen on the Tablet PC display is derived from the image captured by a camera mounted on the rear of the tablet. The characteristics of the camera view are, in turn, determined by the camera intrinsic and extrinsic parameters. The intrinsic parameters of a camera describe how the camera will convert objects within the camera's field of view into an image. The extrinsic parameters describe the position and orientation of the camera in space. Figure 1 illustrates a perspective camera projection model. The optical axis, which is orthogonal to the retinal plane ℜ, passes through the center of projection C and intersects with at the principal point c on the image plane. The distance between the center of projection C and the retinal plane ℜ is the camera focal length f. Let M denote the world coordinate of some point on the tip of the wine bottle. The corresponding point m on the retinal plane is the intersection of the line that passes through M and C and the retinal planeℜ. Thus, intuitively, what the camera can see is the volume inside the infinite pyramid whose apex is C and the four lines that form the edges of the pyramid pass through the four corners of the retinal plane, as illustrated in Figure 1.

Figure 1 Perspective camera projection model

This pyramid is referred to as the graphics frustum. Graphics frustums for rendering are often truncated with near and far clipping planes, where the near clipping plane avoids rendering of objects too close to the camera or at the singularity point C and the far clipping plane avoids rendering objects so far away from the camera as to be considered no longer visible. The near and far clipping planes also serve a practical function of limiting the required precision of depth buffer values. The graphics frustum for the virtual elements must match the camera viewpoint and intrinsic parameters in order to mimic the view of the camera and have the virtual objects accurately registered with the camera image. The far and near clipping planes must clear any rendered virtual elements.

3.2.1 Registration:

Assuming a calibrated camera, a reasonable assumption as camera calibration is a common process, the camera focal length f, the center of projection C, the principal point c and the size and position of retinal plane ℜ are known to the system. Before setting up the viewing frustum, the viewport rectangle needs to be set as the same resolution (viewport rectangle size in pixels) as that of the camera retinal plane in order to have the same view of the camera if the frustum is set according to the camera's intrinsic parameters. For example, if the camera resolution is 640 by 480 pixels, then the size of the viewport rectangle needs to be set as 640 by 480 as well. The parameters defining a viewing frustum that match the camera point of view are (l, b, -n), which specifies the 3-D coordinates of the lower left corner of the near clipping plane, and (r, t, -n), which specifies the upper right corner of the near clipping plane . The values of l, r, t, and b are as follows.

l = -n.C/f;

r = n(w-C)/f;

t = -n.c/f;

b= n(h-c)/f;

(C,c)is the 2-D coordinate of principal point c in the retinal plane ℜ, w and h are the width and height of the retinal plane ℜ, f is the camera focal length. These parameters much be measured in the same unit, usually pixels. n is the distance from the camera to the near clipping plane and is of the unit as l, r, t, and b. The virtual objects that are rendered in this frustum are well aligned with the camera image.

3.2.2 Zooming:

Figure 2 shows the different effects of the Tablet PC window when the display is from the camera's point of view and adjusted to the user's point of view. A cereal box is behind the Tablet PC and in the scene of the camera. The PromoPad captures the cereal box and augments the view with a nutrition bar and a piece of advertising information. Figure 2a shows the augmented view from the camera's viewpoint. The adjusted view is shown in Figure 2b with the effect of a 'magic frame'.

a. display from the camera's point of view b. display zoomed to user's point of view

Figure 2 Tablet PC displays from different viewpoints

4. Dynamic contextualization in the PromoPad:

With dynamic contextualization, the administrators of the system can control the interest flow of the users by virtually modifying the focal entity's context information. In a shopping environment, dynamic contextualization is a business strategy that the retailers can use to virtually change the product settings placed objects in more complementary settings or removing competing or uncomplimentary products that may be in close proximity to the focal product.

4.1 Dynamic contextualization with augmented reality:

Product contextualization is the placement of a product in a particular setting that will resonate with consumers and make clear the product consumption practices or situations. Product contextualization is often seen in store displays and advertising. In electronic commerce, product contextualization can be simulated with 3-D visualization, which can offer a variety of ways for the consumer to arrange a focal product with other complimentary products on the computer screen. Researchers use this virtual contextualization to place complimentary products with a focal product in 3-D visualization in order to affect the user's perception of the focal product. For example, the user can arrange a set of furniture in different settings in 3-D on a website to select the preferable combination. Research has demonstrated that virtual contextualization can lead to better consumer experience, brand attitude, and hence influence purchase intention.

5. Summary:

This paper presents the concept of a shopping assistant that utilizes augmented reality technologies to provide personalized advertising and in-store shopping assistance based on dynamic contextualization. This PromoPad system is a step towards pervasive and ubiquitous computing in the highly lucrative grocery shopping segment. The development goal is to offer a pleasant and inviting shopping experience that is mediated by an augmented reality-based Tablet PC. The paper describes the technical implementation of the video see-through augmentation system and how this technology makes possible the concept of dynamic contextualization, the modification of context to direct the interest flow of users. Dynamic contextualization, the real-time modification of context, can be enabled by augmented reality technologies with augmentations and diminishments of the perceived visual context. Dynamic contextualization is based on, but extends beyond, the spatial and temporal context of the user. Location context, user context, and product context are integrated in this design to address the requirements of an intelligent context-aware shopping assistant.

The concept of dynamic contextualization and the design methodology of the PromoPad system can be extended to other circumstances such as tourism guides, training assistants, etc. Nevertheless, designers of other systems need to carefully consider the context factors based on the requirements of an application domain.

Although this article has addressed several important issues in designing the PromoPad, there are still some important issues that must be addressed in the future development of PromoPad as an e-commerce system.

User privacy has to be protected. The privacy issue arises when the retailers collect the consumption activities and attempt to predict the consumer's interest based on her previous shopping behaviour. It is necessary to balance the trade-off between automation and privacy to meet the needs of both retailers and consumers. Consumers may be willing to sacrifice certain degree of their privacy in return for certain perceived value, and retailers definitely should respect the privacy of their customers. A lot of on-going research works are concerned of privacy and security of online e-commerce systems. The goal of the initial development of PromoPad, however, is to maximize the automation of the shopping experience and explore the potential of dynamic contextualization and the possible applications of augmentations and diminishments in the perception of a short shelf image. Hence the privacy issue is beyond of the scope of this initial work and this article.

Another dilemma in this design is the trade-off between user flexibility and automation. Maximizing the automation requires little user effort but limits the user's flexibility at the same time. Although the design has been deliberated in tailoring the information flow to fit individual needs, some advanced user would like to have more control over the augmentations and diminishments in the PromoPad. This dilemma can be arbitrated by analysing a survey of usability questionnaire to samples of consumers with various education backgrounds. Also, options giving consumer-appropriate control over the PromoPad can be provided for advanced users to balance the trade-off.