Mobile Interaction As Human Computer Interaction Computer Science Essay

Published: Last Edited:

This essay has been submitted by a student. This is not an example of the work written by our professional essay writers.

In this paper the term Human Computer Interaction is defined well other than that the Mobile Computer Interaction is also defined here. It is mentioned that with the modern technology such as sensing technology mobile interaction can improve in the scenarios like situational context. And it describes not only the current situation but also the predictable future. The research is about the mobile devices such as mobile phones, PDA. Those mobiles have specific restricted input and output requirements. The challenges of mobile devices are limited screen size, limited memory, use one screen at a time, interact with one application at a time, design for a widespread population and designing for limited input/output facilities.


The research is about the mobile interaction as a part of the human computer interaction. The research is about the mobile devices such as mobile phones, PDA. Those mobiles have specific restricted input and output requirements. The challenges of mobile devices are limited screen size, limited memory, use one screen at a time, interact with one application at a time, design for a widespread population and designing with limited input/output facilities. While discussing about those areas, the study area can be divide in to 3 parts as past, present and future of mobile interaction.

The mobile interaction starts after mobile devices came to lot of people's hand. It happened basically around 1990. On those days mobiles are only used for the purpose of talking someone far away. So the interaction was minimized [2].

Nowadays the technology is so much developed and most of features are added to the mobile devices and became something essential for everyone. Therefore the interaction with the phone must be in a comfortable way. For fulfill that compatible OS, compatible web pages, utility application (performs a simple task with minimum user input) are used [1].

New trends of the mobile technology and the interaction are stereo audio input, high resolution color screens for PDA, different themes, vibration, reminders, To do lists privacy settings (when you are not doing anything switch off the backlight) can be mentioned. In the future there can be weird and wonderful features with mobile devices that people even can't think about. To predict about those things researches, conferences must be done. [8]

The rest of the paper includes more information about Mobile Interaction. Section 2 provides an overview of it. Section 3 has details on the major research areas. Past and present mobile interaction is in section4. Section 5 contains future direction. Section 6&7 contain my contribution and the discussion. At last in section 8, "My contribution" explains classifications, identifications, integrations, proposals, etc.

Overview - Mobile Interaction

This section provides an overview of Mobile Interaction including its importance, key researches, applications and advancements.


Mobile devices are something which is very much familiar to all most everyone in the society because nowadays kids also know how to work with them. Mobile phones become one of the most essential devices for the people in day today life. As an example people use missed calls as a way of passing messages. Without saying a single word and without typing anything people communicate with each other. So the interaction between the human and the mobile phones, PDA, iPhones, mobile music players are increasing rapidly.

Mobile devices are becoming popular due to the portability nature. People use these mobile devices to manage personal information, do simple work with poor processing requirement. In general mobile devices are come into the domains of education, business, military, etc.

key researches

The topic mobile interaction is not a new thing to the world because it is talked all around the world even in the conferences. Examples for those conferences are MobileHCI, PERMID and WIMP.[7] These conferences are held in each year and discuss about the HCI related HCI because it is really important to the world. It is changing say by day. So there are many researches going on related to this area. The main topics related to mobile interaction are [5],

Interactions between mobile devices and the real world

Using mobile devices as user interfaces for terminals and vending machines

Usage of sensors of mobile devices (camera, microphone, GPS, etc.) for pervasive applications

Multimodal interaction taking mobile devices into account

Interactive context-aware services on mobile devices

User experience, user studies

Applications and scenarios


Generally mobile phones are used to make calls among the people who are in far away but nowadays the reason of using mobile phones is different and very broad. As example mobiles are use to text messages, access internet, listen to music and use as camera. So there must be more advanced applications in the mobile devices.

Same as the normal computers mobile devices have operating system to operate. It determines the functions, features of the mobile device. It clearly describes the thumbwheel, keyboards, wireless security, synchronization with applications, e-mail, text messaging and more. The most commonly used mobile operating systems are shown below: [4]

Symbian OS - For smartphones this is the standard operating system. This is suitable for the specific requirements of 2.5G and 3G mobile phones

Windows Mobile  - Windows Mobile software available on Motorola, HP, Dell, Palm and i-mate products.

Palm OS - This platform has essential business tools, ability to access Internet

User interfaces of mobile phones also play a major role in this area. User interfaces are also becoming more complicated and advanced.

C:\Users\Heshani\Desktop\hesh-reserch\Mobile with HCI\iPhone-WVGA-theme-for-Window-Mobile-phones.jpg

Figure 1- User interface of a mobile phone

Figure 1 shows one of the user interfaces that the latest mobile phones have.

Nowadays lot of people use mobiles to access web. Thus other than the basic operating system, the other important thing in mobile user interfaces is mobile browser. Examples for mobile browsers are

jB5 Browser

Polaris Browser


Internet Explorer Mobile


Mobile is more than just calling someone. It developed since 1980. After few generations they come to this status. Mobile devices had/ have so many limitations.

small-sized display with poor resolution, fewer colors, and different width/height ratio from the normal 4:3

limited CPU processing and memory capacities

slow connection with fluctuated bandwidth

non-user-friendly input facilities (ordinarily used keyboard and handwriting demanding lots of screen space, resulting in quite inaccurate results)

Currently and in future those limitations will be solved and many improvements can be take place. Due to the large differences, the classic desktop solutions have limited applicability to mobile user interface design. So as mentioned in the applications different types of applications are needed. Modern mobile phones support many additional services, and accessories, such as SMS (or text) messages, email, Internet access, gaming, Bluetooth, infrared, camera, MMS messaging, MP3 player, radio and GPS.

Researches in Mobile Interaction

This area may have so many sub research areas because it is very important to everyone. And also there are so many things that people want to add and improve in the mobile interaction. Depending on that this research paper pays attention on following areas;

Interactions between mobile devices and the real world

When considering this area interaction it is basically goes with user and the computer (mobile device). First user create plan of action and execute it at the computer interface. Then the device processes the request and displays the result. Finally the user can evaluate the result.

The main parts of the device that affect the interaction are

input devices - text entry and pointing

output devices - screen

physical interaction - e.g. sound, haptics, bio-sensing

memory -capacity & access

processing - speed of processing, networks

When these features are highly connected to each other then the there is a rich interaction.

Keypad - This is the main input device. Not like the way keys in a normal keyboard of a computer in a mobile phone both numbers and letters are in the same keys.

2 - a b c 6 - m n o

3 - d e f 7 - p q r s

4 - g h i 8 - t u v

5 - j k l 9 - w x y z

But nowadays there are phones which have numbers and letters separately so that users can easily type their messages and dial numbers. Other than these keys there are another keys to select, go back, to clear, go up & down. This description is about normal mobile phones. In the other devices like iphone there is no keypad instead, a virtual keyboard and keypad are displayed on the touch screen when required. The technique that is using in the PDA's is handwriting recognition. That means Text can be input into the computer, using a pen. But there are few problems with that natural interaction because different people have different handwriting and it is difficult to understand each letter separately. IPod devices and the cameras also have the relevant type of keypad. [1]

Figure 2- Keypad of a mobile phone

Figure 2 shows one of the keypads used in the mobile phones.

Screen - This is the main output device of a mobile device. Screen provides everything that the users want from the mobile device. When considering about the normal mobile phone it shows the missed calls, received messages, screen server, various themes, colors, various lights and pictures. That is because users must be attracted to the mobile phone. If the output gives them more than expectance then the user's attitude among the phone are growing better. It directly affects the interaction between the human and the device. Screen is the factor of deciding the quality of the device.

In PDAs, iPhones there's no keypad instead touch screen is available. Just with a touch of a finger you can connect to the world. The touch screen is used by smartphones, multimedia and music phones.

Figure 3- Screen of a mobile phone

Sound - Sound is also used in the mobile devices to interact with the people. Simple example is when a call come to a particular phone it rings. So the user can easily identify that he got a call. Other than as ringing tone sounds are message tone, alarm tone, play music, keypad tones, power on/off tones, error tone, connect tone, etc. In iPods and other music players sound plays a major role because they nothing without proper sound system. In a camera also there are situation where people need simple sounds to recognize things. Al these sounds are used to help the user to easily handle the device. [2]

Human - Human wants the output from the input he provides. The human interaction is differing from person to person. And it has a big connection with the emotions, reactions of the user. Different people have different interpretations, vision, hearing, memory, movements and needs when they interact with a particular device. So the mobile interaction changes with the personal choices. Some may like to use the phones with touch screen and some may take it as a mess. Therefore the people who develop software, UI must consider about these differences if they really consider about the better human interaction with the mobile devices.

Mobile Device User Interface designing

Although the mobile devices are also computers design of user interfaces for the mobile device is totally different from the user interfaces in the normal computers. Following are the things that should be consider in designing user interfaces for mobile devices: [3]

Designing for mobility -

Users are mobile so they will not have many things to support the work. So there must be a way to handle that work with small devices.

Designing for widespread population -

Users will not normally have any formal training in their technologies and consider them as devices to be used rather than computers to be maintained.

Designing for limited input/output facilities -

The main limitations are screen size, user interface, processor speed, memory constraints, limited bandwidth and low battery life. Screen sizes will improve in resolution in terms of color support and pixels per cm, but will always be small due to the need for portability. Sound output quality is often very poor with restricted voice recognition on input. Keyboards are limited in size and number of keys and other pointing devices are often hard to use when on the move.

Designing for (incomplete and varying) context information -

Through various sensors and networks, mobile devices can be made aware of their context (e.g. current location through the Global Positioning System). This gives new information to the systems but brings problems of implying task and user level activities from sensor information and unreliable or patchy sensor coverage. Work on position aware tourism guides, for example, highlight many of these problems.

Designing for multiple users

Not like the normal desktop users the mobile device users may not having proper knowledge about how the device works because every levels of people use mobile phones and other devices. Therefore the user interface designers must pay more attention about designing user friendly UI for the people who are not familiar with computers.

The major drawbacks or the limitations of mobile user interface design are as follows,[11]

Not visible under sunlight

Cannot give inputs through embedded keyboard,

Letter recognizer or transcriber

Suitable for certain input methods. People cannot

Give inputs while wearing gloves

Difficult to type letters than a normal keyboard

Mouse is not used

Interactive context-aware services on mobile devices

This is basically new idea that goes with the mobile devices. This system minimizes the usability problem of the users by providing only the relevant information to the user. This avoids the manual typing of the user. The different people want different things from a particular mobile device. So the requirements are differs. That means the context must be organized according to that context. So the mobile devices must be capable of contain the user circumstances and surrounding environment. That means user find the results with less effort.

In this case context can be defined in many ways. That means if the user want different phone ringing ways depending on the situation. That facility is available in mobile phones. Followings are the examples for that.

Hand - When the user holds the phone in her/his hand, the audio alarm is not needed. The phone rings by vibrating

Table - Here we assume a meeting situation. The phone is almost silent. Incoming call is indicated by a very gentle sound

Silent - Phone is silent. Here we assume that the phone is put a way in a box or suitcase, and must be silent. The phone still receives calls, so that the callers' numbers can be recalled later

Outside - Here the ring volume goes as high as possible and vibra-alarm is on. All possible ways to get the users attention are used

General - General mode is used when none of the above apply

There are devices which are able to act and react according to the situational context in which they are used. The way people interact with each other and also the situation in which they interaction carries information that is directly describes the way they want to interact with the computers. [1] Therefore the human to human interaction is considered in context based mobile interaction. Examples for contextual information are gestures, body language, and voice. e.g. in a discussion with a student pointing at a computer the term 'sun' has a different meaning than the same term when on the beach together with friends.

In this scenario the mobile device should understand the meaning or the containing idea of action performed by the particular user. It is not necessarily a direct command to the device. Interaction is based on the assumption that the computer has a certain understanding of our behavior in the given situation. This knowledge is then considered as an additional input to the computer while doing a task.

A simple example is the garbage bin [1] that scans in the bar code of products and reproduces the information for a suggested shopping list. The action performed by the user is the same as with any other garbage bin. The recognition of the system (by scanning the bar code) and the built-in interpretation of the system (all things that go into the bin may be on the next shopping list again) make use of the action performed by the user. The user herself does not explicitly interact with the computer.

This research basically goes with the following building blocks;

The ability to have perception of the use, the environment, and the circumstances

Mechanisms to understand what the sensors see, hear and feel

Applications that can make use of this information.

Figure 4- Context- aware mobile interactionThis is about the situational context and the applications that context enabled.

3.4 Interactive mobile web search

Context aware services also important when considering about accessing of web. Most of the mobile phone users are try to download ring in tones, wall papers, screen savers and pictures from internet through their mobile phone. So the mobile user interfaces must facilitate that. It must be considered when designing UI for mobile devices. That is to limit the problems that can occur because of the small physical size of the mobile device. The other problems are limited resolution screens and limited input/ output faciities. Therefore to overcome all these problems there must be suitable solution for mobile devices. And thus there are mobile browsers such as jB5 Browser, Polaris Browser, Blazer, Internet Explorer Mobile and many more.

Designing web browser for a mobile phone is very difficult due to the differences that are available within the phones. [10] That means the UI design that is done in one kind of phone may not be suitable for another phone.

Accessing web through a mobile device differs from a computer because the keyboards are numeric and cramped. Connections can be unreliable, with low bandwidth and high latency. Most importantly, screens are small and often monochrome. Wireless markup languge is widely used here to solve this problem.

An example for mobile web access among most of the young generation is using mobile to log to their facebook, gmail account. The owners of those sites must have introduce different features for those users. Eg:- facebook Mobile

Past and present of mobile interaction

Few years ago mobile phone was considered as luxury thing that only the rich people can have it. Therefore mobile interaction was not there very much. [2]

Figure 5- One of the past mobile phonesC:\Users\Heshani\Desktop\hesh-reserch\Mobile with HCI\images4.jpg

The mobile phone has numbers and user can dial the number and make the call. But there is no screen so they can't do many things with that phone.

And with the generations as 1st, 2nd, 3rd… onwards the mobile devices developed and with that development the mobile interaction also developed.

Later on mobile phones get the features like SMS, MMS, clock, alarm, calendar, calculator, phone book, push to talk, Bluetooth, infrared, roaming facility, GSM and CDMA. To more personalize there are ringing tones, SMS tones, logos, themes, screen savers and interchangeable covers. With all sort of features that the mobile devices have people more attracted towards the mobile devices. It shows that the mobile interaction is everywhere. Nowadays almost everyone has a mobile phone. Therefore they should know how to work with the device as they want. To do that user interface design and other design must be user friendly.

Within last few years there was a fast development in the technologies used in mobile devices. Although the development is predictable it change both mobile user interfaces physically and graphically. These graphical user interfaces helps to achieve more complex services in mobile devices. [3]

Nowadays mobile device interfaces are focusing on information presentation through diverse channels including visual, audio, and tactile channels. For various smart phones many complex programs are available such as Symbian and Windows Mobile. [4] And with the facility of 3G users can log in to web. Therefore it is very clear that currently mobile interaction is in a higher place. The idea is the design of mobile interaction has been identified the human needs, desires, requirements and capabilities. Combination of web with the mobile phone is the best example of mobile interaction. People use mobile to log in to facebook, gmail, skype,etc.

Future direction

In future technologies like multimodal interaction affect the user. Multimodality is considered as the next wave of mobile interaction.[12] The basic meaning of multimodality is that it merges the voice, touch and relevant screen displays so that user gets enhanced experience. As an example a person can call to his friend as "call Mike". And then the device would reply as "select the Mike you want to call from the keypad". It is a very fast and efficient way to interact with the mobile device. That means it directly affect the usability. Multimodality allows users to use the mobile phone easily while driving or walking.

Future mobile interaction is going into a new territory/context and finding out more about the people and then envisioning useful things within that space. People want to know everything through their phone. That means information overload. The most interesting need that surfaced from using different methods of observation and interviews was of overloaded, perpetually connected and highly social people. [3, 5]

The designers expect diverse, radical, sometimes perplexing set of futures. Sometimes these futures are so radical; it feels that we will have to move to these new worlds, leaving everything we know behind. Of course, our technological future is not a place we leap to in time travel; it emerges from and is shaped by our past and current practices.

Future of the mobile devices depends on the mobile human interaction and transmitting feelings among the technology and liveliness. The interaction with children also will come in a context that suitable for children. Mobile technology will surely benefit children in ways we cannot predict.

It is not very easy to forecast about the future because it depend on human requirements and the capabilities of developing those requirements.


Mobile Interaction plays a significant role as an aspect of human computer interaction because nowadays almost everyone has a mobile phone, digital camera, iPhone, iPod. Therefore the topic is very important. Currently mobile interaction is in a very major position and in the future it will be more advanced. Basically in the context of usage simultaneous activities, single hand use, information overload and competing user goals must be considered. That is in the side of the human. In the device side it is important to consider text input, small screen, less battery life.

With the consideration of above scenario, the field is improving day by day with new mobile applications. People get the feeling of familiarity due to mobile devices connect people to each other, provide entertainment, managing schedules and make their life easier.

My Contribution

When I was researching about the mobile interaction, I used internet as the way for finding recourses. However I learned lot of things to do this research paper. At the present I am getting to know various new things that weren't in my mind 2 months ago. I'm sure that in the date of completion I will be able get a good idea about the mobile interaction.

From the beginning up to now I had to put lot of effort to achieve this success. I hope that I will be able to do a good research by doing work like this. This experience will be an added advantage in my future.


I like to specially thank my supervisor, Mr. P.M.Karunarathne, for encouraging me and helping me out in every possible way for this independent study.