The paper highlights the procedural consequences of naturalistic observation elicitation method and a sharp look at advantages and disadvantages of this method. Requirement gathering is done in this regard by observing multiple users and by noting down the views they want to express about UCLAN VLE (virtual learning environment) e-learn. The method performed is a direct observation technique in which investigator makes its presence necessary during the methodological procedure. This is kept in consideration that requirements are being gathered from the daily routine users of e-learn, moreover they were given instructions and proper guidance throughout the observational procedure to express whatever they feel about this system.
Naturalistic observation is a research method which commonly associated for requirement gathering in psychology and social science. This method is carried out mostly by observing users within their comfort natural surroundings. This technique is often used where lab research does not provide realistic (unnatural effect over user behaviour affecting the purity of observations) approach to experiment performed or where researcher have to cut down the extra expenses involved in procedure.
Naturalistic observations are quite different from structural observations as investigator cannot intervene in subject's natural behaviour and environment. For example mostly the use of natural observations is apparent in behavioural studies of animals; number of different documentaries are made and then presented over different TV channels in this manner. Moreover this research method is used mostly in psychology, computer interaction design and usability, accessibility requirements for different web applications etc.
One of the main advantages of this kind of research methodology is that it helps to acquire pure natural findings without any manipulation of Lab environment. More it helps an investigator to study the under consideration scenario beyond the ethical boundaries for Lab environment. For example if someone has to gather the information about prisoner's behaviour inside the prison and their psychology, one would not like to perform it as a Lab work because of ethical concerns for creating a sample prison in the Lab and study the subjects in there. So it's most suitable to study natural and real prison environment. Furthermore it can support the generalized validity of research carried out, because whenever a research is carried out in the Lab environment; it is really necessary to confirm that it will work in broader perspective in a real time environment. So a researcher would be able to confirm the validity of its research by carrying out naturalistic observations for his finding quite easily.
The main disadvantage of naturalistic observation is that it is difficult to judge the exact cause of behaviour of the user, more over this technique somehow provides less control over outside variables during experiment execution. User's behaviour could differ a lot when they come to knew that they are being observed by anybody and they would not show their real intentions. Moreover when users are aware of idea of research, they might change their way of thinking and may act over their ideas which they think are more appropriate for the research idea. Also there is a possibility that if same idea of research is carried out by different researchers, the conclusions of observations may differ at great range for the same inspected behaviour.
Researchers may utilize a number of different techniques to collect data from naturalistic observation. This might involve writing down the number of times certain behaviour occurred in a specific period of time, or making an actual video-recording of the subjects of interest.
There are several methods to note down observations and carry out the research through this method. Two main approaches are as follows:
â€¢ Tally counts: The investigator writes down about certain behaviours that when and how many times specific behaviours occurred.
â€¢ Observer narratives: In this method the observer tries to take notes during the experiment session and then review it later for conclusions.
â€¢ Audio or video recordings: if such a situation is encountered where it becomes necessary to take notes in audio or video recording; a researcher may use these resources for data collection.
The process for obtaining a representative's life samples can occur in different ways such as:
â€¢ Time sampling: Noting down the behavioural constraints for specific intervals of time.
â€¢ Situation sampling: Noting down the behaviour constraints in different situations.
As it is impossible for a researcher to gather the data of subject's every life aspect; so investigators often prefer to take samples of subject's behaviour. In this manner this is made sure that samples are taken from such an angle that they should be able to represent the whole life structure of the subject.
Naturalistic observation research method is one of the highlighted methodologies being used in HCI designs. Researchers believe that observing a subject in his natural environment can really be helpful in improving the capabilities of (Vyas et al., 2009) computer hardware and software designs. The latest hardware and software developments have revolutionized the concept of computer oriented education. E-learning has played a great role in reshaping education as fun oriented phenomena, where learners can learn while having fun. Even though researchers have not depended over naturalistic observations alone; they have been using agile mixture of research methodologies to get conclusions (Vyas et al., 2009) to their satisfaction. But still the importance of observation techniques have been playing a vital role and acted has a great source of requirement elicitation. As web development evolved as globally recognized source of education and accessing information, the need to make it more accessible and usable for community enhanced too. So developers and designers used naturalistic observation environments to make them according to standards and user expectations. So in result of this today it's simply has become essential to put the user expectations at top priority. Use of observational methods in making games (Costabile et al., 2008) more interactive for children and using them as motivation to learn have done a great service in the education field. More over for mobile learning which is a combination of e-learning and mobile computing (Costabile et al., 2008) observational research techniques have really played a vital role by taking it to next level and made it mobile technology user's favourite by providing compactness and mobility at the same time. Furthermore House hold electronic essentials like Digital TV and Digital video recorders (DARNELL, 2007) vendors also used naturalistic observations as an effective source to get accurate requirements from the user point of view. User initiated environments (UIEs) (Kim et al., 2008) requirement analysis has also been done through naturalistic observation research, which help investigators to acquire the instrument structural hierarchy very easily. Naturalistic observation techniques have also been used in improvement of other requirement elicitation techniques like ethnographic personas (\#197 et al., 2002) used for making the design of any computer system more interactive and usable. On campus researches to encourage student as active part of society which was done at Lancaster University was done through observational methodology (Day et al., 2007) the name of "breaking the bubble". So for development of real time social research naturalistic observations have played a handy part too.
One of the main strengths of naturalistic observations is that they are really helpful in supporting a new discovery (Woods, 1995; Mumaw, Roth, Vicente and Burns, 2000). They draw attention towards fairly large phenomena and help in concluding new ideas which can be justified through additional studies. Field observations provide an opportunity to have a gaze over complex structure of working environment, more over it helps to judge the impact of inventions to the nature of work in process.
Naturalistic observation method or User test research method (Tanaka et al., 2005).
Observational procedure is being performed taking 5 UCLAN students living in Derwent residential hall taking in consideration that they access e-learn at least thrice a week, so they can highlight the good and bad features of this virtual learning environment. Accessibility, usability and gender issues were excluded in this simple observational test because no disable user has been observed during the procedure.
5 different scenarios have been generated for testing them for each participant and they have been asked to do simple tasks in their accommodation rooms (to give them a natural feel), so time to go through the particular task and the problem which every participant has with each task of e-learn noted down. Participants have been requested to be open when expressing problems which they think encountered with certain scenario of UCLAN WebCT.
Results and Findings
The following table consists of readings and observations noted during naturalistic observation procedure performed over UCLAN e-learn.
This includes time taken for each task asked to complete for each participant and their behaviour and response they have shown in accordance to advantages and disadvantages of WebCT, whether do they find it interactive, accessible, usable, fun oriented?
Pilot study was conducted to obtain target best and worst time for each task, for comparisons with actual readings concluded through naturalistic observations.
AVERAGE BEST TIME TO PERFORM
AVERAGE WORST TIME TO PERFORM
AVERAGE TIME DURING PROCEDURE
LOGIN INTO WebCT
ACCESSING THE MODULE INFORMATION
GETTING BACK TO MY WebCT
DOWNLOADING A LECTURE
The comparison of timing information gathered clearly shows that even though usual users of e-learn encounter problems with the some features. But common point of view expressed is that e-learn should be more interactive and should be sustained with some main feature design revisions.
\#197, BLOMQUIST, S. & ARVOLA, M. 2002. Personas in action: ethnography in an interaction design team. Proceedings of the second Nordic conference on Human-computer interaction. Aarhus, Denmark: ACM.
COSTABILE, M. F., ANGELI, A. D., LANZILOTTI, R., ARDITO, C., BUONO, P. & PEDERSON, T. 2008. Explore! possibilities and challenges of mobile learning. Proceeding of the twenty-sixth annual SIGCHI conference on Human factors in computing systems. Florence, Italy: ACM.
DARNELL, M. J. 2007. How do people really interact with TV?: naturalistic observations of digital tv and digital video recorder users. Comput. Entertain., 5, 10.
DAY, N., SAS, C., DIX, A., TOMA, M., BEVAN, C. & CLARE, D. 2007. Breaking the campus bubble: informed, engaged, connected. Proceedings of the 21st British HCI Group Annual Conference on People and Computers: HCI...but not as we know it - Volume 2. University of Lancaster, United Kingdom: British Computer Society.
KIM, J. H., GUNN, D. V., SCHUH, E., PHILLIPS, B., PAGULAYAN, R. J. & WIXON, D. 2008. Tracking real-time user experience (TRUE): a comprehensive instrumentation solution for complex systems. Proceeding of the twenty-sixth annual SIGCHI conference on Human factors in computing systems. Florence, Italy: ACM.
Mumaw, R. J., Roth, E. M., Vicente, K. J. & Burns, C. M. (2000). There is more to monitoring a nuclear power plant than meets the eye. Human Factors, vol 42, # 1, 36-55.
VYAS, D., HEYLEN, D., NIJHOLT, A. & VEER, G. V. D. 2009. Experiential role of artefacts in cooperative design. Proceedings of the fourth international conference on Communities and technologies. University Park, PA, USA: ACM.
Potter, S. S., Roth, E. M., Woods, D. D. & Elm, W. (2000). Bootstrapping multiple converging cognitive task analysis techniques for system design. In J. M. Schraagen, S. F. Chipman & V. L. Shalin (Eds.) Cognitive Task Analysis (pp. 317-340). Mahwah, NJ: Lawrence Erlbaum Associates, Inc.
TANAKA, E. H., S\, \#237, AM\, L., \#233, BIM, L., HELO\ & ROCHA, S. V. D. 2005. Comparing accessibility evaluation and usability evaluation in Hag\áQu\ê. Proceedings of the 2005 Latin American conference on Human-computer interaction. Cuernavaca, Mexico: ACM.
Vicente, K. J., Roth, E. M., Mumaw, R. J. (2001). How do operators monitor a complex, dynamic work domain? The impact of control room technology. International Journal of Human Computer Studies, 54, 831-856. Available online at: http://www.idealibrary.com
Woods, D. D. (1995). Process-tracing methods for the study of cognition outside the experimental psychology laboratory. In G. A. Klein, J. Orasanu, R. Calderwood & C. E. Zsambok (Eds.) Decision Making in Action: Models and Methods. Norwood, NJ: Ablex Publishing Corporation.