Characteristics of Experimental Research

1) Hattie’s (2003) participants were classified as experienced or expert according to the National Board for Professional Teaching Standards certification process. This process is based on the best available evidence of what constitutes high standards of teaching and learning and has been rigorously researched and evaluated. On the boards website it says

“All of the research contributes to understanding and improving the National Board Certification process. Yet, it is misleading to draw major conclusions about the overall value and impact of National Board Certification based solely on individual studies. No single study or small group of studies can effectively describe the range of impact of the National Board Certification process.”

(see http://www.nbpts.org/resources/research/impact_of_certification#impact)

They stop short of saying that the research “proves” that accreditation improves the quality of teaching and learning and this is in accord to with what Johnson and Christensen (2008) say

“you [should] eliminate the word prove from your vocabulary when you are talking about research”? p22

What does Johnson and Christensen (2008) mean and why are the NBPST so modest about their claims?

On the contrary to the misunderstandings by many that a hypothesis can be tested and proven to be true through research, Johnson and Christensen (2008) asserts that research is not a tool based on which a phenomena can be proved to be true. It is a systematic process which acquires data of a relevant topic or an issue, analyses the data, interprets and reports information. Research helps to understand a particular assumption or a claim to be reasonable, unreasonable or assist to understand the reality. Johnson and Christensen (2008) chapters one, two and three helps to understand that there are two approaches to research: qualitative and quantitative research, which have led to different methods of inquiry in order to understand what is to be true or what constitute reality. Hattie’s (2003) research indicates the importance of educational research in understanding the important factors contributing to teaching and learning. However, as each educational setting differs from another, educators need to be able to read and to interpret research findings to their own contexts to be able to have better implications. According to Johnson and Christensen (2008), and McMillan (2004), educators also need to be able to question the quality of the research and take into consideration the credibility of the research and the usefulness of the outcomes in making informed decisions about teaching and learning in their own contexts.

Johnson and Christensen (2008) suggest that better judgment cannot be based on a single or small sample research studies but multiple studies using mixed methods research methods.

Generalizing the research outcomes in to the educational settings needs to be based on the quality and the nature of study taken place, participants and the research methods used and comparing other studies to see if there are any correlating conclusion on previous studies

2) What is the key defining characteristic of experimental research?

Active manipulation of an independent variable is the key defining characteristic of an experimental research.

The characteristic of experimental research is where the researcher objectively observes a phenomena which is made to occur in a strictly controlled situation where one or more variables are systematically changed by manipulation.

“Active manipulation is not involved in any other types of research. Because of this …experimental research provides the strongest evidence of all the research methods about the existence of cause and effect relationships” (Johnson, Christensen 2008,p.41)

3) A researcher discovered that students who studied mathematics at University scored higher on a test of problem solving ability at the end of their degree than students who did not. The researcher argued that the study of mathematics should be encouraged because “it improves a student’s ability to solve problems” What type of design is this? Do you agree with the researcher’s conclusion? Give reasons.

What type of design is this?

Quantitative, causal-comparative research design.

Do you agree with the researcher’s conclusion? Give reasons.

Do not agree with the research conclusion.

Johnson & Christensen suggesrts that a “well design experimental research is virtually always better for determining cause and effect than causal-comparative research or any other type of non experimental research”(p.43). Due to the fact that non experimental research does not allow for manipulation of independent variables and weaker techniques of controlling extraneous variables, cause and effect relationships are unable to be substantiated. Cause and effect is better explained in experimental research than in causal-comparative research (pp.43, 48).

According to Johnson & Christensen (2008.pp.43-48), and McMillan (2004. P194), there are times that causal-comparative research necessary to be conducted. The purpose of this research intended to be to investigate causal relationships when an experiment is not possible. However, the research needs to have prior research evidence to substantiate or validate claims made by the research been conducted.

Causal comparative research needs to demonstrate that all extraneous variables are being recognized and considered for an impartial conclusion. Due to the fact that the existing groups are used in the comparison, attention needs to be given to the possible differences in characteristics and other factors which might influence the research outcome. Causal comparative research often fails to demonstrate the influences of extraneous variables on the research participants.

4) Contrast the main characteristics of phenomenological and ethnographic approaches to research. Give an illustration of how each approach could be applied to questions that are relevant to your workplace

Phenomenological and ethnographical research approaches are two of five Qualitative research methods and all of which share some common characteristics. Phenomenology understands the essence of experience. Phenomenological study focuses on how research participants experience certain phenomena. The researchers describe and interpret the experience of the participants in order to understand their own perspectives of the phenomenon. As Merleau-Ponty (1962) states, ‘We can only really understand phenomenology by doing it’ there is a difference between comprehending phenomenology intellectually and understanding it from the inside (van Manen 1984). (Observation: a complex research method.(Ethnological methods). http://find.galegroup.com.simsrad.net.ocs.mq.edu.au/

Ethnographic research however, is not about individual perception or experience, it’s about a group perceptions, people or a cultural belonging or social norms. Ethnographic study intends to provide in-depth descriptions and identification of cultural norms or values such as attitudes, practices and norms among a group of people. The researchers include a holistic account in describing how a someone become a part of a group: how they interact and how they identify themselves as a group.

Ethnography at the work place would identify how the team operates within the work environment; the identity of the team; common practices which would identify the characteristic of the team. Ethnography enables people to adapt or adopt to be part of the work environment; to belong to the team. Phenomenology would identify how individuals operate and perceives situations, concepts and issues. It helps to analyse personal experiences of different individuals. E.g. how one person experience or understands an issue or a concept might be different to another person. Therefore the experience and the interpretations are different.

5) In his paper, Hattie (2003) draws upon a number of lines of evidence to show that it is what teachers do that makes a difference to student outcomes, and he then reports on a study which attempts to show what expert teachers do. Firstly, he alludes to studies that employ Hierarchical Linear Modeling techniques which demonstrate that what teachers do has an important impact on student learning (see page 1). Next, he describes the results of a series of Meta Analyses which show that different the sorts of activities which teachers do have different effects on student learning (see page 3). Then, he reports on the results of a literature review which identifies the characteristics of expert teachers (see page 5).

Finally, he describes a study (see page 10) which demonstrates that these characteristics can in fact differentiate between expert and merely experienced teachers. In other words he has shown that these are meaningful characteristics and are therefore things that teachers should aspire to as they are markers of high standards of teaching and learning.

The following table shows the forms of data which Hattie (2003) used in his study to assess the 15 dimensions they used to distinguish expert from experienced teachers.

Name the research paradigm he used in this study and describe its advantages. Which specific type of design did he employ (please give reasons for your choice?)

The Research paradigm Hattie (2003) uses is the Mixed methods research Design, a combination of Qualitative and Quantitative research methods. In his study, Hattie identifies meaningful characteristics and proposes a paradigm, a view that teachers should aspire to become expert teachers as opposed to becoming experienced in order to promote better outcomes for teaching and learning. He distinguishes the expert and the experienced teacher, and the influence they have on children’s learning outcomes. He arrived at these conclusions by using Mixed method research model in order to demonstrate credibility of his study and the research outcome. The research framework used by a researcher is important as it guides the research and it affects the research outcomes. The quality of a research depends on the methods of data collection, and the quality is influenced and enhanced by multiple methods of data collection. If only one method of research is used, the research findings might be influenced by the limitations of the techniques used and therefore considered as a weaker study and the limitations needs to be explained. Multiple methods allow for credibility and strengthens the research outcome therefore allows for a strong proposition of the study being conducted. Johnson and Christensen 2004, (p.51) suggest using multiple perspectives and research methods in a single study strengthens the purpose of the study and provides clarity and credibility compared to a study used a single research methodology. Although different research methods have its own strengths and weaknesses, the researcher’s ability to use the mixed method design successfully prevents the research outcomes showing overlapping weaknesses. “The fundamental process of mixed method research” suggests “it is wise to collect multiple sets of data using different research methods and approaches in such a way that the resulting mixture or combination has complementary strength and non overlapping weaknesses” (p.51). As Johnson and Christensen suggest that an experimental research is able to better demonstrate causality but limited in realism due to the experimental setting that the research is being conducted. Realism is well demonstrated in ethnographic studies where the researcher is able to observe participants in their natural settings but limited in demonstrating causality compared to the experimental methodologies.

As Hattie’s study consist of a variety of methods, it suggest a strong validity for his findings and therefore able to use it as an argument proposing that teacher needs to aspire to become experts in promoting better learning outcomes for children.

6) Identify the research topic(s), research problem(s), research purpose(s) and research question(s) in Brighton and Hertberg’s (2004).

Brighton, C., M. & Hertberg, H., L. (2004). Reconstructing the vision: Teachers’ responses to the invitation to change. Research in Middle Level Education Online, 27(2), 1-20.

Identify the research topic(s),

Teacher Attitudes; Teacher Response; Educational Change; Classroom Techniques; Instructional Innovation; Attitude Change; Teaching Models

variety of factors, including teachers’ pre-existing beliefs about teaching and learning, and teachers’ willingness and capacity for reflection greatly influence their responses to differentiation in heterogeneous, middle school classrooms.

Research problem(s),

Many teachers in these settings downplay the differences among the students and “teach to the middle”, not catering for children’s individual needs

Not catering for increasingly diverse student population

heterogeneously grouped classrooms are consistent with middle school philosophy as well as recommendations from leading middle school advocacy groups, Educational philosophy

Research purpose(s)

Address the variation in student academic readiness, interests, and learning profiles.

.

Research question(s)

1. What are teachers’ responses to the invitation to change their beliefs and practices to better address students’ academic diversity?

2. What patterns of teachers’ responses to this invitation emerge?

7) The Director of a local Preschool wants to know whether children should begin the day with a flag raising ceremony and the singing of the National Anthem. Is this a researchable question as stated? Why or why not? If not, how might you modify it to become one?

Is this a researchable question as stated? No

Why or why not? It is unethical

Best practice of educational research is to avoid questions which are sensitive to moral and ethical principles and practices, and should not be subjected to empirical research for solutions.

Flag and the National Anthem inform nationhood and identity of a country and a group of people. Therefore, suggesting whether the flag should be raised is unethical and unethical questions are inappropriate to be researched.

Christenson and Johnson states “these ideas typically involves making judgments of aesthetics, morality and religions”(p.64).what is morally right, and proper and improper differs from one culture to another culture.

The director’s intention to raise the flag and to sing the national Anthem has potential to foster a sense of cultural identity within the preschool community. One could question which flag will be raised? What is the dynamic of the preschool setting? What is the contributing factor to initiate such a practice? Is the whole community represented with one flag? Is the flag representative of all cultures or a particular group? Is it going to foster unity or divide? What is the purpose of this suggestion?

Individual beliefs and practices are influenced by the cultural practices and the society that people are part of and these beliefs and values are instilled from an early age. People are therefore sense and feel differently about issues and practices. Raising the flag and singing the National Anthem is culturally specific and the need to initiate such a ceremony needs to be clearly explained.

Searching for ethics provided a valuable article which is worth mentioning, perhaps not directly relevant to the question being asked, yet provides an insight about humanity. “Recent Russian history should have put to rest the view that everything is learned and man is infinitely malleable. After 75 years of cruel tyranny during which every effort was made to destroy civil society to create the New Soviet Man, we learn that people kept civil society alive, if not well. The elemental building blocks of that society were not isolated individuals easily trained to embrace any doctrine or adopt any habits; they were families, friends, and intimate groupings in which sentiments of sympathy, reciprocity, and fairness survived and struggled to shape behavior…..Mankind’s moral sense is not a strong beacon light, radiating outward to illuminate in sharp outline all that it touches. It is, rather, a small candle flame, casting vague and multiple shadows, flickering and sputtering in the strong winds of power and passion, greed and ideology. But brought close to the heart and cupped in one’s hands, it dispels the darkness and warms the soul.

The moral sense: presidential address, American Political Science Association, 1992. American Political Science Review 87.n1 (March 1993): pp1(11). http://find.galegroup.com.simsrad.net.ocs.mq.edu.au/

If not, how might you modify it to become one?

Children growing up knowing and singing the National Anthem and being part of flag raising ceremonies can be positive experiences..

One way the question can be modified to reflect the preschool program is to focus on the routine by explaining the reason for such a proposition and to explain how this will be carried out.

Question:

Dear parents and families,

Would you like to give permission for your child to participate in a morning routine where the children will have the opportunity to raise the flag and to sing the National Anthem?

Through this experience the children will have the opportunity to become familiar with the National Anthem and will have the opportunity participate in the practical experience of raising the flag. Children will do this by being part of small groups and will have turns in raising the flag on different days.

Please circle yes or no or feel free to write any comments as your inputs are valuable to our program.

Yes

No

Other comments:______________________________________

8) How do quantitative and qualitative research questions differ in structure?

In quantitative research, the questions are clearly and consciously articulated to communicate what will be studied. An important aspect of the question formulation is to convey the information about the variables which will be investigated. Knowing or understanding the variable is an important element in quantitative studies. An example of a quantitative question would be “Do children from low socio economic families, who attend preschool programs five days a week develop fluent writing skills compared to who do not? In the quantitative studies, the question will specify or name the variables and often may explain how they may be related in single statements.

Qualitative studies, like quantitative research involves looking at a general problem. However structuring of specific statements and questions differ greatly. Purpose and the logic of questions differ compared to quantitative questions. Qualitative questions are more open ended, general, and does not involves variables. As an example, “What can preschool teachers do to enhance children’s literacy learning? In general, qualitative questions are more open ended and quantitative questions are close ended.

(Johnson& Christensen, McMillan 2004)

9) An investigator wishes to identify the factors that predict “persistence” in problem solving tasks. In an application to the Ethics committee he describes a study in which observations are made of preschool children who are given 5 minutes to solve a puzzle. However, in reality the puzzle has no solution. What are the major ethical issues raised by this study and what procedures should the experimenter employ to comply with ethical standards?

Child development and early child hood education and care literatures inform that children’s ability to complete tasks or to engage in problem solving experiences are guided by their environments, the adults around them, their moods, interests and many other factors of the times. A child’s ability to persist cannot be measured by a 5 minutes observation with manipulating a puzzle. Children’s skills and abilities vary according to their developmental stages and the opportunities they have with learning experiences. Each child is unique in his or her developmental status. Perhaps the research could have focused on children’s ability to concentrate in tasks by providing a number of activities for children to choose from and by observing their attention with the chosen activity. Johnson and Christensen states “many of the tests that are used with preschool children are referred to as screening tests rather than intelligence tests or academic achievement tests primarily because the predictive validity of many of the preschool tests is weak” (p160). Therefore, an achievement test of puzzle and problem solving is not appropriate for a preschool age group.

Procedures that the experimenter needs to employ to comply with ethical standards:

The researcher needs to have informed consent from the participants, and in this case from parents or guardians, and given the purpose of the study, procedures, risks, benefits, alternative procedures and limits of confidentiality.

10) The purpose of this question is to start you thinking more deeply about an area of practice that interests and to introduce you and your interests to the group. This will enable individuals with similar interests to find one other and begin working cooperatively designing studies in subsequent modules – qualitative (Module 3), non-experimental quantitative (Module 4) and experimental studies (Module 5). Please note that you are not bound by what you write here, you can change your area of interest at any time during the unit.

Briefly describe

please give a brief, informative and relevant title to your area of interest

School readiness or transition to school programs in early childhood education

an area of practice that you would like to research

parents expectations of children being ready for school in early childhood settings, concepts and beliefs around the topic, Education Department’s suggestions of preferred skills and knowledge as the children start school

some problems that are relevant to the ongoing debate in the area

some parents expect that their children to be able to write their names and know the alphabet before commencing school

early childhood education in the longs day care settings and the teaching methods employed in the preschool settings

parents assumptions of learning occurs at preschool and care is provided at the early child hood settings

11) Compile a literature review: On-line database searching skills

Before you begin this question you should read the Finding resources module and follow the links to the Library’s support documents. Of particular interest are

Library research methods: Educational Studies and, depending on your area of interest, either

Researching a topic: Early Childhood Education or

Researching a topic: Education

Conduct four searches using Macquarie Library’s information sources (ie catalogue or on-line databases recommended for Education or Early Childhood Education) for information about your area of research interest and summarise your experience in the table below (NB this is adapted from the Research Log contained in Library research methods: Educational Studies). Briefly comment on any difficulties you faced and any strategies you might use to improve the results of subsequent searches.

Search #

Source

Catalogue eg: MU Library Catalogue

Database eg: ERIC

Search Type

Basic or Advanced

Keyword

Subject

Author

Search Strategy

Search terms

Operators (and, or)

Limits

Truncation

Total # of hits

Related terms i.e.:

Descriptors

Subject headings

Preferred terms

1

Eric database

CSA illumina

Basic Key word search

–

16738

School readiness

2

Eric database

CSA illumina

Advanced, key word search

Full text, Peer reviewed articles, Boolean search “and” “or”,

Limit to 2009-2010, Truncation Child*, preschool*

399

Transition to school

Writing skills

Early learning

Early childhood

3

A+ Education

Informit

Advanced, key word search

Full text, Peer reviewed articles, Boolean search “and” “or”,

Limit to 2005-2010, Truncation

early childhood education*

30

School readiness

Writing skills

Early childhood education

Preschool education

4

Expanded Academic Asap database

Infotrac provider

Advanced, key word search

Boolean search “and” ‘or”

Limited to Full text, peer reviewed articles

All dates

Truncation children*

2

literacy learning

transition to school

Preschool children literacy learning

Comments: Need more practice on using the truncations effectively.

Question

Learning outcome

Source

1

Understands the scientific approach to knowledge generation in terms of the basic assumptions of science, scientific method, building and testing theories and the central role that evidence plays in the process [K1.1].

Chapter 1 pp 16-23

2

Can describe the key characteristics of experimental research [K3.1].

Chapter 2, pp 41-

3

Understands the characteristics of the various types of non-experimental quantitative approaches to educational research and why it is difficult use the results of such designs to make definitive statements about cause and affect [K3.2].

Chapter 2, pp 43-

4

Understands the characteristics of the five main types of qualitative research and how they can be used to address educational problems [K3.4].

Chapter 2, pp 48-

5

Understands the types of mixed research approaches, their strengths and the contexts in which they are appropriate [K3.5].

Chapter 2 pp 51-52

Chapter 16 pp 446-448

6

Understands how research ideas are developed [K1.2].

Chapter 3 pp63-64, pp74-80

7

Understands that certain types of ideas are not directly researchable although such ideas may be able to be researched indirectly [K1.3].

Chapter 3, p 64

8

Understands how researchers define research problems, describe the purpose of their study, and develop research appropriate questions and how this process is different depending on whether the research is qualitative or quantitative in nature [K1.4].

Chapter 3 pp 74-80

9

Is aware of the ethical principles that shape and guide the research process (eg informed consent, deception, freedom to withdraw, protection from mental or physical harm, confidentiality and anonymity) and can identify which principles apply in a variety of research scenarios [K7.1]

Chapter 5

10

Knows how research evidence contributes to knowledge about education policy and practice [K1.5].

Can identify critical issues which warrant further investigation [A2.1].

11

Knows how to access existing stores of knowledge [K2.1].

Can explore the literature relating to particular aspects of professional practice [A1.1].