Critical Evaluation Of Research Papers Psychology Essay

Critiquing is a systematic method of appraising the strengths and limitations of a piece of research in order to determine its credibility and/or its applicability to practice (Jha, 2008). Seeking only limitations in a study is criticism and critiquing and criticism are not the same (Sachdeva, 2009). A critique is an impersonal evaluation of the strengths and weakness of the research study being. There are numerous tools available to help both novice and advanced reviewers to critique research studies (Fisher, 2005).

The researcher is going to critique two different studies, one of them from China and the second study is local relevant to Oman. In his article Entry Barriers in the Creation and Development of Small and Micro Enterprises in Nizwa, Sultanate of Oman: An Exploratory Study (Manalo, 2012) attempts to examined the Entry Barriers faced by forty-four (44) small and micro entrepreneurs in the creation and development of their enterprises in Souq, Nizwa, Sultanate of Oman. (Manalo, 2012) used the quantitative and qualitative approaches on his study. The quantitative analysis used the self-made questionnaires while the qualitative analysis used the interview method.

The second study called Awaking Dragons: An exploration Of The Internationalization Of Chinese SMEs From The Electronic Sectors, (Bell, 2011). (Bell, 2011) attempts to explore the internationalization behavior of 50 Chinese electronics SMEs. (Bell, 2011) emphasized on qualitative perspectives away from survey methodologies and towards more qualitative methods.

Research Problem

A research problem is often first presented to the reader in the introduction to the study (Maginn, 2008). It aims to identify and challenge the assumptions underlying existing theories as well as to identify gaps in existing literature that need to be filled. In either case the statement should at least broadly indicate to the reader what is to be studied (Dunne, 2005). Broad problems are often multi-faceted and will need to become narrower and more focused before they can be researched.

(Manalo, 2012) conducted research problem concisely which gave it more accuracy and clarity that was about the Entry Barriers of small and micro entrepreneurs in the creation and development of their enterprises in Souq, Nizwa, Sultanate of Oman. While, (Bell, 2011) failed again to show research problem clearly.

Research Questions

Experimental and quasi-experimental studies should clearly state a hypothesis identifying the variables to be manipulated, the population that is being studied and the predicted outcome (Fischer, 2005). (Manalo, 2012) extracted two major problems:

1. What are the entry barriers faced by Omani entrepreneurs in the creation of small and micro enterprises in Nizwa, Oman?

2. Are these entry barriers significantly associated to the gender of Omani entrepreneurs?

The questions which drive Manalo’s research in Entry Barriers in the Creation and Development of Small and Micro Enterprises in Nizwa, Sultanate of Oman: An Exploratory Study stated explicitly within the text. While (Bell, 2011) failed to add a clear outlining of this question and indeed, one may argue that this is not the thrust behind the research. That is, as the paper takes on a multifaceted focus, it is difficult to narrowly define the author’s goals.

Research Objectives

The aims and objectives of a study, the research question and the research hypothesis is to form a link between the initially stated purpose of the study or research problem and how the study will be undertaken (Jha, 2008). They should be clearly stated and be congruent with the data presented in the literature review. The use of these items is dependent on the type of research being performed.

Some descriptive studies may not identify any of these items but simply refer to the purpose of the study or the research problem; others will include either aims and objectives or research questions (Jha, 2008). None of these studies outlined any of research objectives.

Significance of the Study

The output of this study can be used as a source in variety of ways such as: (1) baseline data for Small and Micro Enterprise (SMEs) Program of banks and various financial institutions; (2) basis for policy formulation and implementation by the local and national government in regulating and reinforcing small business industry; and (3) reference for future comprehensive study in the field of entrepreneurship (Manalo, 2012). (Manalo, 2012) addressed the significance of his study clearly while (Bell, 2011) failed to outline it in his study.

Literature review

The literature review should give an overview of the available literature which frames or surrounds the problem being researched. It should look at the similarities and differences between the literature, as well as the strengths and limitations. It should illustrate how the current study fits into the existing framework of research or how it fills a gap in theliterature. The primary purpose of the literature review is to define or develop the research question while also identifying an appropriate method of data collection (Dunne, 2005). It should clarify the nature of the gap in the existing knowledge in the field, and illustrate what additional information is suggested to fill that.The most of studies included should be recent and less than five years old. It is important also that the review should include some historical as well as contemporary material in order to put the subject being studied into context.

A good review usually begins with an introduction which identifies the key words used to conduct the search and information about which databases were used. The themes that emerged from the literature should then be presented and discussed (Sachdeva, 2009). In presenting previous work it is important that the data is reviewed critically, highlighting both the strengths and limitations of the study. It should also be compared and contrasted with the findings of other studies (Glenn, 2010).

(Manalo, 2012) was successful in gathering information and data and presenting them in an organized and flowing manner. The resources and the literature that have been reviewed are all significant and related to the topic that the researcher sought to explore. He outlined two relevant studies helped in developing the research questions. While (Bell, 2011) failed to provide any of previous studies relevant to his topic.

Theoretical framework

Following the identification of the research problem and the review of the literature the researcher should present the theoretical framework (Maginn, 2008). Theoretical frameworks are a concept that novice and experienced researchers find confusing. It is initially important to note that not all research studies use a defined theoretical framework (Fischer, 2005). A theoretical framework can be a conceptual model that is used as a guide for the study (Dunne, 2005) or themes from the literature that are conceptually mapped and used to set boundaries for the research (Glenn, 2010).

A sound framework also identifies the various concepts being studied and the relationship between those concepts (Jha, 2008). Such relationships should have been identified in the literature. The research study should then build on this theory through empirical observation. Some theoretical frameworks may include a hypothesis. Theoretical frameworks tend to be better developed in experimental and quasi-experimental studies and often poorly developed or non-existent in descriptive studies (Maginn, 2008).The theoretical framework should be clearly identified and explained to the reader. Both of the studies showed relevant theoretical framework helped in identification the research problem and enabled the reader to know the research problem.

Research Methodology

Methodology refers to the nuts and bolts of how a research study is undertaken. There are a number of important elements that need to be referred to here and the first of these is the research design. There are several types of quantitative studies that can be structured under the headings of true experimental, quasi-experimental and non-experimental designs (Fischer, 2005).

Research design

Although, within each of these categories there are a range of designs that will impact on how the data collection and data analysis phases of the study are undertaken. However, (Fischer, 2005) states these designs are similar in many respects as most are concerned with patterns of group behaviour, averages, tendencies and properties. (Manalo 2012) used the quantitative and qualitative approaches to research. He used the descriptive or survey method for quantitative analysis; and interview method for qualitative analysis.

While (Bell, 2011) emphasized on qualitative perspectives away from survey methodologies and towards more qualitative methods which allowed the study to use descriptive statistics and the data gathered were analysed using thematic content analysis.

Sample and sample size

The degree to which a sample reflects the population it was drawn from is known as representativeness and inquantitative research this is a decisive factor in determining the adequacy of a study (Jha, 2008). In order to select a sample that is likely to be representative and thus identify findings that are probably generalizable to the target population a probability sample should be used (Dunne, 2005). The size of the sample is also important in quantitative research as small samples are at risk of being overly representative of small subgroups within the target population.

The risk of sampling errors decrease as larger sample sizes are used (Maginn, 2008). In selecting the sample the researcher should clearly identify who the target population is and what criteria were used to include or exclude participants. It should also be evident how the sample was selected and how many were invited to participate (Sachdeva, 2009). (Manalo, 2012) and (Bell, 2011) both failed to outline details about sampling and sample size that was drawn from research population as well as type of sample used in the studies.

Instrument Design

After identifying the appropriate data gathering method the next step that needs to be considered is the design of the instrument. Researchers have the choice of using a previously designed instrument or developing one for the study and this choice should be clearly declared for the reader. Designing an instrument is a protracted and sometimes difficult process (Glenn, 2010).

If a previously designed instrument is selected the researcher should clearly establish that chosen instrument is the most appropriate. This is achieved by outlining how the instrument has measured the concepts under study. Previously designed instruments are often in the form of standardized tests or scales that have been developed for the purpose of measuring a range of views, perceptions, attitudes, opinions or even abilities. There are a multitude of tests and scales available; therefore the researcher is expected to provide the appropriate evidence in relation to the validity and reliability of the instrument (Fischer, 2005).

(Manalo, 2012) used in his study two types of research instruments: the questionnaire and interview. The questionnaire was the primary instrument while interview was the supplementary instrument used in the course of this study. He used the Likert’s 5-point scale in the quantitative interpretation of the responses. Respondents were to choose from 1 to 5, where 1 means it is not a barrier and 5 means it is a major barrier when starting a business. While (Bell, 2011) failed to outline details about his research instruments, in fact there was not any information about it at all.

Validity and Reliability

One of the most important features of any instrument is addressed under the broad headings of validity and reliability respectively. In general, validity is described as the ability of the instrument to measure what it is supposed to measure and reliability the instrument’s ability to consistently and accurately measure the concept under study (Jha, 2008). In these circumstances the researcher should indicate how the reliability and validity of the adapted instrument was established (Sachdeva, 2009).

To establish if the chosen instrument is clear and unambiguous and to ensure that the proposed study has been conceptually well planned a mini-version of the main study, referred to as a pilot study, should be undertaken before the main study. Reliability refers to consistency in measurement. In common terms the reliability of a test is the extent to which subsequent administrations would give similar results. A test which is not reliable will give different results every time. Validity is the complement to reliability and refers to the extent to which what is being measured reflects what was expected to measure.

A critique survey sheet was provided to six experts composed of Training Managers, Entrepreneurs, Researchers, and Academicians for the purpose of validity test (Manalo, 2012). The reliability coefficient of the test was computed using Cronbach’s Alpha Coefficient, a statistical tool commonly used to measure the internal consistency or reliability of a psychometric test score for a sample of examinees (Manalo, 2012).While (Bell, 2011) failed to outline any details about validity and or reliability used in his study, in fact there was not any information about it at all.

Data Collection

Data collection is the process of acquiring subjects and collecting the data needed for the study. The actual steps of collecting the data are specific to each study and are dependent on the research design and measurement techniques. The purpose of data analysis, regardless of the type of data collected, is to impose some order on a large body of information so that some general conclusions can reached and communicated in a research report. The researchers discussed the data collection technique, which they used in collecting data. The researchers explained the tool that they used in analyzing the data. The data analysis procedure used by the researchers was explicitly discussed making the research reliable and validity. (Manalo, 2012) clearly indicated that questionnaire and interview were the tools of data collection while (Bell, 2011) clearly indicates that In-depth interviewing was the selected method of data collection. Such a method is clearly a valid approach in terms of critical research.

Data Analysis

The researcher should clearly identify what statistical tests were undertaken, why these tests were used and what were the results. Inferential statistical tests are used to identify if a relationship or difference between variables is statistically significant. Statistical significance helps the researcher to rule out one important threat to validity and that is that the result could be due to chance rather than to real differences in the population. Quantitative studies usually identify thelowest level of significance as Ps O.O5 (P = probability) (Dunne, 2005).

To enhance readability researchers frequently present their findings and data analysis section under the headings of the research questions (Glenn, 2010). This can help the reviewer determine if the results that are presented clearly answer the research questions. Tables, charts and graphs may be used to summarize the results and should be accurate, clearly identified and enhance the presentation of results (Glenn, 2010).

The percentage of the sample who participated in the study is an important element in considering the generalizability of the results. At least fifty percent of the sample is needed to participate if a response bias is to be avoided (Sachdeva, 2009).

The study used the percentage and the weighted arithmetic mean. The mean scores of the respondents were used as representative answer of the population, and the Chi-Square test for association was used to establish if association existed among variables (Manalo, 2012). (Bell, 2011) failed to outline any details about how data been analyzed and what was the specific statistical analysis used.

Finding and Recommendations

In this section, the implications of the research results are evaluated and interpreted in relation to the research question.

This is where the findings and the selected theoretical framework come together. The discussion should contain a clear statement of support or otherwise of the original hypothesis or research question. The results of this study and those of other studies should be discussed, and any suggestions for improvements or further research are made here.

The discussion of the findings should be related back to the literature review thus placing the study in context (Glenn, 2010). Any interpretations or inferences drawn should be clearly identified as such and consistent with the results.

The significance of the findings should be stated but these should be considered within the overall strengths and limitations of the study (Dunne, 2005). In this section some consideration should be given to whether or not the findings of the study were generalizable, also referred to as external validity. Not all studies make a claim to generalizability but the researcher should have undertaken an assessment of the key factors in the design, sampling and analysis of the study to support any such claim.

Finally the researcher should have explored the significance and relevance of the study. Applying findings in practice should be suggested with caution and will obviously depend on the nature and purpose of the study.

In addition, the researcher should make relevant and meaningful suggestions for future research in the area (Jha, 2008).

(Manalo, 2012) enabled in identifying major barriers such as the lack of or inadequate funds and other barriers include many locals have wrong attitude towards work (socio-cultural and the Lack of excellent skilled workers among the locals (economic barrier. Manalo succeeded in presenting recommendations which included conduct similar studies on Regional or National level in order to further validate the findings of the study and re-consider full-Omanization in small enterprise sector in the future due to lack of skilled labor among the locals (Manalo, 2012). (Bell, 2011) also recommnded further research on a larger scale as a longitudinal studies (Bell, 2011).

Logical consistency

There should also be a clear link between the steps beginning with the purpose of the study and following through the literature review, the theoretical framework, the research question, the methodology section, the data analysis, and the findings (Fischer, 2005). In general (Manalo, 2012) was satisfied the conditions of consistency for literature review, the theoretical framework, the research question, the methodology section, the data analysis, and the findings. But about (Bell, 2011) study it failed to satisfy these conditions for most of them.

Ethical considerations

(Maginn, 2008) identify four fundamental moral principles: autonomy, non-maleficence, beneficence and justice. Autonomy infers that an individual has the right to freely decide to participate in a research study without fear of coercion and with a full knowledge of what is being investigated. Non-maleficence imphesan intention of not harming and preventing harm occurring to participants both of a physical and psychological nature (Dunne, 2005). Beneficence is interpreted as the research benefiting the participant and society as a whole (Glenn, 2010). Justice is concerned with all participants being treated as equals and no one group of individuals receiving preferential treatment because, for example, of their position in society (Jha, 2008). (Dunne, 2005) also identify four moral rules that are both closely connected to each other and with the principle of autonomy. Both of the studies failed to mention any of ethical considerations while studies being conducted.

Conclusion

The process of critiquing involves an in-depth examination of each stage of the research process. It is not a criticism but rather an impersonal scrutiny of a piece of work using a balanced and objective approach, the purpose of which is to highlight both strengths and weaknesses, in order to identify whether a piece of research is trustworthy and unbiased.

(Bell, 2011) used a qualitative research which used a naturalistic approach that seeks to understand phenomena in context-specific settings. Qualitative research, broadly defined, means ‘any kind of research that produces findings not arrived at by means of statistical procedures or other means of quantification’. Qualitative paradigm stems from antipositivistic, interpretative approach, is idiographic, thus holistic in nature, and the main aim is to understand social life and the meaning that people attach to everyday life (Fischer, 2005).

According to (Sachdeva, 2009) a qualitative research involves an interpretative, naturalistic approach of the subject matter. Qualitative research is about studying things in their natural settings. A researcher conducting qualitative research attempts to make sense of, or interpret, phenomena in terms of the meaning people bring to them. Qualitative research involves different methods of gathering and collecting of empirical materials such as case study personal experience, introspective, life story, interview, observational, historical, interactions, and visual texts.

Determining the reliability and validity of a qualitative research is different from determining reliability and validity of a quantitative research. In quantitative research, the extent to which results are consistent over time and an accurate representation of the total population under study is referred to as reliability and if the results of a study can be reproduced under a similar methodology, then the research instrument is considered to be reliable (Dunne, 2005). Likewise, in quantitative research validity determines whether the research truly measures that which it was intended to measure or how truthful the research results are. The means of measurement used by the researchers were accurate and they are actually measuring what they intended to measure (Maginn, 2008).

Determining the reliability and validity of a qualitative research is different. The reliability and validity of the research depends on the ability and effort of the researcher. Determining the reliability and validity of the research encompasses the study of the credibility and trustworthiness of the researcher. Determining validity is also different in qualitative research. Validity is related to reliability and in qualitative research; this is determined by the trustworthiness of researcher and the quality of the research.