This essay has been submitted by a student. This is not an example of the work written by our professional essay writers.
This study was designed to determine the differences in quality that middle school mathematics teachers experience when teaching outside of their field (math). By combining subject content knowledge and pedagogical content knowledge scores for teachers both in their field and out of their field a more direct understanding of the difference in quality was revealed. A descriptive research methodology was used for this study.
The purpose of this chapter is to present a description of the research design, explain the population used for this study and sampling procedures, discuss the access and permission information, explain the data collection process, detail the survey instrumentation, general procedures of this overall study, and describe the data analysis.
A combination of descriptive and correlational research methodology was used for this study. Two assessments were administered to a stratified sample of middle school mathematics teachers of Illinois. The assessments used in this study were developed by the Diagnostic Assessments for Middle School Teachers (DTAMS, 2008). DTAMS has created a variety of assessments for measuring Number/Computation, Geometry/Measurement, Probability/Statistics, Algebraic Ideas, Physical Science, Life Science, and Earth/Space Science. For this study the participants took the Algebraic Ideas and the Physical Science for the in-field and out-of-field assessments, respectfully.
The completed DTAMS assessments were then sent to The University of Louisville Center for Research in Mathematics and Science Teacher Development (CRMSTD) to be scored by professionally trained staff.
Â Â Â Once scored, CRMSTD staff sent a detailed summary which included scores on individual items. The Algebraic Ideas assessment scores included measurements of the four different knowledge types: (1) memorized knowledge, (2) conceptual understanding, (3) higher-order thinking, and (4) pedagogical content knowledge. The Physical Science assessment scores included measurements of five different knowledge types: (1) declarative knowledge, (2) scientific inquiry and procedures, (3) schematic knowledge, (4) pedagogical content knowledge, and (5) science, technology, and society knowledge. These scores then formed the basis for which the qualities of the in-field and out-of-field teachers were compared.
Each of the knowledge types are equally weighted allowing for a simple mean value of the scores to represent an individual's in-field and out-of-field quality score. A paired t-test (p = 0.05) was then used to determine the strength of the relationship between in-field and out-of-field quality scores. Furthermore, relationships between subject content knowledge and years of experience and pedagogical content knowledge and years of experience were also addressed. Correlations between other variables were further explored and discussed in the data analysis section.
Population and Sampling Procedures
The population chosen for this study targeted teacher's certified to teach middle school mathematics, specifically, sixth, seventh, and eighth grade. Middle school teachers especially in math and science are commonly expected to teach outside of their chosen field of study. Although English and art certified teachers may also be asked to teach science courses, mathematics teachers should at least have some fundamental knowledge of the physical science curriculum. "Mathematics is therefore widely perceived as the mother/basis of science" (Chiu, 2007; Dehaene, 2003). Many professional development services are designed specifically to target math and science as a single entity.
The stratification of the population stemmed from the method used to distribute the assessments (i.e. mailing list). The mailing list was associated with the professional organization Illinois Council of Mathematics Teachers (ICTM). Although the post has almost 700 subscribers, only about 150 of them were middle school mathematics teachers. Â Therefore, the stratification of the population was the certified Illinois mathematics teachers that are committed to a professional organization, which most-likely makes them a more engaged subset of mathematics teachers. The participants ranged in gender, age, experience, and educational background. These participants then self-selected to participate. Reasons for participation were not investigated.
Access and Permission Information
The Illinois Council of Teacher of Mathematics (ICTM) granted permission to use their email list. Two notifications regarding the survey were sent to the members on the ICTM mailing list in 2012.
Utilizing mailing lists for the Illinois Council of Mathematics Teachers (ICTM), members of ICTM were sent a link to the Physical Science and Algebraic Ideas assessments. The teachers were able to download three Adobe Reader files including both assessments as well as the Letter of Informed Consent. After signing the Letter of Permission and Understanding and completing the assessments (demographic questions are incorporated into each of the assessments) the participant returned the documents to the researcher either through email or physical means. Information such as participants name, name or location of the school at which they work, or any other personal information was not asked for as part of the process. Respondents were anonymous.
Upon receipt of 21 completed assessments, the researcher submitted the assessments to the University of Louisville Center for Research in Mathematics and Science Teacher Development (CRMSTD) staff for scoring. Three to four weeks after CRMSTD received the data the scoring of the assessments was completed and results were sent to the researcher via email.
Saderholm, Ronau, Brown, and Collins (2010) developed several survey instruments designed to measure mathematics knowledge in four content domains and science knowledge in three content domains. The Diagnostic Mathematics Assessment for Middle School Teachers can be used to measure any one of the following four content domains: Number/Computation, Geometry/Measurement, Probability/Statistics, and Algebraic Ideas. The Diagnostic Science Assessment for Middle School Teachers can be used to measure any one of the following three content domains: Physical Science, Life Science, and Earth/Space Science. For this study the Algebraic Ideas and the Physical Science assessment instrument was used. The Physical Science assessment was chosen over the alternative Earth Science and Life Science due to the levels of mathematical content involved in the Physical Science. The intention of this study was to measure a teachers knowledge domain both in and out of their field of study. The science assessment that is most closely related to mathematics was the Physical Science. The mathematics assessment (Algebraic Ideas) is composed of 20 items - 10 multiple-choice and 10 free response questions. The science assessment (Physical Science) is composed of 25 items (20 multiple-choice and five free-response items). These instruments were designed to assure both validity and reliability. Internal reliability for the Algebraic Ideas assessment in 73 cases was established with a Cronbach's alpha of 0.87. Equivalence reliability for the Algebraic Ideas assessment in 73 cases and the Pearson's product moment correlation alpha was 0.69. Finally, the inter-scorer reliability for the Algebraic Ideas assessment using three scorers and 17 assessments found that the inter-class correlation measure to be 0.90.
The design of this study allowed the researcher to quantitatively compare and contrast the in-field knowledge domain scores (SCK and PCK) with their corresponding out-of-field scores. The following sections detail the various components that make up the survey (i.e. Algebraic Ideas Assessment and the Physical Science Assessment).
Diagnostic Teacher Assessment of Mathematics (DTAMS) Instrument
The survey instruments that was used to measure both the pedagogical content knowledge and subject content knowledge of the middle school mathematics teachers involved in this study was the Diagnostic Teacher Assessment of Mathematics and Science Instrument (DTAMS).
The scores for Declarative Knowledge, Scientific Inquiry, Schematic Knowledge, and Science-Technology-Society Knowledge in the Physical Science survey were combined to a single score representing Subject Content Knowledge. This score was then weighted to represent 50% of the out-of-field teacher quality score. Similarly, the Pedagogical Content Knowledge was also weighted to represent 50% of the out-of-field teacher quality score.
The scores for Memorized Knowledge, Conceptual Understanding, and Higher-Order Thinking in the Algebraic Ideas survey were combined to a single score representing Subject Content Knowledge (in-field). This score was then weighted to represent 50% of the in-field teacher quality score. Similarly, the Pedagogical Content Knowledge was also weighted to represent 50% of the in-field teacher quality score.
For each participant, the in-field teacher quality score was directly compared to the out-of-field teacher quality score. The difference in teacher quality was defined as the difference of the in-field and out-of-field scores.
Validity of the DTAMS Instrument
Three strategies were used to ensure the validity of the assessments. Saderholm, Ronau, Brown, and Collins (2010) developed teams of mathematicians, mathematics educators and middle school teachers to develop the mathematical content by using national recommendations, objectives of standardized tests, and research on the common misconception of both middle school teachers and students. Secondly, teams composed of mathematicians, mathematics educators, and middle school teachers were used to develop prototype and parallel assessments. Finally, National reviewers then assessed the appropriateness of each of the items. Each reviewer was given four sets of assessments and asked to identify the mathematical content of the item based on a specific list of topics, identification of the knowledge type (I - Memorized Knowledge, II - Conceptual Understanding, III - Problem-Solving / Reasoning, and IV - Pedagogical Content Knowledge), and finally an indication if the items represent important mathematics content for middle school teachers (High, Medium, or Low levels of importance). Each assessment was reviewed by at least six reviewers with at least one mathematician, one mathematics educator, and one middle school teacher.
Items that were chosen for the DTAMS instrument were accepted only if (1) at least 60% of the reviewers identified it as assessing a particular content topic, (2) at least 60% of the reviewers identified it as assessing a particular knowledge type, (3) at least 75% of the reviewers determined the item represented as being important knowledge for middle school teachers. Items the failed to meet each of these requirements were not used in the assessment.
Reliability of the DTAMS Instrument
From May 2005 through March 2006, 2077 teachers completed two forms each of the DTAMS assessments. From these assessments three types of reliability were computed: Internal, Equivalency, and Inter-Scorer (DTAMS, 2006).
Items that were chosen for the DTAMS instrument were accepted only if (1) at least 60% of the reviewers identified it as assessing a particular content topic, (2) at least 60% of the reviewers identified it as assessing a particular knowledge type, (3) at least 75% of the reviewers determined the item represented as being important knowledge for middle school teachers. Items that failed to meet each of these requirements were not used in the assessment. To compute the internal reliability, Cronbach's alpha correlations were used on 429 cases and generated a reliability coefficient of 0.87 (DTAMS, 2006).
Pearson product moment correlations were used to measure the strength (magnitude) of association between 489 pairs of pre- and post-measures of the teacher assessments. The Pearson product moment for the Algebraic Ideas assessment was 0.69 for 73 cases (DTAMS, 2006).
Three individual raters scored the assessments of 17 participants including all four domains (I - Memorized Knowledge, II - Conceptual Understanding, III - Problem-Solving / Reasoning, and IV - Pedagogical Content Knowledge). Using an inter-class correlation coefficient (0.90) and treating judges as a random variable the inter-scorer reliability was computed. Total scores for each content domain were used computing the agreement factor. Acceptable coefficients were obtained on all three types of reliability (DTAMS, 2006).
The purpose of this study is to clearly detail the differences in quality from in-field and out-of-field teachers by way of using a measure of both subject and pedagogical content knowledge.
The assessments used in this study are designed to measure various aspects of both subject content knowledge and pedagogical content knowledge. Specifically, the Physical Science assessment measures Declarative Knowledge, Scientific Inquiry, Schematic Knowledge, Science-Technology-Society Knowledge and Pedagogical Content Knowledge. The first four measures were combined to a single score representing Subject Content Knowledge. The Pedagogical Content Knowledge score represented the out-of-field Pedagogical Content Knowledge score.
The Algebraic Ideas assessment measures Memorized Knowledge, Conceptual Understanding, Higher-Order Thinking and Pedagogical Content Knowledge. The first three measures were combined to single score representing Subject Content Knowledge. The Pedagogical Content Knowledge score represented the in-field score.
The in-field (Algebraic Ideas) Subject Content Knowledge score represented 50% of the in-field teacher quality score. The in-field (Algebraic Ideas) Pedagogical Content Knowledge score represented the other 50%. Similarly, the out-of-field (Physical Science) Subject Content Knowledge score represented 50% of the out-of-field teacher quality score. The out-of-field (Physical Science) Pedagogical Content Knowledge score represented the other 50%.
The demographic data and the DTAMS scores were then processed though SPSS (version 13.0) to complete the data analysis. The analysis included descriptive statistics as well as correlation tables and paired t-tests analysis.
The raw data from the assessment scores provided by CRMSTD were combined into six sub-scores per participant (In-Field: subject content knowledge, pedagogical content knowledge, and a quality score and Out-of-Field: subject content knowledge, pedagogical content knowledge, and a quality score). The data analysis included a comparison of the in-field and out-of-field quality scores as well as correlations of the demographic information and the six sub-scores.
Strengths and Weaknesses
The stratification of the population, the size of the population sample, and aspects of self-report data contributed to the limitations of this study.
As a voluntary member of ICTM, each participant undoubtedly demonstrates an affinity toward the profession of teaching mathematics. These participants self-selected for participation. Reasons for participation were not investigated.
The size of the sample used in this study is limited in several ways. The assessments involved are extraordinarily time-consuming, and it can be frustrating to take an assessment outside ones chosen field. This alone hindered the number of respondents. Furthermore, the scoring of each of the assessments cost ten dollars since it must be performed by trained staff from the University of Louisville Center for Research in Mathematics and Science Teacher Development (CRMSTD).
This study employed self-reported data, for which there is no way to examine the integrity under which the responses to the assessment were generated. However, there is no reason to conclude that a voluntary member of a professional organization who elects to take an assessment would have any intentions of falsifying their responses.
The Diagnostics Teachers Assessment for Mathematics and Science (DTAMS), which has been shown to be both a valid and reliable instrument, was used in this study to collect the data. A descriptive and correlational research methodology was chosen due to the parallel nature of the in-field and out-of-field assessment scores that were collected. The data collected allowed for the researcher to compare the in-field and out-of-field teacher quality scores using paired sample t-tests, correlations, and linear regression models.
Chapter three was an overview of the research methodology, population sampling, and an overview of the procedures, the instrument, and the data analysis. Some of the strengths and weaknesses of the study were also explored. The following chapter presents the data collected, its analysis and a summarized understanding of what the data means with regard to this study.