Analysis of the Child Behaviour Checklist
Disclaimer: This dissertation has been submitted by a student. This is not an example of the work written by our professional dissertation writers. You can view samples of our professional work here.
Any opinions, findings, conclusions or recommendations expressed in this material are those of the authors and do not necessarily reflect the views of UK Essays.
Chapter II: Literature Review
As suggested in the introduction, numerous researchers have explored the prevalence of emotional and behavioural problems across the globe. Researchers have also investigated correlates (e.g., age and gender) associated with emotional and behavioural problems. The psychometric properties of instruments assessing emotional and behavioural problems have also been a subject of interest. In addition, researchers have also investigated cross-cultural similarities and disparities among emotional and behavioural problems. The extensive literature that addresses these issues, and which also helped formulate the rationale for the current study, is presented in five sections. The first section highlights the problems associated with epidemiological studies and compares the two main approaches to epidemiological studies, namely the categorical and the empirical approach. The second section provides a detailed description of the CBCL including the evolution of the measure, its psychometric properties, its advantages and disadvantages, as well as its range of applicability. The third section provides a description of the theoretical rationale for assessing cultural similarities and disparities associated with emotional and behavioural problems. Multicultural findings based on the CBCL as well as age and gender differences associated with emotional and behavioural problems are also reported. The fourth section consists of a review of the various processes involved in assessing the psychometric properties of instruments and findings based on psychometric properties of the various translations of the CBCL. The fifth section consists of a brief cultural and socio-political description of Pakistani society followed by a description of the salient features (i.e., family, community and cultural factors) in relation to emotional and behavioural problems in Pakistani society. Finally, there is a description of the objectives of the current study.
Epidemiology of Emotional and Behavioural Problems
Current reviews of epidemiological studies indicate that there is a high prevalence of emotional and behavioural problems among children and adolescents around the world (Costello et al., 2004; Hackett & Hackett, 1999; Waddell et al., 2002). In one review, Costello et al. compared findings across several developed countries (including Canada, the United States, the United Kingdom, Germany and Australia) to investigate the prevalence of emotional and behavioural problems as well as that of other psychological problems. Based on their findings, the overall prevalence rates of psychological problems among children and adolescents had a very broad range (0.1% to 42%), with varying rates for each category of disorder. Categories include disruptive behaviour disorders (i.e., conduct disorder, oppositional disorder and attention deficit hyperactivity disorder), mood disorders (i.e., major depressive disorder and bipolar disorder), anxiety disorders (i.e., phobias, generalized anxiety disorder, obsessive compulsive disorder, and post-traumatic stress disorder) as well as substance abuse and dependence. A critical examination of the studies included in the review revealed that variations in prevalence rates may be attributed to methodological flaws such as substantial disparity across studies with regard to sample size and the age range assessed. Moreover, differences across studies in terms of the measures used, the criteria employed as well as the type of informant may also have influenced the findings.
In contrast to Costello et al.'s (2004) review, Waddell et al.'s (2002) review was based on more stringent criteria; studies based on samples of similar size and age range, as well as using similar methodology were compared. Based on Waddell et al.'s review, the prevalence rates of emotional and behavioural problems varied between 10% and 20%. Although findings from both reviews vary considerably, the prevalence rates of emotional and behavioural problems across developed countries is still high and warrants serious attention. Moreover, methodological disparities across studies underscore the need for a uniform methodology to investigate the prevalence of emotional and behavioural problems.
In contrast to developed countries, there are few researchers investigating prevalence rates in developing countries (e.g., Bangladesh, India, Sri lanka, Sudan, and Uganda) (Costello, 2009: Fleitlich-Bilyk & Goodman, 2004; Mullick & Goodman, 2005; Nikapota, 1991; Prior, Virasinghe, & Smart, 2005). Moreover, there is a scarcity of reviews of the existing studies. In one review, Hackett and Hackett (1999) compared results from India, Puerto Rico, Malaysia and Sudan, and the prevalence rates of psychological disorders ranged from 1% to 49%. Similar to research in developed countries, researchers attribute variations in findings to methodological problems across studies, which include an inadequate sample size, paucity of explicit and internationally accepted diagnostic criteria, as well as inconsistencies in assessment procedures (Fleitlich-Bilyk & Goodman, 2004). Moreover, prevalence rates among developing countries may also partly be linked to the social, economic and medical environment. For example, lack of medical resources and awareness about psychological problems may result in parents not knowing how to seek help (Gadit, 2007). Social taboos further compound the problem, preventing people from reporting problems and deterring help-seeking behaviour (Samad, Hollis, Prince, & Goodman, 2005). More importantly, cultural variations in the conceptualization and identification of psychological problems may result in varied reporting of symptoms (Gadit, 2007). These environmental differences and methodological inconsistencies across studies emphasize the need for a cross-culturally robust methodology to investigate the prevalence of emotional and behavioural problems.
Along with methodological problems and environmental differences, emotional and behavioural problems merit investigation because they affect multiple aspects of children's functioning such as academic performance and social adjustment (Montague et al., 2005; Nelson et al., 2004; Vitaro et al., 2005). Researchers also state that there is high comorbidity among emotional and behavioural problems, (SteinHausen, Metze, Meier, & Kannenberg, 1998) which creates multiple problems for children and their caregivers. Moreover, many childhood disorders continue and influence functioning during adulthood. In fact, many adult disorders are now recognized as having roots in childhood vulnerabilities (Maughan & Kim-Cohen, 2005; Tremblay et al., 2005). Furthermore, recognizing and treating problems early can reduce the burden of the enormous human and financial costs associated with the assessment and intervention, especially in countries where resources are scarce (Costello, Egger, & Angold, 2005; James et al., 2002; Waddell et al., 2002). In addition, cross-cultural epidemiology of children's emotional and behavioural problems may also better inform current knowledge about the characteristics, course, and correlates of such problems, which in turn provide a scientific basis for appropriate mental health planning (Achenbach & Rescorla, 2007; Waddell et al.). Therefore, there is a strong need for a methodology that can be utilized for clinical as well as research purposes to assess emotional and behavioural problems among children and adolescents across cultures.
Current literature indicates that there are two main approaches to investigate the epidemiology of emotional and behavioural problems, namely the categorical and the empirical approach. There are several differences in both approaches including conceptualization of psychological problems as well as the methodology employed for their assessment. Both approaches will be discussed briefly.
The categorical approach. The categorical approach, based on the biomedical perspective, views psychological problems as a group of maladaptive and distressing behaviours, emotions and thoughts which are qualitatively different from the typical (Cullinan, 2004). That is, similar to medical diseases, an individual may or may not have a specific psychological disorder. Traditional epidemiological studies are based on the categorical approach as embodied in various editions of the Diagnostic and Statistical Manual for Mental Disorders (DSM) (American Psychiatric Association (APA), 1980; 1987; 1994; 2000) and the International Classification of Diseases (WHO, 1978; 1992). Examples of instruments used in traditional epidemiological studies to derive DSM diagnoses include the Diagnostic Interview Schedule for Children (DISC) (Costello, Edelbrock, Kalas, Kessler, & Klaric, 1982) and the children's version of the Schedule for Affective Disorders and Schizophrenia (Kiddie-SADS) (Puig-Antich & Chambers, 1978). At present, there is considerable debate about the validity of epidemiological studies based on the categorical approach. Researchers have highlighted that inconsistencies in prevalence rates may be due to conceptual and methodological issues linked with the DSM as well as methodological disparities among studies (Achenbach & Rescorla, 2007; Waddell et al., 2002). Each of these factors will be discussed briefly.
DSM related problems. Multiple conceptual and methodological problems are associated with the DSM. First, the DSM does not provide a methodology to operationally define different psychological disorders (Widiger & Clark, 2000). To operationally define DSM criteria, various diagnostic interviews such as the DISC have been developed. Unfortunately, meta-analyses indicate that the diagnoses based on the DISC and other diagnostic interviews are not in agreement with diagnoses made through comprehensive clinical interviews, which indicate that, neither diagnostic nor clinical interviews provide good validity criteria for testing DSM categories (Achenbach, 2005; Costello et al., 2005; Lewczyk et al., 2003). Second, the diagnostic categories and criteria provided in the DSM continue to change as reflected in the changes across the various editions of the DSM, namely the third edition (APA, 1980), third edition revised (APA, 1987), fourth edition (APA, 1994), and fourth edition text revised (APA, 2000), making comparisons across editions problematic (Achenbach, 2005). Third, although the current version, known as the DSM-IV-text revised (APA, 2000), aims at introducing cultural sensitivity in assessment and diagnoses by including an “outline for cultural formulation and a glossary of culture-bound syndromes” (APA., 2000, pg. 897), it does not provide criteria or guidelines regarding the use of the classification system with specific cultural groups (Paniagua, 2005). Since many of the DSM diagnostic criteria are based on Euro-American social norms, it is difficult to use the DSM criteria to identify psychopathology in individuals from other cultures.
In addition, there is growing consensus among researchers that DSM categories need to be more appropriate for children and adolescents of different ages and gender (Doucette, 2002; Segal & Coolidge, 2001). Turk et al. (2007) also highlight the saliency of factors such as age and gender when investigating prevalence rates. However, at present, this is not the case. Costello et al. (2005) have stated that the constant developmental changes of childhood create the need for an age- and gender- specific approach to epidemiology.
Before incorporating a developmental perspective in epidemiological studies, it is essential to have a better understanding of developmental psychopathology. Developmental psychopathology is based on the view that problems arise from different causes, manifest themselves differently at each stage, and may have diverse outcomes. Developmental psychologists do not support a specific theory to explain all developmental issues. Instead, they try to incorporate knowledge from multiple disciplines (Cicchetti & Dawson, 2002). Moreover, developmental psychopathology also includes an analysis of the existing risk and protective factors within the individual and also in his/her environment over the course of development (Cicchetti & Walker, 2003).
According to Costello and colleagues (2004), a developmental perspective in epidemiological studies is based on the inclusion of certain principles. First, precise assessment measures for the different phases in childhood and adolescence are required to compare children's functioning with that of their same-age peers. For example, problems such as fear of dark places is considered typical for 6-year-olds but not for 12-year-olds. Furthermore, the developmental perspective would include longitudinal studies to evaluate the ways in which developmental processes influence the risk of specific psychological disorders. For example, the developmental trajectory of physical aggression is such that there is an increase in Aggressive Behavior during the first few years of childhood, but it progressively decreases until adulthood (Tremblay et al., 2004). Moreover, developmental epidemiology would include frequent assessments to determine the onset of disorders. Frequent assessments would also assist in the identification of environmental and individual factors that contribute to the development of psychopathology. Although the developmental perspective emphasises the need for age- and gender-specific diagnostic criteria, longitudinal studies as well as frequent assessments, it is difficult to incorporate this perspective in studies based on the categorical approach as it is not sensitive to developmental changes.
Methodological disparities. A critical analysis of categorically based epidemiological studies reveals multiple methodological problems. These include inconsistencies in assessment and sampling procedures as well as absence of guidelines about using data from multiple sources. In terms of assessment procedures, both symptoms as well as significant impairment are required to identify children with disorders. This is corroborated by Costello et al. (2004), who report that the disparity in the prevalence rates of phobias (i.e., 0.1% to 21.9%) may be attributed to how phobias were assessed in each study, in particular, whether both symptoms (e.g., fear of open places, snakes) as well as significant functional impairment were taken into account in the identification of phobias. Waddell et al. (2002) state that the use of standardized measures has lead to an improvement in the assessment of symptoms; however, problems still exist with regard to how impairment is gauged or how measures may be combined to include symptoms as well as impairment. Another problem with assessment procedures is that different interview schedules (e.g., DISC and the Kiddie-SADS) and DSM editions have been used across studies, which may have contributed to differences in prevalence rates.
Incompatible sampling procedures may also have led to disparities in overall prevalence rates in categorically based epidemiological studies (Waddell et al., 2002). For example, studies such as the Great Smokey Mountains study (Costello, Angold, Burns, Erkanli, Stangel & Tweed, 1996) were relatively more comprehensive, and investigated a larger number of diagnostic categories than other studies. As a result, higher overall prevalence rates of psychological problems were reported compared to studies that did not assess as many disorders. Another sampling issue is that reviews were based on studies that differed with regard to the age range assessed; some studies focused on a younger age bracket (i.e., between 8 to 11 year olds), others on an older age bracket (i.e., 11 years and older), whereas some researches included a very broad age range (i.e., 6 to 17 year olds). In addition, there were inconsistencies across studies in terms of the type of informant used; some studies relied on parents only, some on children, while some combined data from parents, children as well as teachers. Differences in the age brackets assessed as well as the use of different informants may have contributed to disparities in epidemiological findings.
Another salient issue with regard to categorically based epidemiological studies concerns the coordination and interpretation of information from multiple informants. Since problem behaviours may only occur in specific situations or with specific individuals, multiple informants (e.g., teachers, parents and children) are necessary. However, since the respondent's context and perception have a great impact on the identification of psychological problems, poor agreement among respondents is frequently reported. For example, children normally report higher rates of internalizing symptoms (e.g., anxiety and depression) while parents tends to report higher rates of externalizing symptoms (e.g., Conduct Problems) (Rubio-Stipec, Fitzmaurice, Murphy, & Walker, 2003). Additionally, children are not considered reliable reporters of their own behaviour due to differences in cognitive abilities as well as the ability to report their own behaviour (Achenbach & McConaughy, 2003). Despite such findings, the categorical approach does not provide guidelines regarding obtaining and interpreting data from multiple sources, which complicates matters in terms of how to combine data into yes-or-no decisions about different symptoms.
The various conceptual problems associated with the DSM as well as the methodological flaws in epidemiological reviews highlight the problems associated with using the categorical approach as a basis for epidemiological studies. Moreover, these issues underscore the need for an approach that is methodologically sound and culturally appropriate for cross-cultural comparisons. An alternative to problems linked to the categorical approach, where an a priori criterion is imposed, can be a system that is empirically based and identifies problems as they occur in a population. Such an approach would be helpful in highlighting cultural differences in the manifestation of different emotional and behavioural problems. Moreover, there is also a need for a methodology that can be employed in a standardized, systematic fashion. Although the empirical approach is not a panacea for problems associated with epidemiological studies, it does provide solutions to some of the types of errors in the categorical system.
Empirical or dimensional approach. The empirical or dimensional approach, in accordance with a psychosocial perspective, views mental health as a continuum. The dimensional perspective supports the notion that all individuals experience problems involving behaviours, emotions and thoughts to varying extents. Those who experience such problems to an extreme extent (unusual frequency, duration, intensity, or other aspects) are more likely to have a psychological disorder (Cullinan, 2004). In contrast to imposing a priori criteria on children's emotional and behavioural problems, the empirical approach identifies problems as they present themselves in the population. According to Cullinan (2004), there are certain steps involved in developing a dimensional classification system for emotional and behavioural problems. These steps include creating a collection of items that reflect measurable problem behaviours experienced by children, identifying a group of children to be studied, assessing every child in the group on each problem, and investigating the data to identify items that co-vary, thus leading to the identification of different dimensions or factors. After the dimensions have been derived, the pool of items can be used to assess and classify emotional and behaviour problems among new populations. Given that the empirical approach is based on the identification of co-occurring problem behaviours in the population, instead of imposing a priori criteria, it is a favourable approach for cross-cultural epidemiological studies.
Within empirical approaches, the Achenbach System of Empirically Based Assessment (ASEBA) provides a good framework for epidemiological studies for multiple reasons. First, being empirically based, ASEBA identifies emotional and behavioural problems as they occur in the population. Second, it is based on a developmental perspective, has a uniform methodology, and also provides explicit guidelines about using data from multiple sources (Achenbach & McConaughy, 1997; Achenbach & Rescorla, 2001). Hence it provides solutions to problems that arise in the categorical approach. Moreover, Cullinan (2004) and Krol et al. (2006) state that ASEBA measures have been used more extensively compared to other measures of emotional and behavioural problems, such as the Conners Rating Scale- Revised (Conners,1990) and the Strengths and Difficulties Questionnaire (Goodman, 1997). Achenbach system of empirically based assessment (ASEBA).
Although the ASEBA has a non-theoretical, empirical base per se, it is greatly influenced by the principles of developmental psychopathology. For example, Achenbach highlights that problems may include thoughts, behaviours, and emotions that may manifest themselves differently depending on the age and gender of the individual (Greenbaum et al., 2004). Therefore, each ASEBA form provides norms based on the age and gender of the child, which enables an individual's functioning to be assessed in comparison to same-age peers. Furthermore, ASEBA is a multiaxial system that encompasses a family of standardized instruments for the assessment of behavioural and emotional problems as well as adaptive functioning. The five axes of the assessment model include parent (Axis I) and teacher (Axis II) reports, cognitive (Axis III) and physical (Axis IV) assessments as well as the direct assessment of children (Axis V) (Achenbach & McConaughy, 2003). The use of different ASEBA instruments provides a standardized and uniform methodology to incorporate information from multiple sources.
Furthermore, all ASEBA instruments are empirically based. In accordance with the empirical approach, the construction of the ASEBA forms involved a series of steps (Achenbach & McConaughy, 2003). Initially, a collection of potential symptom behaviours (i.e., items) was derived from multiple sources. These items were operationally defined in such a manner that respondents not trained in psychological theory could use them. In accordance with general item-development procedures, pilot tests were conducted to evaluate the clarity of items, response scales and item distribution. Finally, items that could differentiate between individuals who were not functioning well and their well functioning same-age peers were retained. Multivariate statistical analyses were applied to the retained items in order to identify syndromes of problems that co-occur. Syndromes were identified purely on the basis of co-occurrence, without any link to a particular cause. Subsequently, the syndromes of co-occurring problem items were used to construct scales. These scales were used to assess individuals in order to assess the degree to which they exhibit each syndrome. Since all ASEBA instruments are empirically based, findings can be compared on the basis of the manifestation of different emotional and behavioural problems, thereby providing a clearer picture of cross-cultural similarities and disparities of different emotional and behavioural problems.
In terms of the historical evolution of the system, ASEBA originated to provide a more differentiated assessment of child and adolescent psychopathology than the DSM. When ASEBA was developed, the first edition of the DSM (APA, 1952) had only two categories for childhood disorders, which included adjustment reactions of childhood and schizophrenic reaction childhood type (Achenbach & Rescorla, 2006). In contrast to the DSM, the first ASEBA publication highlighted more syndromes of emotional and behavioural problems (APA, 1952). Moreover, based on factor analyses, Achenbach (1966) identified two broad groupings of problems for which he coined the terms “Internalizing” and “Externalizing.” As described earlier, Internalizing Problems included problems with the self, such as anxiety, depression, withdrawal, and Somatic Complaints, without any apparent physical cause. On the other hand, Externalizing Problems included problems with other people, as well as problems linked to non-conformance to social norms and mores, such as aggressive and delinquent behaviour. Although all ASEBA forms are used extensively in clinical and research environments, the Child Behavior Checklist is the most widely recognized measure for the assessment of emotional and behavioural problems (Greenbaum et al., 2004; Webber & Plotts, 2008).
Child Behavior Checklist
An essential part and the cornerstone of Achenbach's multiaxial, empirical system is the Child Behavior Checklist (CBCL). Although the CBCL assesses social competencies as well as problem behaviours, it is widely recognized as a measure of emotional and behavioural problems as opposed to social competencies. In fact, researchers suggest that the CBCL is the most extensively utilized measure for the assessment of problem behaviours among children and adolescents as observed by their parents and caregivers (Krol et al., 2006; Greenbaum et al., 2004).
Although there have been multiple revisions to the initial CBCL, all versions have the same format and consist of two distinct sections. The first section measures social competencies. Parents are asked to respond to 20 questions regarding the child's functioning in sports, miscellaneous activities, organizations, jobs and chores, and friendships. Items also cover the child's relations with significant others, how well the child plays and works alone, as well as his/her functioning at school. Finally, respondents describe any known illnesses or disabilities, the issues that concern them the most about the child, and the best things about the child (Achenbach & Rescorla, 2006). The second section assesses problem behaviour and consists of 118 items that describe specific emotional and behavioural problems, along with two open-ended items for reporting additional problems. Examples of problem items include “acts too young for age”, “cruel to animals”, “too fearful or anxious”, and “unhappy, sad or depressed”. Problem behaviours are organized in a hierarchical factor structure that consists of eight correlated first-order or narrowband syndromes, two correlated second-order or broadband factors (i.e., Internalizing and Externalizing Problems) and an overall Total Problems factor. Parents/caregivers are asked to rate the child with regard to how true each item is at the time of assessment or within the past 6 months. The following scale is used: 0 = not true (as far as you know), 1 = somewhat or sometimes true, and 2 = very true or often true. In the case of respondents with poor reading skills, a non-clinically trained clincian can also admisnter the CBCL (Achenbach & Rescorla, 2006). For respondents who cannot read English but can read another language, translations are available in over 85 languages (Berube & Achenbach, 2008).
Development of the CBCL.
The first version of the CBCL dates back to 1983. To date, there have been two revisions of the CBCL; the first one in 1991 followed by the second in 2001, leading to considerable improvements in the measure. The main weakness of the initial CBCL was that comparisons across different age groups and respondents were problematic since syndromes had the same names but different items across different age forms (i.e., 4 to 5, 6 to 11, 12 to 16 years) as well as across different respondent forms (i.e., CBCL, teacher report form [TRF], and the youth self report [YSR]) To rectify the problem, the 1991 version included two new types of syndromes, the core and cross-informant syndromes. Core syndromes represented items that clustered together consistently across age and gender groupings on a single instrument. Cross-informant syndromes were based on those items from the core syndromes that appear on at least two of the three different respondent forms (i.e., CBCL, TRF, and YSR) (Greenbaum et al., 2004). These revisions facilitated comparisons across different age groups and informants. Moreover, the 1991 version of the CBCL also had new national level norms, which included norms for seventeen and eighteen year olds. Apart from practical benefits, changes such as a broader age range and precise criteria for different developmental levels, genders and type of respondents, helped make the CBCL and ASEBA instruments more accurately representative of the developmental perspective of child psychopathology (Greenbaum et al.).
Achenbach (1991) also conducted exploratory principal factor analyses of the syndrome scales. Based on the loadings of different syndromes, Achenbach identified Anxious/Depressed, Withdrawn, and Somatic Complaints as indicators of Internalizing Problems, whereas Aggressive and Delinquent Behavior were identified as indicators of Externalizing Problems. Since Social Problems, Thought Problems and Attention Problems did not load consistently on either second-order factor, they were not placed in any group (Achenbach, 1991; Greenbaum et al., 2004). Although Internalizing and Externalizing Problems identify different types of behaviour, the two categories are not mutually exclusive and may co-occur within the same individual. This is supported by research findings that indicate that there was a correlation between the two groups in both clinic-referred (.54) and non-referred (.59) samples matched on the basis of age, sex, race, and income (Achenbach, 1991).
Description of the current CBCL.
The current CBCL was published in 2001 and covers ages 6 to 18 years (CBCL/6-18; Achenbach & Rescorla, 2001). The CBCL/6-18 (Achenbach & Rescorla, 2001) provides raw scores, T- scores and percentiles for the following: (1) the three competence scales (Activities, Social, School); (2) the Total Competence scale; (3) the eight cross-informant syndromes; (4) Internalizing and Externalizing Problems and (5) Total Problems. The cross-informant syndromes of the CBCL/6-18 include Aggressive Behavior, Anxious/Depressed, Attention Problems, Rule-Breaking Behavior, Social Problems, Somatic Complaints, Thought Problems, and Withdrawn/Depressed.
As far as similarities and differences from previous versions are concerned, the current CBCL introduced some major and a few minor changes. One major change was the introduction of the DSM-oriented scales, based on which CBCL and other ASEBA forms can now be scored in terms of scales that are oriented toward categories of the fourth edition of the DSM (A.P.A., 1994). The introduction of the DSM-oriented scales has combined the categorical and empirical approaches and enables users to view problems in both the categorical and dimensional approaches (Achenbach, Dumenci & Rescorla, 2003; Achenbach & Rescorla, 2006). The DSM-oriented scales include six categories, namely Affective Problems, Anxiety Problems, Somatic Problems, Attention Deficit/Hyperactivity problems, Oppositional Defiant Problems as well as Conduct Problems. These scales are based on problem items that mental health experts from sixteen cultures across the world rated as being consistent with particular DSM diagnostic categories. Similar to the empirically based syndromes, the DSM- oriented scales also have age-, gender- and respondent-specific norms.
Another major change was that new normative data was collected using multistage probability sampling in forty U.S. states as well as the District of Columbia. The selected homes were considered to be representative of the continental United States with respect to geographical region, socio-economic status, ethnicity and urbanization (Achenbach & Rescorla, 2001). Moreover, complex new analyses based on new clinical and normative samples were conducted. However, the eight syndromes and Internalizing and Externalizing groupings published in 1991 were replicated with minor changes. Research findings indicated that correlations between scores on the 1991 syndromes and their 2001 counterparts ranged from .87 to 1.00 (Achenbach & Rescorla, 2001).
In terms of minor changes, a few items were replaced by newer ones and the wording of some was improved. Moreover, all syndromes except two were given the same names. Specifically, the 1991 “Delinquent Behavior” syndrome was renamed “Rule-Breaking” Behavior as some of the behaviour problems could be present among children too young to be adjudicated. Due to cross-loading of items, the 1991 “Withdrawn” syndrome was replaced by “Withdrawn-Depressed”, as some problem behaviours indicative of depression were found to load on the same factor (Achenbach & Rescorla, 2006).
Psychometric characteristics of the CBCL.
Psychometric properties generally refer to different aspects of reliability and validity. Reliability or the stability of scores can be assessed in different ways. Reliability can be measured in terms of test-retest reliability, internal consistency or inter-rater reliability. According to Kline (2005), test-retest reliability measures the stability over time of a set of scores on a specific test for a given sample, which means that the same test is given to the same group of individuals at two different points in time. On the other hand, internal consistency assesses the stability of the scores across items and is based primarily on the inter-correlations between items. Although different methods of measuring internal consistency exist, Cronbach's alpha (α) is one of the most common of the internal consistency indices. Inter-rater reliability is the stability of scores across informants or judges and is measured on the basis of the correlation between two informants (Kline, 2005).
The ASEBA manuals present extensive data on the reliability of ASEBA measures (Achenbach & Rescorla, 2006). There is evidence of test-retest reliability, internal consistency, as well as inter-rater reliability of the CBCL 6/18 (Achenbach & Rescorla, 2001). The Bibliography of Published Studies Using the ASEBA (Berube & Achenbach, 2008) also lists several studies that report on the reliability of the CBCL and other ASEBA instruments. In research in the social sciences, reliability values of .70 or higher are considered acceptable, but in complex studies a few reliability coefficients between .60 and .69 are common and are considered marginally acceptable (Gliner, Morgan & Leech, 2009).
With regard to the CBCL, researchers indicate high test-retest correlations for the empirical syndromes, DSM-oriented scales, Internalizing, Externalizing and Total Problems as values range from .80 to .94 (Achenbach & Rescorla, 2006). Similarly, researchers indicate high internal consistency for each of the empirical syndromes (alpha value .78 to .94), DSM-oriented scales (.72 to .91), Internalizing, Externalizing and Total problem score (.90, .94, .97, respectively). In terms of inter-rater reliability, it would be appropriate to compare results across comparable raters/informants (e.g., fathers and mothers). Since parents and teachers have a different role, level of interaction and knowledge of the individual being assessed, it would not be appropriate to compare their results. Achenbach and Rescorla (2001) have reported inter-parent agreement on different scales of the CBCL. The inter-parent correlations of the empirically based syndromes range from .65 to .82 and the DSM-oriented scales range from .63 to .88. Internalizing, Externalizing and Total problem scores have inter-parent reliability values of .72, .85 and .80, respectively. Unlike the test-retest and internal consistency coefficients, a few of the inter-parent correlations lie between the .60 and .69 range, which is not ideal but is still marginally acceptable. In general, it can be stated that different reliability indices show that the CBCL results in reliable sample scores, and lend support to its use in clinical as well as research settings.
Validity refers to the extent to which the test measures what it is designed to measure. Similar to reliability, each aspect of validity is assessed in different ways. These include face, content, criterion, and construct validity (Kline, 2005). Face validity refers to superficial visual inspection of a test by sophisticated or unsophisticated reviewers and is considered a weak and unreliable measure of validity. Content validity is the degree of correspondence between an instrument's items and what the instrument was designed to measure. Criterion validity investigates the relationship between test scores with one or more external criteria known to measure the attribute under study (Urbina, 2004). Criterion validity has two forms: predictive and concurrent validity. Predictive validity involves the use of future performance of the criterion whereas concurrent validity measures the criterion at about the same time. Construct validity refers to the degree to which a test may be interpreted to measure a specific construct or trait (Kline, 2005).
To assess content validity, Achenbach and Rescorla (2001) have conducted statistical analyses comparing referred and non-referred children matched on age and gender on each of the problem behaviour items. Results indicated that all items except “allergy” and “asthma” significantly discriminated between the two groups. Moreover, there was also evidence about particular items being identified by international panels of experts as being consistent with the DSM-IV diagnostic categories (Achenbach & Rescorla, 2000, 2001, 2003). In terms of concurrent criterion validity, research findings indicate that the CBCL results have significant correlations with findings based on behaviour assessment measures such as the Conners' Parent Rating Scales (Conners,1997), the Behavior Assessment System for Children (Reynolds & Kamphaus, 1992) and the Minnesota Multiphasic Personality Inventory Second Edition (MMPI-2; Butcher, Dahlstrom, Graham, Tellegen & Kaemmer, 1989) (Achenbach & Rescorla, 2006). Moreover, researchers have also documented predictive validity, including longitudinal studies indicating that the CBCL is predictive of outcomes such as substance abuse, referral for mental health services, and psychiatric diagnoses over long periods of time (Achenbach, Howell, McConaughy, & Stanger, 1998; Ferdinand, Blum, & Verhulst, 2001; Hofstra, van der Ende, & Verhulst, 2000). Moreover, the CBCL broadband scales accurately predicted subsequent psychopathology after five years (Petty, Rosenbaum, Hirshfeld-Becker, Henin, Hubley, LaCasse, Faraone & Biederman, 2009). Separately for each gender, multiple regression analyses of the scale scores on referral status, age, ethnicity, and socio-economic status (SES) have indicated that the CBCL/ 6-18 scale scores were associated with the referral status much more strongly than demographic variables (Achenbach & Rescorla, 2007). With regard to construct validity, researchers state that confirmatory factor analyses indicate a good theorized factor structure (i.e., overall fit, pattern of loadings) of the CBCL among different types of respondents including group care (Albrecht, Veerman, Damen, & Kroes, 2001) and mental health (Dutra, Campbell, & Westen, 2004) workers. Based on the current findings regarding the empirical basis and sound psychometric properties, the CBCL appears to have strong support across clinical and research settings. Since the CBCL has been used in multiple cultures, many researchers have investigated the construct validity of different translations of the CBCL in different populations. Findings based on those studies will be discussed later.
Advantages and disadvantages of the CBCL.
A close critical appraisal of the CBCL reveals that the instrument has multiple advantages and disadvantages. One of the main advantages is that empirical derivations of syndromes produce reliable and homogenous groupings of symptoms, with cut-off points that are referenced to a population rather than being dependent on an arbitrary number or criterion (Bird et al., 1996). McConaughy (2001) also provides a number of advantages of the CBCL. First, unlike the categorical approach, which can only identify children who meet the diagnostic criteria, the CBCL, based on the dimensional approach, helps identify children who do not meet the criteria (i.e., sub-syndromal conditions) but have functional impairment. Therefore, the CBCL provides a more comprehensive understanding of the full range of child behaviour problems. In addition, the large pool of items on the CBCL provides a more comprehensive view of the child's functioning and allows clinicians to investigate a wide spectrum of possible problems rather than focusing on a single complaint. McConaughy (2001) also indicates that results based on the CBCL can be tested in terms of psychometric standards of reliability and validity. Furthermore, using the relevant forms for different informants enables users to integrate information across a variety of situations. Finally, the low cost, ease of administration and scoring of the CBCL is another advantage (McConaughy, 2001).
CBCL's main limitations include lack of strong evidence in support of the competence scales and limited data on its use for younger children (Elliot & Busse, 2004). With regard to the competence scales, Drotar, Stein, and Perrin (1995) indicate that the Social Competence section of the CBCL is limited as it assesses selected aspects of social functioning, whereas social competence is a multidimensional construct which includes a wide range of social skills (e.g., competence in social situations, peer acceptance, and social skills). Moreover, the equal weighting of items that assess the quantitative (i.e., frequency) and qualitative dimensions of competence is considered problematic.
There is also criticism about the problem behaviour section. Furlong and Wood (1998) state that there may be issues in connection with objectivity and consistency in scoring as CBCL items are equally weighted, implying that the meaning of the response scale is the same for each item (e.g., “cruel to animals” may receive a rating of 2 when the behaviour is exhibited once or twice in general while “cries a lot” may receive a rating of 2 if it occurs once or twice in a day). In response, Achenbach (1991) has presented various rules and clarifications for using the response scale for selected items. Since the validity of the CBCL is largely based on its ability to discriminate between clinic-referred versus non-referred children, there is some criticism regarding its sensitivity in identifying changes in symptoms among children whose scores lie within the typical/normal range. Drotar et al. (1995) state that since the T-scores for the narrow band syndromes are truncated at both ends of the distribution, the ability of T-scores to differentiate between scores within the normal range is affected. In response, Achenbach has recommended using raw scores in which the full range of scores can be retained (Greenbaum et al., 2004). A final limitation of the CBCL is that similar to other assessment measures, the CBCL cannot be employed to determine specific forms of treatment or placement recommendations. Clinicians need to incorporate information based on the CBCL with information obtained from other sources before developing appropriate and meaningful interventions (McConaughy, 2001).
Range of applicability of the CBCL.
Despite some weaknesses, the CBCL is broadly employed in clinical, educational and research settings due to its demonstrated reliability and validity, as well as ease of administration and scoring (Achenbach & Rescorla, 2006; McConaughy, 2001). The recent additions in 2001 of larger normative and clinical samples as well as complex statistical analyses have made the CBCL more attractive to researchers. In the mental health domain, the CBCL is used in initial intake assessments (Achenbach & McConaughy, 2003). It is also used as a screening tool for emotional and behavioural problems including ADHD (DuPaul & Stoner, 2003), Conduct Problems (Lowe, 1998), and Obsessive-Compulsive Disorder (OCD) (Nelson, Stage, Duppong-Hurley, Synhorst, & Epstein, 2007) among others. Results based on the CBCL enable clinically useful comparisons, including the severity of the problem as well as the degree of deviancy in comparison to peers. A child's profile on the CBCL reveals essential taxonomic and diagnostic information in terms of whether there is elevation in single or multiple syndromes. Additionally, it also helps determine whether deviant behaviours are occurring in different contexts, such as home and school (Greenbaum et al., 2004).
In the clinical domain, repeated administration of the CBCL after intervals allows comparisons of the child's functioning over time, which can be informative in two ways (McConnaughy, 2001). First, this information is useful in assessing whether the child's problems are transient short-term issues, or chronic long-term patterns. Such information is salient in educational settings such as schools, which have long-term responsibilities to children. Second, information based on the CBCL can also be used in conjunction with other information to determine if treatment has been successful or if there is a need for intervention. Similarly, the CBCL is also useful when a child's adaptive functioning and home or school situation are questionable, as CBCL results, along with other information can help determine if an alternative placement is warranted (McConaughy, 2001).
The CBCL has been used for addressing multiple issues including theory, practical policy and methodological issues. It has been used as a measure for evaluation of outcomes in U.S. government programs (Greenbaum, Dedrick, Friedman, & Kutash, 2004). Moreover, it has been used in policy research on mental health services including the Fort Bragg study (Bickman, 1996) and the Center for Mental Health Services' National Evaluation of the Comprehensive Community Mental Health Services for Children and their Families Program (Holden, Stephans, & Connor, 2001). The CBCL has also been used in large-scale descriptive studies such as the National Adolescent and Child Treatment study (Greenbaum et al., 1996), which investigated mental health outcomes of nearly 800 children with serious emotional disturbances over a 7-year period. Additionally, the CBCL has been used to investigate normative developmental trajectories of children's behaviour problems to help detect developmental deviance in childhood and adolescence (Bongers, Koot, van der Ende, & Verhulst, 2003).
Similarly, at the international level there is an abundance of research based on the CBCL in the clinical as well as research contexts. These studies cover a variety of topics including epidemiology and risk factors for different emotional and behavioural problems, use of mental health services, treatment planning and outcome monitoring, as well as investigating psychological problems among children with different medical conditions (Achenbach & Rescorla, 2007; Fung & Tsang, 2006; Liu, Kurita, Guo, Miyake, Za, Ca et al., 1999; Rodenburg, Stams, Meijer, Aldenkamp, & Dekovic, 2005; Tick, van der Ende, & Verhulst, 2008). Although the CBCL is used in different domains, its demonstrated reliability and validity, ease of administration and scoring as well as applicability to cross-cultural samples have also contributed to its use as a measure to assess the epidemiology of emotional and behavioural problems around the world. At present there are multitudinous studies in which the CBCL has been used to investigate bicultural and multicultural similarities and disparities in the prevalence of emotional and behavioural problems (Crijnen et al., 1997; Rescorla et al., 2007).
Cross-cultural use of the CBCL.
In terms of the historical evolution of the cross-cultural use of the CBCL in epidemiological studies, Verhulst and his colleagues were the first to introduce the CBCL to Europe in 1982. They provided a well-designed epidemiological study of Dutch children, which included a standardization of the CBCL and the teacher report form. This was followed by the use of the CBCL across numerous countries in Europe and across the world. At present, the CBCL has been translated into more than 85 languages and there are over six thousand published studies from 72 countries (Berube & Achenbach, 2008). These studies are based on samples that have been collected from societies that share similar social structures, values, and views of child psychopathology, but differ in language and culture as well as societies that are drastically different in all aspects. Examples of countries where the CBCL has been used to investigate epidemiological data include Algeria (Petot, Petot, & Achenbach, 2008), China (Liu et al., 1999), Germany (Barkmann & Schulte-Markwort, 2005), Greece (Roussos, Karantanos, Richardson, Hartman, Karajiannis, et al., 1999), Israel (Zilber, Aurerbach, & Lerner, 1994), Jamaica (Lambert, Knight, Taylor, & Achenbach, 1994), Lithuania (Zukauskiene, Ignataviviene, & Daukantaite, 2003), Russia (Slobodskaya, 1999), Saudi Arabia (Abdel-Fattah, Asal, Al-Asmali, Al-Helali, Al-Jabban, & Arafa, 2004), Thailand (Weisz et al., 1987), and Turkey (Dumenci, Erol, Achenbach, & Simsek, 2004). In collaboration with other researchers, Achenbach has also developed national norms in the United States as well as in Australia, China, France, Greece, Israel, and Puerto Rico among others (Bilenberg, 1999). Moreover, extensive research has been conducted comparing data between two cultures as well as among multiple cultures simultaneously.
Although existing theoretical models of cultural influence provide a good basis for conceptualizing cross-cultural similarities and disparities, current studies have focused on exploring a host of environmental risk factors within different cultures to assess their association with emotional and behavioural problems (Kohrt, Kohrt, Waldman, Saltzman, & Carrion, 2004). The study of environmental factors is salient for the following two reasons. First, to investigate the direct impact of environmental factors and second to examine the influence of culture mediated through environmental factors such as psychosocial processes (e.g., parenting style, socialization) related to children's functioning and psychopathology (Weisz et al., 1987). Since the current study focuses on cross-cultural similarities and disparities, it is essential to address the theoretical foundations of environmental influences.
Theoretical Basis for Cultural Variations
Based on Bronfenbrenner (1979) and Belsky's (1980) ecological model as well as Cicchetti and Rizley's (1981) transactional model, the ecological-transactional model provides a sound theoretical framework to conceptualize the influence of various environmental factors, as it views the development of a child within a particular context and also provides a heuristic to understand the interplay of multiple factors. According to the ecological-transactional model, children function within multiple levels or ecologies in an environment, where factors at each level may influence each other as well as child development. These levels or ecologies can be described as a set of nested structures, with each one at different levels of proximity to the child. These ecologies include the macrosystem, the exosystem, the microsystem and the ontogenic level. The distal ecologies include the macrosytem, which contains the beliefs and values of the culture, and the exosystem, which includes aspects of the community in which the children and families reside. The more proximal ecologies are the microsystem, which comprises the immediate setting in which the individual lives, such as the family, school and peers, while onotogenic development consists of factors within the individual that contribute towards development and adaptation. Due to the transactional nature of the factors, factors at one level may influence outcomes in surrounding levels thereby determining the degree of risk posed to the individual. Moreover, at each level there exists an array of enduring and transient vulnerability and protective factors (Cicchetti & Toth, 1997). Therefore, it is important to investigate the relationship between factors at each level (e.g., children's characteristics, family, community and cultural factors) to assess children at risk for emotional and behavioural disorders (Forness, 2003).
At the ontogenic or individual/child level, physiological and medical factors (e.g., premature birth, low birth weight, and slow neurological development) are considered risk factors for emotional and behavioural disorders (Bhutta, Cleves, Casey, Craplock, & Anand, 2002; McCormick, McCarton, Brooks-Gunn, Belt, & Gross, 1998). At the microsystem level, family factors such as family structure (e.g., single or two parent households) as well as family functioning (e.g., parental marital discord, child maltreatment as well as parenting style) are considered strong predictors of problem behaviour (Harland et al., 2002; Turk et al., 2007; Vostanis, Graves, Meltzer, Goodman, Jenkins, & Brugha, 2006). Detailed analysis reveals that it is not family structure (i.e., single or divorced families vs. intact families) per se that leads to emotional and behavioural problems. In fact, it is factors such as economic hardship linked to being a single parent as well as the probability of there having been a period of disharmonious family relationships leading to separation or divorce, when child care was impaired, that may cumulatively increase the risk of emotional and behavioural problems (Turk et al.). In terms of childcare, parental warmth and sensitivity along with consistent control and structure are crucial for healthy development of children. Children raised by authoritarian parents who are harsh and rigid in enforcing rules are more likely to exhibit psychological problems than children of authoritative parents who, along with being nurturing and responsive are able to reason with their children (Webster-Stratton & Hammond, 1999). Researchers also state that parental mental health is also linked to the development of emotional and behavioural problems among children (Papp, 2004; Steinhausen, Mas, Ledermann, & Metzke, 2006). There is also substantial evidence linking SES of the family and emotional and behavioural problems (Bradley & Crowyn, 2002). Researchers suggest that children from low SES families tend to manifest symptoms of psychological problems and maladaptive social functioning more often than their affluent counterparts (Brooks-Gunn & Duncan, 1997; McCoy, Firck, Loney & Ellis, 1999).
At the exosystem or community level, impoverished neighbourhoods, violence, crime, and lack of social support have been identified as risk factors for emotional and behavioural problems (Jaffee et al., 2005; Mazza & Overstreet, 2000; Turk et al., 2007). It is pertinent to highlight that community factors directly as well as indirectly (i.e., through influencing the family) influence children's psychological development (Jaffee et al., 2005). For example, exposure to violence in the community may directly influence the child's perceptions about safety as well as emotion regulation (Margolin & Gordis, 2000), thereby increasing the risk of maladaptive behaviour. Moreover, chronic exposure to violence may desensitize children to Aggressive Behavior or may reinforce the belief that aggressive acts are acceptable and effective methods of coping, both of which lead to an increase in children's aggression levels (Schwartz & Proctor, 2000). Exposure to violence may also indirectly influence children's psychological functioning as it is associated with family conflict and parental distress, which in turn have a negative impact on children's mental health (Linares, Heeren, Bronfman, Zuckerman, Augustyn, & Tronick , 2001; Overstreet & Cerebone, 2005). Furthermore, stressful life events at the family or community level have been associated with emotional and behavioural problems among children (Harland et al., 2002; Mesman & Koot, 2001; Turk et al.). These stressful events can include illnesses, hospitalization or death of a family member, divorce, parental loss of employment, community violence, and the birth of a sibling, all of which have direct as well as indirect influences on children's social and emotional development.
At the cultural level, two models have been offered to explain how cultural values and beliefs may influence children's development and psychopathology. Although the models have been derived from cross-cultural studies between Thai and American children only, they provide a basis to conceptualize cultural influence. These models include the Problem Suppression-Facilitation model and the Adult Distress Threshold Model, which describe how cultural beliefs influence children's problems and adults' perspectives to children's problems, respectively. According to the Problem Suppression-Facilitation model (Weisz et al., 1987), culturally mediated beliefs, values, and traditions through associated child-rearing and socialization practices may help shape both the occurrence and maintenance of different types of children's problems. Cultural values may influence specific problems by suppressing (e.g., by punishment or social pressure) the development of behaviours that are disapproved and encouraging (e.g., by teaching, modelling or rewarding) the development of behaviours that are acceptable in the culture. The second model, namely the Adult Distress Threshold model (Weisz et al., 1987), has two forms. The general form of the model proposes that culture influences adults' attitudes towards children's behaviour in general, helping to determine how distressing the child's behaviour will be to adults and the likelihood of adults seeking help through clinical intervention. The pattern-specific form proposes that cultures will vary with certain problems raising more concern in some cultures than others.
Weisz and his colleagues conducted multiple studies to explore how culture may influence differences in the reporting of emotional and behavioural problems between Thai and American parents. Weisz, McCarty, Eastman, Chaiyasit, & Suwanlert (1997) provide a methodical synthesis of their logical line of inquiry encompassing multiple studies and utilizing diverse methodological approaches. Based on their initial study, Weisz et al. concluded that among children and adolescents referred to mental health clinics, Thai parents reported more Internalizing Problems whereas American parents reported more Externalizing Problems. They attributed these findings to cultural differences in terms of Thai culture being based on values that encourage self-control, emotional restraint, and strict compliance with rules. These cultural values may have contributed to children developing more Internalizing Problems as opposed to Externalizing Problems, thus supporting the Problem Suppression-Facilitation model. Based on subsequent research, Weisz et al. (1987) and Weisz, Sigman, Weiss, and Mosk (1993) stated that consistent with the clinical population even in the general population, Thai children had significantly more Internalizing Problems than U.S. children. However, the reverse did not hold true for Externalizing Problems, as there were no significant cultural differences. Weisz and colleagues (1997) did not provide an explanation for the lack of significant differences with regard to Externalizing Problems, but explained the differences in Internalizing Problems by stating that Buddhist values, which encourage self-control and emotional restraint, may have contributed towards Thai children expressing distress in a manner that is socially acceptable. These findings provide partial support for the Problem Suppression-Facilitation model as the results hold true for Internalizing but not Externalizing Problems.
To test the Adult Distress Threshold model, Weisz and colleagues (1991) assessed the likelihood of refer-ability of Internalizing and Externalizing Problems. In general, American parents compared to their Thai counterparts referred children more often for both Internalizing as well as Externalizing Problems. Within each culture, Internalizing Problems were more often the reason for referral in Thailand whereas the converse was true in the U.S. (i.e., Externalizing Problems were more often the reason for referral in the U.S.). The higher degree of refer-ability of Internalizing Problems among Thai parents is in contrast to the pattern-specific form of the Adult Threshold Model. The pattern-specific form of the model proposes that, based on Thai notions of social self-restraint, Thai adults may have a lower threshold of Externalizing Problems leading to a higher referral of Externalizing Problems as opposed to Internalizing Problems. Weisz et al. (1997) concluded that since Externalizing Problems are highly unacceptable and in violation of Thai social norms, parents may be embarrassed to disclose such problems, admitting via clinical referral their inability to prevent such problems.
Weisz et al. (1997) also stated that overall differences in refer-ability (i.e., American parents referring children more often for both Internalizing and Externalizing Problems than Thai parents) may be linked to distinct adult attitudes towards children's problems in general. Research findings indicated that consistent with some literature on Thai Buddhist values (Jumsai, 1980), Thais (as compared to Americans) rated both Internalizing and Externalizing Problems to be less serious, less worrisome, less likely to reflect personality traits and more likely to improve with time. These cultural differences support the general form of the Adult Distress Threshold model, which proposes that cultural values influence adults' response to children's problems. Weisz et al. stated that cultural differences might also be attributed to adult beliefs regarding aetiology and remediation of children's problems. Thai parents were more likely to attribute the problem to faulty child rearing, socialization or teaching, whereas American parents considered child personality traits or psychodynamic attributions (e.g., the child having internal conflicts) as a cause of the problems. Similarly, in remediation, Thais preferred verbal interventions (e.g., “talk to the child”, “reassure the child”) while Americans favoured more behavioural approaches regarding reward and punishment.
Although adults' perspective or threshold plays an important role in determining the reporting of symptoms, Weisz et al. (1991) concluded that refer-ability may also be influenced by other cultural differences. These differences include the extent to which a problem is thought to require professional intervention as opposed to remediation within the home, the availability and capability of specialized professionals, the willingness of parents to disclose problems that might otherwise be hidden at home, and the level of exposure to child psychology.
The American-Thai comparison studies illustrate that although there is insufficient support for the two models of cultural influence, yet exploring their association with emotional and behavioural problems has uncovered a host of salient cultural factors. These cultural factors go beyond the two cultural models and highlight the need to investigate the relationship between different environmental factors and emotional and behavioural problems. A deeper understanding of these factors is essential to interpret cross-cultural research studies and develop a comprehensive understanding of cultural similarities and disparities.
Bicultural CBCL comparisons.
Numerous researchers have investigated bi-cultural variations between U.S. and other cultures including China (Weine, Phillips, & Achenbach, 1995), France (Stanger, Fombonne, & Achenbach, 1994), Greece (MacDonald, Tsiantis, Achenbach, Stefanidi, & Richardson, 1995), the Netherlands (Verhulst, Achenbach, Ferdinand, &
Cite This Dissertation
To export a reference to this article please select a referencing stye below: