Chapter II: Literature Review
As suggested in the introduction, numerous researchers have explored the prevalence of emotional and behavioural problems across the globe. Researchers have also investigated correlates (e.g., age and gender) associated with emotional and behavioural problems. The psychometric properties of instruments assessing emotional and behavioural problems have also been a subject of interest. In addition, researchers have also investigated cross-cultural similarities and disparities among emotional and behavioural problems. The extensive literature that addresses these issues, and which also helped formulate the rationale for the current study, is presented in five sections. The first section highlights the problems associated with epidemiological studies and compares the two main approaches to epidemiological studies, namely the categorical and the empirical approach. The second section provides a detailed description of the CBCL including the evolution of the measure, its psychometric properties, its advantages and disadvantages, as well as its range of applicability. The third section provides a description of the theoretical rationale for assessing cultural similarities and disparities associated with emotional and behavioural problems. Multicultural findings based on the CBCL as well as age and gender differences associated with emotional and behavioural problems are also reported. The fourth section consists of a review of the various processes involved in assessing the psychometric properties of instruments and findings based on psychometric properties of the various translations of the CBCL. The fifth section consists of a brief cultural and socio-political description of Pakistani society followed by a description of the salient features (i.e., family, community and cultural factors) in relation to emotional and behavioural problems in Pakistani society. Finally, there is a description of the objectives of the current study.
Epidemiology of Emotional and Behavioural Problems
Current reviews of epidemiological studies indicate that there is a high prevalence of emotional and behavioural problems among children and adolescents around the world (Costello et al., 2004; Hackett & Hackett, 1999; Waddell et al., 2002). In one review, Costello et al. compared findings across several developed countries (including Canada, the United States, the United Kingdom, Germany and Australia) to investigate the prevalence of emotional and behavioural problems as well as that of other psychological problems. Based on their findings, the overall prevalence rates of psychological problems among children and adolescents had a very broad range (0.1% to 42%), with varying rates for each category of disorder. Categories include disruptive behaviour disorders (i.e., conduct disorder, oppositional disorder and attention deficit hyperactivity disorder), mood disorders (i.e., major depressive disorder and bipolar disorder), anxiety disorders (i.e., phobias, generalized anxiety disorder, obsessive compulsive disorder, and post-traumatic stress disorder) as well as substance abuse and dependence. A critical examination of the studies included in the review revealed that variations in prevalence rates may be attributed to methodological flaws such as substantial disparity across studies with regard to sample size and the age range assessed. Moreover, differences across studies in terms of the measures used, the criteria employed as well as the type of informant may also have influenced the findings.
In contrast to Costello et al.'s (2004) review, Waddell et al.'s (2002) review was based on more stringent criteria; studies based on samples of similar size and age range, as well as using similar methodology were compared. Based on Waddell et al.'s review, the prevalence rates of emotional and behavioural problems varied between 10% and 20%. Although findings from both reviews vary considerably, the prevalence rates of emotional and behavioural problems across developed countries is still high and warrants serious attention. Moreover, methodological disparities across studies underscore the need for a uniform methodology to investigate the prevalence of emotional and behavioural problems.
In contrast to developed countries, there are few researchers investigating prevalence rates in developing countries (e.g., Bangladesh, India, Sri lanka, Sudan, and Uganda) (Costello, 2009: Fleitlich-Bilyk & Goodman, 2004; Mullick & Goodman, 2005; Nikapota, 1991; Prior, Virasinghe, & Smart, 2005). Moreover, there is a scarcity of reviews of the existing studies. In one review, Hackett and Hackett (1999) compared results from India, Puerto Rico, Malaysia and Sudan, and the prevalence rates of psychological disorders ranged from 1% to 49%. Similar to research in developed countries, researchers attribute variations in findings to methodological problems across studies, which include an inadequate sample size, paucity of explicit and internationally accepted diagnostic criteria, as well as inconsistencies in assessment procedures (Fleitlich-Bilyk & Goodman, 2004). Moreover, prevalence rates among developing countries may also partly be linked to the social, economic and medical environment. For example, lack of medical resources and awareness about psychological problems may result in parents not knowing how to seek help (Gadit, 2007). Social taboos further compound the problem, preventing people from reporting problems and deterring help-seeking behaviour (Samad, Hollis, Prince, & Goodman, 2005). More importantly, cultural variations in the conceptualization and identification of psychological problems may result in varied reporting of symptoms (Gadit, 2007). These environmental differences and methodological inconsistencies across studies emphasize the need for a cross-culturally robust methodology to investigate the prevalence of emotional and behavioural problems.
Along with methodological problems and environmental differences, emotional and behavioural problems merit investigation because they affect multiple aspects of children's functioning such as academic performance and social adjustment (Montague et al., 2005; Nelson et al., 2004; Vitaro et al., 2005). Researchers also state that there is high comorbidity among emotional and behavioural problems, (SteinHausen, Metze, Meier, & Kannenberg, 1998) which creates multiple problems for children and their caregivers. Moreover, many childhood disorders continue and influence functioning during adulthood. In fact, many adult disorders are now recognized as having roots in childhood vulnerabilities (Maughan & Kim-Cohen, 2005; Tremblay et al., 2005). Furthermore, recognizing and treating problems early can reduce the burden of the enormous human and financial costs associated with the assessment and intervention, especially in countries where resources are scarce (Costello, Egger, & Angold, 2005; James et al., 2002; Waddell et al., 2002). In addition, cross-cultural epidemiology of children's emotional and behavioural problems may also better inform current knowledge about the characteristics, course, and correlates of such problems, which in turn provide a scientific basis for appropriate mental health planning (Achenbach & Rescorla, 2007; Waddell et al.). Therefore, there is a strong need for a methodology that can be utilized for clinical as well as research purposes to assess emotional and behavioural problems among children and adolescents across cultures.
Current literature indicates that there are two main approaches to investigate the epidemiology of emotional and behavioural problems, namely the categorical and the empirical approach. There are several differences in both approaches including conceptualization of psychological problems as well as the methodology employed for their assessment. Both approaches will be discussed briefly.
The categorical approach. The categorical approach, based on the biomedical perspective, views psychological problems as a group of maladaptive and distressing behaviours, emotions and thoughts which are qualitatively different from the typical (Cullinan, 2004). That is, similar to medical diseases, an individual may or may not have a specific psychological disorder. Traditional epidemiological studies are based on the categorical approach as embodied in various editions of the Diagnostic and Statistical Manual for Mental Disorders (DSM) (American Psychiatric Association (APA), 1980; 1987; 1994; 2000) and the International Classification of Diseases (WHO, 1978; 1992). Examples of instruments used in traditional epidemiological studies to derive DSM diagnoses include the Diagnostic Interview Schedule for Children (DISC) (Costello, Edelbrock, Kalas, Kessler, & Klaric, 1982) and the children's version of the Schedule for Affective Disorders and Schizophrenia (Kiddie-SADS) (Puig-Antich & Chambers, 1978). At present, there is considerable debate about the validity of epidemiological studies based on the categorical approach. Researchers have highlighted that inconsistencies in prevalence rates may be due to conceptual and methodological issues linked with the DSM as well as methodological disparities among studies (Achenbach & Rescorla, 2007; Waddell et al., 2002). Each of these factors will be discussed briefly.
DSM related problems. Multiple conceptual and methodological problems are associated with the DSM. First, the DSM does not provide a methodology to operationally define different psychological disorders (Widiger & Clark, 2000). To operationally define DSM criteria, various diagnostic interviews such as the DISC have been developed. Unfortunately, meta-analyses indicate that the diagnoses based on the DISC and other diagnostic interviews are not in agreement with diagnoses made through comprehensive clinical interviews, which indicate that, neither diagnostic nor clinical interviews provide good validity criteria for testing DSM categories (Achenbach, 2005; Costello et al., 2005; Lewczyk et al., 2003). Second, the diagnostic categories and criteria provided in the DSM continue to change as reflected in the changes across the various editions of the DSM, namely the third edition (APA, 1980), third edition revised (APA, 1987), fourth edition (APA, 1994), and fourth edition text revised (APA, 2000), making comparisons across editions problematic (Achenbach, 2005). Third, although the current version, known as the DSM-IV-text revised (APA, 2000), aims at introducing cultural sensitivity in assessment and diagnoses by including an “outline for cultural formulation and a glossary of culture-bound syndromes” (APA., 2000, pg. 897), it does not provide criteria or guidelines regarding the use of the classification system with specific cultural groups (Paniagua, 2005). Since many of the DSM diagnostic criteria are based on Euro-American social norms, it is difficult to use the DSM criteria to identify psychopathology in individuals from other cultures.
In addition, there is growing consensus among researchers that DSM categories need to be more appropriate for children and adolescents of different ages and gender (Doucette, 2002; Segal & Coolidge, 2001). Turk et al. (2007) also highlight the saliency of factors such as age and gender when investigating prevalence rates. However, at present, this is not the case. Costello et al. (2005) have stated that the constant developmental changes of childhood create the need for an age- and gender- specific approach to epidemiology.
Before incorporating a developmental perspective in epidemiological studies, it is essential to have a better understanding of developmental psychopathology. Developmental psychopathology is based on the view that problems arise from different causes, manifest themselves differently at each stage, and may have diverse outcomes. Developmental psychologists do not support a specific theory to explain all developmental issues. Instead, they try to incorporate knowledge from multiple disciplines (Cicchetti & Dawson, 2002). Moreover, developmental psychopathology also includes an analysis of the existing risk and protective factors within the individual and also in his/her environment over the course of development (Cicchetti & Walker, 2003).
According to Costello and colleagues (2004), a developmental perspective in epidemiological studies is based on the inclusion of certain principles. First, precise assessment measures for the different phases in childhood and adolescence are required to compare children's functioning with that of their same-age peers. For example, problems such as fear of dark places is considered typical for 6-year-olds but not for 12-year-olds. Furthermore, the developmental perspective would include longitudinal studies to evaluate the ways in which developmental processes influence the risk of specific psychological disorders. For example, the developmental trajectory of physical aggression is such that there is an increase in Aggressive Behavior during the first few years of childhood, but it progressively decreases until adulthood (Tremblay et al., 2004). Moreover, developmental epidemiology would include frequent assessments to determine the onset of disorders. Frequent assessments would also assist in the identification of environmental and individual factors that contribute to the development of psychopathology. Although the developmental perspective emphasises the need for age- and gender-specific diagnostic criteria, longitudinal studies as well as frequent assessments, it is difficult to incorporate this perspective in studies based on the categorical approach as it is not sensitive to developmental changes.
Methodological disparities. A critical analysis of categorically based epidemiological studies reveals multiple methodological problems. These include inconsistencies in assessment and sampling procedures as well as absence of guidelines about using data from multiple sources. In terms of assessment procedures, both symptoms as well as significant impairment are required to identify children with disorders. This is corroborated by Costello et al. (2004), who report that the disparity in the prevalence rates of phobias (i.e., 0.1% to 21.9%) may be attributed to how phobias were assessed in each study, in particular, whether both symptoms (e.g., fear of open places, snakes) as well as significant functional impairment were taken into account in the identification of phobias. Waddell et al. (2002) state that the use of standardized measures has lead to an improvement in the assessment of symptoms; however, problems still exist with regard to how impairment is gauged or how measures may be combined to include symptoms as well as impairment. Another problem with assessment procedures is that different interview schedules (e.g., DISC and the Kiddie-SADS) and DSM editions have been used across studies, which may have contributed to differences in prevalence rates.
Incompatible sampling procedures may also have led to disparities in overall prevalence rates in categorically based epidemiological studies (Waddell et al., 2002). For example, studies such as the Great Smokey Mountains study (Costello, Angold, Burns, Erkanli, Stangel & Tweed, 1996) were relatively more comprehensive, and investigated a larger number of diagnostic categories than other studies. As a result, higher overall prevalence rates of psychological problems were reported compared to studies that did not assess as many disorders. Another sampling issue is that reviews were based on studies that differed with regard to the age range assessed; some studies focused on a younger age bracket (i.e., between 8 to 11 year olds), others on an older age bracket (i.e., 11 years and older), whereas some researches included a very broad age range (i.e., 6 to 17 year olds). In addition, there were inconsistencies across studies in terms of the type of informant used; some studies relied on parents only, some on children, while some combined data from parents, children as well as teachers. Differences in the age brackets assessed as well as the use of different informants may have contributed to disparities in epidemiological findings.
Another salient issue with regard to categorically based epidemiological studies concerns the coordination and interpretation of information from multiple informants. Since problem behaviours may only occur in specific situations or with specific individuals, multiple informants (e.g., teachers, parents and children) are necessary. However, since the respondent's context and perception have a great impact on the identification of psychological problems, poor agreement among respondents is frequently reported. For example, children normally report higher rates of internalizing symptoms (e.g., anxiety and depression) while parents tends to report higher rates of externalizing symptoms (e.g., Conduct Problems) (Rubio-Stipec, Fitzmaurice, Murphy, & Walker, 2003). Additionally, children are not considered reliable reporters of their own behaviour due to differences in cognitive abilities as well as the ability to report their own behaviour (Achenbach & McConaughy, 2003). Despite such findings, the categorical approach does not provide guidelines regarding obtaining and interpreting data from multiple sources, which complicates matters in terms of how to combine data into yes-or-no decisions about different symptoms.
The various conceptual problems associated with the DSM as well as the methodological flaws in epidemiological reviews highlight the problems associated with using the categorical approach as a basis for epidemiological studies. Moreover, these issues underscore the need for an approach that is methodologically sound and culturally appropriate for cross-cultural comparisons. An alternative to problems linked to the categorical approach, where an a priori criterion is imposed, can be a system that is empirically based and identifies problems as they occur in a population. Such an approach would be helpful in highlighting cultural differences in the manifestation of different emotional and behavioural problems. Moreover, there is also a need for a methodology that can be employed in a standardized, systematic fashion. Although the empirical approach is not a panacea for problems associated with epidemiological studies, it does provide solutions to some of the types of errors in the categorical system.
Empirical or dimensional approach. The empirical or dimensional approach, in accordance with a psychosocial perspective, views mental health as a continuum. The dimensional perspective supports the notion that all individuals experience problems involving behaviours, emotions and thoughts to varying extents. Those who experience such problems to an extreme extent (unusual frequency, duration, intensity, or other aspects) are more likely to have a psychological disorder (Cullinan, 2004). In contrast to imposing a priori criteria on children's emotional and behavioural problems, the empirical approach identifies problems as they present themselves in the population. According to Cullinan (2004), there are certain steps involved in developing a dimensional classification system for emotional and behavioural problems. These steps include creating a collection of items that reflect measurable problem behaviours experienced by children, identifying a group of children to be studied, assessing every child in the group on each problem, and investigating the data to identify items that co-vary, thus leading to the identification of different dimensions or factors. After the dimensions have been derived, the pool of items can be used to assess and classify emotional and behaviour problems among new populations. Given that the empirical approach is based on the identification of co-occurring problem behaviours in the population, instead of imposing a priori criteria, it is a favourable approach for cross-cultural epidemiological studies.
Within empirical approaches, the Achenbach System of Empirically Based Assessment (ASEBA) provides a good framework for epidemiological studies for multiple reasons. First, being empirically based, ASEBA identifies emotional and behavioural problems as they occur in the population. Second, it is based on a developmental perspective, has a uniform methodology, and also provides explicit guidelines about using data from multiple sources (Achenbach & McConaughy, 1997; Achenbach & Rescorla, 2001). Hence it provides solutions to problems that arise in the categorical approach. Moreover, Cullinan (2004) and Krol et al. (2006) state that ASEBA measures have been used more extensively compared to other measures of emotional and behavioural problems, such as the Conners Rating Scale- Revised (Conners,1990) and the Strengths and Difficulties Questionnaire (Goodman, 1997). Achenbach system of empirically based assessment (ASEBA).
Although the ASEBA has a non-theoretical, empirical base per se, it is greatly influenced by the principles of developmental psychopathology. For example, Achenbach highlights that problems may include thoughts, behaviours, and emotions that may manifest themselves differently depending on the age and gender of the individual (Greenbaum et al., 2004). Therefore, each ASEBA form provides norms based on the age and gender of the child, which enables an individual's functioning to be assessed in comparison to same-age peers. Furthermore, ASEBA is a multiaxial system that encompasses a family of standardized instruments for the assessment of behavioural and emotional problems as well as adaptive functioning. The five axes of the assessment model include parent (Axis I) and teacher (Axis II) reports, cognitive (Axis III) and physical (Axis IV) assessments as well as the direct assessment of children (Axis V) (Achenbach & McConaughy, 2003). The use of different ASEBA instruments provides a standardized and uniform methodology to incorporate information from multiple sources.
Furthermore, all ASEBA instruments are empirically based. In accordance with the empirical approach, the construction of the ASEBA forms involved a series of steps (Achenbach & McConaughy, 2003). Initially, a collection of potential symptom behaviours (i.e., items) was derived from multiple sources. These items were operationally defined in such a manner that respondents not trained in psychological theory could use them. In accordance with general item-development procedures, pilot tests were conducted to evaluate the clarity of items, response scales and item distribution. Finally, items that could differentiate between individuals who were not functioning well and their well functioning same-age peers were retained. Multivariate statistical analyses were applied to the retained items in order to identify syndromes of problems that co-occur. Syndromes were identified purely on the basis of co-occurrence, without any link to a particular cause. Subsequently, the syndromes of co-occurring problem items were used to construct scales. These scales were used to assess individuals in order to assess the degree to which they exhibit each syndrome. Since all ASEBA instruments are empirically based, findings can be compared on the basis of the manifestation of different emotional and behavioural problems, thereby providing a clearer picture of cross-cultural similarities and disparities of different emotional and behavioural problems.
In terms of the historical evolution of the system, ASEBA originated to provide a more differentiated assessment of child and adolescent psychopathology than the DSM. When ASEBA was developed, the first edition of the DSM (APA, 1952) had only two categories for childhood disorders, which included adjustment reactions of childhood and schizophrenic reaction childhood type (Achenbach & Rescorla, 2006). In contrast to the DSM, the first ASEBA publication highlighted more syndromes of emotional and behavioural problems (APA, 1952). Moreover, based on factor analyses, Achenbach (1966) identified two broad groupings of problems for which he coined the terms “Internalizing” and “Externalizing.” As described earlier, Internalizing Problems included problems with the self, such as anxiety, depression, withdrawal, and Somatic Complaints, without any apparent physical cause. On the other hand, Externalizing Problems included problems with other people, as well as problems linked to non-conformance to social norms and mores, such as aggressive and delinquent behaviour. Although all ASEBA forms are used extensively in clinical and research environments, the Child Behavior Checklist is the most widely recognized measure for the assessment of emotional and behavioural problems (Greenbaum et al., 2004; Webber & Plotts, 2008).
Child Behavior Checklist
An essential part and the cornerstone of Achenbach's multiaxial, empirical system is the Child Behavior Checklist (CBCL). Although the CBCL assesses social competencies as well as problem behaviours, it is widely recognized as a measure of emotional and behavioural problems as opposed to social competencies. In fact, researchers suggest that the CBCL is the most extensively utilized measure for the assessment of problem behaviours among children and adolescents as observed by their parents and caregivers (Krol et al., 2006; Greenbaum et al., 2004).
Although there have been multiple revisions to the initial CBCL, all versions have the same format and consist of two distinct sections. The first section measures social competencies. Parents are asked to respond to 20 questions regarding the child's functioning in sports, miscellaneous activities, organizations, jobs and chores, and friendships. Items also cover the child's relations with significant others, how well the child plays and works alone, as well as his/her functioning at school. Finally, respondents describe any known illnesses or disabilities, the issues that concern them the most about the child, and the best things about the child (Achenbach & Rescorla, 2006). The second section assesses problem behaviour and consists of 118 items that describe specific emotional and behavioural problems, along with two open-ended items for reporting additional problems. Examples of problem items include “acts too young for age”, “cruel to animals”, “too fearful or anxious”, and “unhappy, sad or depressed”. Problem behaviours are organized in a hierarchical factor structure that consists of eight correlated first-order or narrowband syndromes, two correlated second-order or broadband factors (i.e., Internalizing and Externalizing Problems) and an overall Total Problems factor. Parents/caregivers are asked to rate the child with regard to how true each item is at the time of assessment or within the past 6 months. The following scale is used: 0 = not true (as far as you know), 1 = somewhat or sometimes true, and 2 = very true or often true. In the case of respondents with poor reading skills, a non-clinically trained clincian can also admisnter the CBCL (Achenbach & Rescorla, 2006). For respondents who cannot read English but can read another language, translations are available in over 85 languages (Berube & Achenbach, 2008).
Development of the CBCL.
The first version of the CBCL dates back to 1983. To date, there have been two revisions of the CBCL; the first one in 1991 followed by the second in 2001, leading to considerable improvements in the measure. The main weakness of the initial CBCL was that comparisons across different age groups and respondents were problematic since syndromes had the same names but different items across different age forms (i.e., 4 to 5, 6 to 11, 12 to 16 years) as well as across different respondent forms (i.e., CBCL, teacher report form [TRF], and the youth self report [YSR]) To rectify the problem, the 1991 version included two new types of syndromes, the core and cross-informant syndromes. Core syndromes represented items that clustered together consistently across age and gender groupings on a single instrument. Cross-informant syndromes were based on those items from the core syndromes that appear on at least two of the three different respondent forms (i.e., CBCL, TRF, and YSR) (Greenbaum et al., 2004). These revisions facilitated comparisons across different age groups and informants. Moreover, the 1991 version of the CBCL also had new national level norms, which included norms for seventeen and eighteen year olds. Apart from practical benefits, changes such as a broader age range and precise criteria for different developmental levels, genders and type of respondents, helped make the CBCL and ASEBA instruments more accurately representative of the developmental perspective of child psychopathology (Greenbaum et al.).
Achenbach (1991) also conducted exploratory principal factor analyses of the syndrome scales. Based on the loadings of different syndromes, Achenbach identified Anxious/Depressed, Withdrawn, and Somatic Complaints as indicators of Internalizing Problems, whereas Aggressive and Delinquent Behavior were identified as indicators of Externalizing Problems. Since Social Problems, Thought Problems and Attention Problems did not load consistently on either second-order factor, they were not placed in any group (Achenbach, 1991; Greenbaum et al., 2004). Although Internalizing and Externalizing Problems identify different types of behaviour, the two categories are not mutually exclusive and may co-occur within the same individual. This is supported by research findings that indicate that there was a correlation between the two groups in both clinic-referred (.54) and non-referred (.59) samples matched on the basis of age, sex, race, and income (Achenbach, 1991).
Description of the current CBCL.
The current CBCL was published in 2001 and covers ages 6 to 18 years (CBCL/6-18; Achenbach & Rescorla, 2001). The CBCL/6-18 (Achenbach & Rescorla, 2001) provides raw scores, T- scores and percentiles for the following: (1) the three competence scales (Activities, Social, School); (2) the Total Competence scale; (3) the eight cross-informant syndromes; (4) Internalizing and Externalizing Problems and (5) Total Problems. The cross-informant syndromes of the CBCL/6-18 include Aggressive Behavior, Anxious/Depressed, Attention Problems, Rule-Breaking Behavior, Social Problems, Somatic Complaints, Thought Problems, and Withdrawn/Depressed.
As far as similarities and differences from previous versions are concerned, the current CBCL introduced some major and a few minor changes. One major change was the introduction of the DSM-oriented scales, based on which CBCL and other ASEBA forms can now be scored in terms of scales that are oriented toward categories of the fourth edition of the DSM (A.P.A., 1994). The introduction of the DSM-oriented scales has combined the categorical and empirical approaches and enables users to view problems in both the categorical and dimensional approaches (Achenbach, Dumenci & Rescorla, 2003; Achenbach & Rescorla, 2006). The DSM-oriented scales include six categories, namely Affective Problems, Anxiety Problems, Somatic Problems, Attention Deficit/Hyperactivity problems, Oppositional Defiant Problems as well as Conduct Problems. These scales are based on problem items that mental health experts from sixteen cultures across the world rated as being consistent with particular DSM diagnostic categories. Similar to the empirically based syndromes, the DSM- oriented scales also have age-, gender- and respondent-specific norms.
Another major change was that new normative data was collected using multistage probability sampling in forty U.S. states as well as the District of Columbia. The selected homes were considered to be representative of the continental United States with respect to geographical region, socio-economic status, ethnicity and urbanization (Achenbach & Rescorla, 2001). Moreover, complex new analyses based on new clinical and normative samples were conducted. However, the eight syndromes and Internalizing and Externalizing groupings published in 1991 were replicated with minor changes. Research findings indicated that correlations between scores on the 1991 syndromes and their 2001 counterparts ranged from .87 to 1.00 (Achenbach & Rescorla, 2001).
In terms of minor changes, a few items were replaced by newer ones and the wording of some was improved. Moreover, all syndromes except two were given the same names. Specifically, the 1991 “Delinquent Behavior” syndrome was renamed “Rule-Breaking” Behavior as some of the behaviour problems could be present among children too young to be adjudicated. Due to cross-loading of items, the 1991 “Withdrawn” syndrome was replaced by “Withdrawn-Depressed”, as some problem behaviours indicative of depression were found to load on the same factor (Achenbach & Rescorla, 2006).
Psychometric characteristics of the CBCL.
Psychometric properties generally refer to different aspects of reliability and validity. Reliability or the stability of scores can be assessed in different ways. Reliability can be measured in terms of test-retest reliability, internal consistency or inter-rater reliability. According to Kline (2005), test-retest reliability measures the stability over time of a set of scores on a specific test for a given sample, which means that the same test is given to the same group of individuals at two different points in time. On the other hand, internal consistency assesses the stability of the scores across items and is based primarily on the inter-correlations between items. Although different methods of measuring internal consistency exist, Cronbach's alpha (α) is one of the most common of the internal consistency indices. Inter-rater reliability is the stability of scores across informants or judges and is measured on the basis of the correlation between two informants (Kline, 2005).
The ASEBA manuals present extensive data on the reliability of ASEBA measures (Achenbach & Rescorla, 2006). There is evidence of test-retest reliability, internal consistency, as well as inter-rater reliability of the CBCL 6/18 (Achenbach & Rescorla, 2001). The Bibliography of Published Studies Using the ASEBA (Berube & Achenbach, 2008) also lists several studies that report on the reliability of the CBCL and other ASEBA instruments. In research in the social sciences, reliability values of .70 or higher are considered acceptable, but in complex studies a few reliability coefficients between .60 and .69 are common and are considered marginally acceptable (Gliner, Morgan & Leech, 2009).
With regard to the CBCL, researchers indicate high test-retest correlations for the empirical syndromes, DSM-oriented scales, Internalizing, Externalizing and Total Problems as values range from .80 to .94 (Achenbach & Rescorla, 2006). Similarly, researchers indicate high internal consistency for each of the empirical syndromes (alpha value .78 to .94), DSM-oriented scales (.72 to .91), Internalizing, Externalizing and Total problem score (.90, .94, .97, respectively). In terms of inter-rater reliability, it would be appropriate to compare results across comparable raters/informants (e.g., fathers and mothers). Since parents and teachers have a different role, level of interaction and knowledge of the individual being assessed, it would not be appropriate to compare their results. Achenbach and Rescorla (2001) have reported inter-parent agreement on different scales of the CBCL. The inter-parent correlations of the empirically based syndromes range from .65 to .82 and the DSM-oriented scales range from .63 to .88. Internalizing, Externalizing and Total problem scores have inter-parent reliability values of .72, .85 and .80, respectively. Unlike the test-retest and internal consistency coefficients, a few of the inter-parent correlations lie between the .60 and .69 range, which is not ideal but is still marginally acceptable. In general, it can be stated that different reliability indices show that the CBCL results in reliable sample scores, and lend support to its use in clinical as well as research settings.
Validity refers to the extent to which the test measures what it is designed to measure. Similar to reliability, each aspect of validity is assessed in different ways. These include face, content, criterion, and construct validity (Kline, 2005). Face validity refers to superficial visual inspection of a test by sophisticated or unsophisticated reviewers and is considered a weak and unreliable measure of validity. Content validity is the degree of correspondence between an instrument's items and what the instrument was designed to measure. Criterion validity investigates the relationship between test scores with one or more external criteria known to measure the attribute under study (Urbina, 2004). Criterion validity has two forms: predictive and concurrent validity. Predictive validity involves the use of future performance of the criterion whereas concurrent validity measures the criterion at about the same time. Construct validity refers to the degree to which a test may be interpreted to measure a specific construct or trait (Kline, 2005).
To assess content validity, Achenbach and Rescorla (2001) have conducted statistical analyses comparing referred and non-referred children matched on age and gender on each of the problem behaviour items. Results indicated that all items except “allergy” and “asthma” significantly discriminated between the two groups. Moreover, there was also evidence about particular items being identified by international panels of experts as being consistent with the DSM-IV diagnostic categories (Achenbach & Rescorla, 2000, 2001, 2003). In terms of concurrent criterion validity, research findings indicate that the CBCL results have significant correlations with findings based on behaviour assessment measures such as the Conners' Parent Rating Scales (Conners,1997), the Behavior Assessment System for Children (Reynolds & Kamphaus, 1992) and the Minnesota Multiphasic Personality Inventory Second Edition (MMPI-2; Butcher, Dahlstrom, Graham, Tellegen & Kaemmer, 1989) (Achenbach & Rescorla, 2006). Moreover, researchers have also documented predictive validity, including longitudinal studies indicating that the CBCL is predictive of outcomes such as substance abuse, referral for mental health services, and psychiatric diagnoses over long periods of time (Achenbach, Howell, McConaughy, & Stanger, 1998; Ferdinand, Blum, & Verhulst, 2001; Hofstra, van der Ende, & Verhulst, 2000). Moreover, the CBCL broadband scales accurately predicted subsequent psychopathology after five years (Petty, Rosenbaum, Hirshfeld-Becker, Henin, Hubley, LaCasse, Faraone & Biederman, 2009). Separately for each gender, multiple regression analyses of the scale scores on referral status, age, ethnicity, and socio-economic status (SES) have indicated that the CBCL/ 6-18 scale scores were associated with the referral status much more strongly than demographic variables (Achenbach & Rescorla, 2007). With regard to construct validity, researchers state that confirmatory factor analyses indicate a good theorized factor structure (i.e., overall fit, pattern of loadings) of the CBCL among different types of respondents including group care (Albrecht, Veerman, Damen, & Kroes, 2001) and mental health (Dutra, Campbell, & Westen, 2004) workers. Based on the current findings regarding the empirical basis and sound psychometric properties, the CBCL appears to have strong support across clinical and research settings. Since the CBCL has been used in multiple cultures, many researchers have investigated the construct validity of different translations of the CBCL in different populations. Findings based on those studies will be discussed later.
Advantages and disadvantages of the CBCL.
A close critical appraisal of the CBCL reveals that the instrument has multiple advantages and disadvantages. One of the main advantages is that empirical derivations of syndromes produce reliable and homogenous groupings of symptoms, with cut-off points that are referenced to a population rather than being dependent on an arbitrary number or criterion (Bird et al., 1996). McConaughy (2001) also provides a number of advantages of the CBCL. First, unlike the categorical approach, which can only identify children who meet the diagnostic criteria, the CBCL, based on the dimensional approach, helps identify children who do not meet the criteria (i.e., sub-syndromal conditions) but have functional impairment. Therefore, the CBCL provides a more comprehensive understanding of the full range of child behaviour problems. In addition, the large pool of items on the CBCL provides a more comprehensive view of the child's functioning and allows clinicians to investigate a wide spectrum of possible problems rather than focusing on a single complaint. McConaughy (2001) also indicates that results based on the CBCL can be tested in terms of psychometric standards of reliability and validity. Furthermore, using the relevant forms for different informants enables users to integrate information across a variety of situations. Finally, the low cost, ease of administration and scoring of the CBCL is another advantage (McConaughy, 2001).
CBCL's main limitations include lack of strong evidence in support of the competence scales and limited data on its use for younger children (Elliot & Busse, 2004). With regard to the competence scales, Drotar, Stein, and Perrin (1995) indicate that the Social Competence section of the CBCL is limited as it assesses selected aspects of social functioning, whereas social competence is a multidimensional construct which includes a wide range of social skills (e.g., competence in social situations, peer acceptance, and social skills). Moreover, the equal weighting of items that assess the quantitative (i.e., frequency) and qualitative dimensions of competence is considered problematic.
There is also criticism about the problem behaviour section. Furlong and Wood (1998) state that there may be issues in connection with objectivity and consistency in scoring as CBCL items are equally weighted, implying that the meaning of the response scale is the same for each item (e.g., “cruel to animals” may receive a rating of 2 when the behaviour is exhibited once or twice in general while “cries a lot” may receive a rating of 2 if it occurs once or twice in a day). In response, Achenbach (1991) has presented various rules and clarifications for using the response scale for selected items. Since the validity of the CBCL is largely based on its ability to discriminate between clinic-referred versus non-referred children, there is some criticism regarding its sensitivity in identifying changes in symptoms among children whose scores lie within the typical/normal range. Drotar et al. (1995) state that since the T-scores for the narrow band syndromes are truncated at both ends of the distribution, the ability of T-scores to differentiate between scores within the normal range is affected. In response, Achenbach has recommended using raw scores in which the full range of scores can be retained (Greenbaum et al., 2004). A final limitation of the CBCL is that similar to other assessment measures, the CBCL cannot be employed to determine specific forms of treatment or placement recommendations. Clinicians need to incorporate information based on the CBCL with information obtained from other sources before developing appropriate and meaningful interventions (McConaughy, 2001).
Range of applicability of the CBCL.
Despite some weaknesses, the CBCL is broadly employed in clinical, educational and research settings due to its demonstrated reliability and validity, as well as ease of administration and scoring (Achenbach & Rescorla, 2006; McConaughy, 2001). The recent additions in 2001 of larger normative and clinical samples as well as complex statistical analyses have made the CBCL more attractive to researchers. In the mental health domain, the CBCL is used in initial intake assessments (Achenbach & McConaughy, 2003). It is also used as a screening tool for emotional and behavioural problems including ADHD (DuPaul & Stoner, 2003), Conduct Problems (Lowe, 1998), and Obsessive-Compulsive Disorder (OCD) (Nelson, Stage, Duppong-Hurley, Synhorst, & Epstein, 2007) among others. Results based on the CBCL enable clinically useful comparisons, including the severity of the problem as well as the degree of deviancy in comparison to peers. A child's profile on the CBCL reveals essential taxonomic and diagnostic information in terms of whether there is elevation in single or multiple syndromes. Additionally, it also helps determine whether deviant behaviours are occurring in different contexts, such as home and school (Greenbaum et al., 2004).
In the clinical domain, repeated administration of the CBCL after intervals allows comparisons of the child's functioning over time, which can be informative in two ways (McConnaughy, 2001). First, this information is useful in assessing whether the child's problems are transient short-term issues, or chronic long-term patterns. Such information is salient in educational settings such as schools, which have long-term responsibilities to children. Second, information based on the CBCL can also be used in conjunction with other information to determine if treatment has been successful or if there is a need for intervention. Similarly, the CBCL is also useful when a child's adaptive functioning and home or school situation are questionable, as CBCL results, along with other information can help determine if an alternative placement is warranted (McConaughy, 2001).
The CBCL has been used for addressing multiple issues including theory, practical policy and methodological issues. It has been used as a measure for evaluation of outcomes in U.S. government programs (Greenbaum, Dedrick, Friedman, & Kutash, 2004). Moreover, it has been used in policy research on mental health services including the Fort Bragg study (Bickman, 1996) and the Center for Mental Health Services' National Evaluation of the Comprehensive Community Mental Health Services for Children and their Families Program (Holden, Stephans, & Connor, 2001). The CBCL has also been used in large-scale descriptive studies such as the National Adolescent and Child Treatment study (Greenbaum et al., 1996), which investigated mental health outcomes of nearly 800 children with serious emotional disturbances over a 7-year period. Additionally, the CBCL has been used to investigate normative developmental trajectories of children's behaviour problems to help detect developmental deviance in childhood and adolescence (Bongers, Koot, van der Ende, & Verhulst, 2003).
Similarly, at the international level there is an abundance of research based on the CBCL in the clinical as well as research contexts. These studies cover a variety of topics including epidemiology and risk factors for different emotional and behavioural problems, use of mental health services, treatment planning and outcome monitoring, as well as investigating psychological problems among children with different medical conditions (Achenbach & Rescorla, 2007; Fung & Tsang, 2006; Liu, Kurita, Guo, Miyake, Za, Ca et al., 1999; Rodenburg, Stams, Meijer, Aldenkamp, & Dekovic, 2005; Tick, van der Ende, & Verhulst, 2008). Although the CBCL is used in different domains, its demonstrated reliability and validity, ease of administration and scoring as well as applicability to cross-cultural samples have also contributed to its use as a measure to assess the epidemiology of emotional and behavioural problems around the world. At present there are multitudinous studies in which the CBCL has been used to investigate bicultural and multicultural similarities and disparities in the prevalence of emotional and behavioural problems (Crijnen et al., 1997; Rescorla et al., 2007).
Cross-cultural use of the CBCL.
In terms of the historical evolution of the cross-cultural use of the CBCL in epidemiological studies, Verhulst and his colleagues were the first to introduce the CBCL to Europe in 1982. They provided a well-designed epidemiological study of Dutch children, which included a standardization of the CBCL and the teacher report form. This was followed by the use of the CBCL across numerous countries in Europe and across the world. At present, the CBCL has been translated into more than 85 languages and there are over six thousand published studies from 72 countries (Berube & Achenbach, 2008). These studies are based on samples that have been collected from societies that share similar social structures, values, and views of child psychopathology, but differ in language and culture as well as societies that are drastically different in all aspects. Examples of countries where the CBCL has been used to investigate epidemiological data include Algeria (Petot, Petot, & Achenbach, 2008), China (Liu et al., 1999), Germany (Barkmann & Schulte-Markwort, 2005), Greece (Roussos, Karantanos, Richardson, Hartman, Karajiannis, et al., 1999), Israel (Zilber, Aurerbach, & Lerner, 1994), Jamaica (Lambert, Knight, Taylor, & Achenbach, 1994), Lithuania (Zukauskiene, Ignataviviene, & Daukantaite, 2003), Russia (Slobodskaya, 1999), Saudi Arabia (Abdel-Fattah, Asal, Al-Asmali, Al-Helali, Al-Jabban, & Arafa, 2004), Thailand (Weisz et al., 1987), and Turkey (Dumenci, Erol, Achenbach, & Simsek, 2004). In collaboration with other researchers, Achenbach has also developed national norms in the United States as well as in Australia, China, France, Greece, Israel, and Puerto Rico among others (Bilenberg, 1999). Moreover, extensive research has been conducted comparing data between two cultures as well as among multiple cultures simultaneously.
Although existing theoretical models of cultural influence provide a good basis for conceptualizing cross-cultural similarities and disparities, current studies have focused on exploring a host of environmental risk factors within different cultures to assess their association with emotional and behavioural problems (Kohrt, Kohrt, Waldman, Saltzman, & Carrion, 2004). The study of environmental factors is salient for the following two reasons. First, to investigate the direct impact of environmental factors and second to examine the influence of culture mediated through environmental factors such as psychosocial processes (e.g., parenting style, socialization) related to children's functioning and psychopathology (Weisz et al., 1987). Since the current study focuses on cross-cultural similarities and disparities, it is essential to address the theoretical foundations of environmental influences.
Theoretical Basis for Cultural Variations
Based on Bronfenbrenner (1979) and Belsky's (1980) ecological model as well as Cicchetti and Rizley's (1981) transactional model, the ecological-transactional model provides a sound theoretical framework to conceptualize the influence of various environmental factors, as it views the development of a child within a particular context and also provides a heuristic to understand the interplay of multiple factors. According to the ecological-transactional model, children function within multiple levels or ecologies in an environment, where factors at each level may influence each other as well as child development. These levels or ecologies can be described as a set of nested structures, with each one at different levels of proximity to the child. These ecologies include the macrosystem, the exosystem, the microsystem and the ontogenic level. The distal ecologies include the macrosytem, which contains the beliefs and values of the culture, and the exosystem, which includes aspects of the community in which the children and families reside. The more proximal ecologies are the microsystem, which comprises the immediate setting in which the individual lives, such as the family, school and peers, while onotogenic development consists of factors within the individual that contribute towards development and adaptation. Due to the transactional nature of the factors, factors at one level may influence outcomes in surrounding levels thereby determining the degree of risk posed to the individual. Moreover, at each level there exists an array of enduring and transient vulnerability and protective factors (Cicchetti & Toth, 1997). Therefore, it is important to investigate the relationship between factors at each level (e.g., children's characteristics, family, community and cultural factors) to assess children at risk for emotional and behavioural disorders (Forness, 2003).
At the ontogenic or individual/child level, physiological and medical factors (e.g., premature birth, low birth weight, and slow neurological development) are considered risk factors for emotional and behavioural disorders (Bhutta, Cleves, Casey, Craplock, & Anand, 2002; McCormick, McCarton, Brooks-Gunn, Belt, & Gross, 1998). At the microsystem level, family factors such as family structure (e.g., single or two parent households) as well as family functioning (e.g., parental marital discord, child maltreatment as well as parenting style) are considered strong predictors of problem behaviour (Harland et al., 2002; Turk et al., 2007; Vostanis, Graves, Meltzer, Goodman, Jenkins, & Brugha, 2006). Detailed analysis reveals that it is not family structure (i.e., single or divorced families vs. intact families) per se that leads to emotional and behavioural problems. In fact, it is factors such as economic hardship linked to being a single parent as well as the probability of there having been a period of disharmonious family relationships leading to separation or divorce, when child care was impaired, that may cumulatively increase the risk of emotional and behavioural problems (Turk et al.). In terms of childcare, parental warmth and sensitivity along with consistent control and structure are crucial for healthy development of children. Children raised by authoritarian parents who are harsh and rigid in enforcing rules are more likely to exhibit psychological problems than children of authoritative parents who, along with being nurturing and responsive are able to reason with their children (Webster-Stratton & Hammond, 1999). Researchers also state that parental mental health is also linked to the development of emotional and behavioural problems among children (Papp, 2004; Steinhausen, Mas, Ledermann, & Metzke, 2006). There is also substantial evidence linking SES of the family and emotional and behavioural problems (Bradley & Crowyn, 2002). Researchers suggest that children from low SES families tend to manifest symptoms of psychological problems and maladaptive social functioning more often than their affluent counterparts (Brooks-Gunn & Duncan, 1997; McCoy, Firck, Loney & Ellis, 1999).
At the exosystem or community level, impoverished neighbourhoods, violence, crime, and lack of social support have been identified as risk factors for emotional and behavioural problems (Jaffee et al., 2005; Mazza & Overstreet, 2000; Turk et al., 2007). It is pertinent to highlight that community factors directly as well as indirectly (i.e., through influencing the family) influence children's psychological development (Jaffee et al., 2005). For example, exposure to violence in the community may directly influence the child's perceptions about safety as well as emotion regulation (Margolin & Gordis, 2000), thereby increasing the risk of maladaptive behaviour. Moreover, chronic exposure to violence may desensitize children to Aggressive Behavior or may reinforce the belief that aggressive acts are acceptable and effective methods of coping, both of which lead to an increase in children's aggression levels (Schwartz & Proctor, 2000). Exposure to violence may also indirectly influence children's psychological functioning as it is associated with family conflict and parental distress, which in turn have a negative impact on children's mental health (Linares, Heeren, Bronfman, Zuckerman, Augustyn, & Tronick , 2001; Overstreet & Cerebone, 2005). Furthermore, stressful life events at the family or community level have been associated with emotional and behavioural problems among children (Harland et al., 2002; Mesman & Koot, 2001; Turk et al.). These stressful events can include illnesses, hospitalization or death of a family member, divorce, parental loss of employment, community violence, and the birth of a sibling, all of which have direct as well as indirect influences on children's social and emotional development.
At the cultural level, two models have been offered to explain how cultural values and beliefs may influence children's development and psychopathology. Although the models have been derived from cross-cultural studies between Thai and American children only, they provide a basis to conceptualize cultural influence. These models include the Problem Suppression-Facilitation model and the Adult Distress Threshold Model, which describe how cultural beliefs influence children's problems and adults' perspectives to children's problems, respectively. According to the Problem Suppression-Facilitation model (Weisz et al., 1987), culturally mediated beliefs, values, and traditions through associated child-rearing and socialization practices may help shape both the occurrence and maintenance of different types of children's problems. Cultural values may influence specific problems by suppressing (e.g., by punishment or social pressure) the development of behaviours that are disapproved and encouraging (e.g., by teaching, modelling or rewarding) the development of behaviours that are acceptable in the culture. The second model, namely the Adult Distress Threshold model (Weisz et al., 1987), has two forms. The general form of the model proposes that culture influences adults' attitudes towards children's behaviour in general, helping to determine how distressing the child's behaviour will be to adults and the likelihood of adults seeking help through clinical intervention. The pattern-specific form proposes that cultures will vary with certain problems raising more concern in some cultures than others.
Weisz and his colleagues conducted multiple studies to explore how culture may influence differences in the reporting of emotional and behavioural problems between Thai and American parents. Weisz, McCarty, Eastman, Chaiyasit, & Suwanlert (1997) provide a methodical synthesis of their logical line of inquiry encompassing multiple studies and utilizing diverse methodological approaches. Based on their initial study, Weisz et al. concluded that among children and adolescents referred to mental health clinics, Thai parents reported more Internalizing Problems whereas American parents reported more Externalizing Problems. They attributed these findings to cultural differences in terms of Thai culture being based on values that encourage self-control, emotional restraint, and strict compliance with rules. These cultural values may have contributed to children developing more Internalizing Problems as opposed to Externalizing Problems, thus supporting the Problem Suppression-Facilitation model. Based on subsequent research, Weisz et al. (1987) and Weisz, Sigman, Weiss, and Mosk (1993) stated that consistent with the clinical population even in the general population, Thai children had significantly more Internalizing Problems than U.S. children. However, the reverse did not hold true for Externalizing Problems, as there were no significant cultural differences. Weisz and colleagues (1997) did not provide an explanation for the lack of significant differences with regard to Externalizing Problems, but explained the differences in Internalizing Problems by stating that Buddhist values, which encourage self-control and emotional restraint, may have contributed towards Thai children expressing distress in a manner that is socially acceptable. These findings provide partial support for the Problem Suppression-Facilitation model as the results hold true for Internalizing but not Externalizing Problems.
To test the Adult Distress Threshold model, Weisz and colleagues (1991) assessed the likelihood of refer-ability of Internalizing and Externalizing Problems. In general, American parents compared to their Thai counterparts referred children more often for both Internalizing as well as Externalizing Problems. Within each culture, Internalizing Problems were more often the reason for referral in Thailand whereas the converse was true in the U.S. (i.e., Externalizing Problems were more often the reason for referral in the U.S.). The higher degree of refer-ability of Internalizing Problems among Thai parents is in contrast to the pattern-specific form of the Adult Threshold Model. The pattern-specific form of the model proposes that, based on Thai notions of social self-restraint, Thai adults may have a lower threshold of Externalizing Problems leading to a higher referral of Externalizing Problems as opposed to Internalizing Problems. Weisz et al. (1997) concluded that since Externalizing Problems are highly unacceptable and in violation of Thai social norms, parents may be embarrassed to disclose such problems, admitting via clinical referral their inability to prevent such problems.
Weisz et al. (1997) also stated that overall differences in refer-ability (i.e., American parents referring children more often for both Internalizing and Externalizing Problems than Thai parents) may be linked to distinct adult attitudes towards children's problems in general. Research findings indicated that consistent with some literature on Thai Buddhist values (Jumsai, 1980), Thais (as compared to Americans) rated both Internalizing and Externalizing Problems to be less serious, less worrisome, less likely to reflect personality traits and more likely to improve with time. These cultural differences support the general form of the Adult Distress Threshold model, which proposes that cultural values influence adults' response to children's problems. Weisz et al. stated that cultural differences might also be attributed to adult beliefs regarding aetiology and remediation of children's problems. Thai parents were more likely to attribute the problem to faulty child rearing, socialization or teaching, whereas American parents considered child personality traits or psychodynamic attributions (e.g., the child having internal conflicts) as a cause of the problems. Similarly, in remediation, Thais preferred verbal interventions (e.g., “talk to the child”, “reassure the child”) while Americans favoured more behavioural approaches regarding reward and punishment.
Although adults' perspective or threshold plays an important role in determining the reporting of symptoms, Weisz et al. (1991) concluded that refer-ability may also be influenced by other cultural differences. These differences include the extent to which a problem is thought to require professional intervention as opposed to remediation within the home, the availability and capability of specialized professionals, the willingness of parents to disclose problems that might otherwise be hidden at home, and the level of exposure to child psychology.
The American-Thai comparison studies illustrate that although there is insufficient support for the two models of cultural influence, yet exploring their association with emotional and behavioural problems has uncovered a host of salient cultural factors. These cultural factors go beyond the two cultural models and highlight the need to investigate the relationship between different environmental factors and emotional and behavioural problems. A deeper understanding of these factors is essential to interpret cross-cultural research studies and develop a comprehensive understanding of cultural similarities and disparities.
Bicultural CBCL comparisons.
Numerous researchers have investigated bi-cultural variations between U.S. and other cultures including China (Weine, Phillips, & Achenbach, 1995), France (Stanger, Fombonne, & Achenbach, 1994), Greece (MacDonald, Tsiantis, Achenbach, Stefanidi, & Richardson, 1995), the Netherlands (Verhulst, Achenbach, Ferdinand, & Kasius, 1993), Puerto Rico (Achenbach et al., 1990), and Thailand (Weisz et al., 1987), among many others. Moreover, there have also been bi-cultural studies comparing children from the Netherlands with children from France (Berg, Fombonne, McGuire, & Verhulst, 1997) and Belgium (Hellinckx, Grietens, & Verhulst, 1994).
These studies have revealed strong similarities and differences across cultures. In 1995, Verhulst and Achenbach reviewed findings across multiple bi-cultural studies. The bi-cultural studies included in the review were based on samples from Australia, China, France, Greece, Jamaica, the Netherlands, Puerto Rico, and Thailand, which had been compared to one of two U.S. samples, whereas one study compared findings between Belgium and Holland. Verhulst and Achenbach (1995) reported that approximately half the bi-cultural comparisons indicated significant cultural differences at the Total Problems level. Although the differences were significant, based on Cohen's (1988) criteria, these effect sizes ranged from small (i.e., .01 to .06) to medium (i.e., .06 to .14). The biggest cross-cultural difference was found between American and Puerto Rican children, with Puerto Rican children having more problems than American children (Verhulst & Achenbach, 1995).
Although the bi-cultural comparison study was rich in information and included large samples, comparisons across bi-cultural studies may be problematic for multiple reasons. Despite the fact that samples in each bi-cultural study were matched on most demographic variables, there was disparity across the bi-cultural studies in terms of the age range assessed. For example, approximately three bi-cultural studies focused on ages 4- to 16-years (e.g., Achenbach et al., 1990) while others focused on ages 6- to 11-years (e.g., MacDonald et al., 1995), while one research was based on a sample of 11- to 15-year-olds (e.g., Weisz et al., 1993). Furthermore, the two U.S. samples that most other countries were compared to had been collected from different regions in the U.S. and were based on different versions of the CBCL.
A detailed review of the individual bi-cultural studies also highlights various factors that may have contributed to cultural differences. These factors include problems linked to translation and interpretation of questions and differences in rates of mental health referrals, among others. Although CBCL translations are developed to idiomatically translate the meaning of CBCL items, there is a lack of ideal cross-linguistic behavioural rules as well as somatic concerns which contribute to some descriptions of children's problems appearing to be more common, more pathological, or more difficult to determine in one language compared to another (Crijnen, Achenbach, & Verhulst, 1997). Multiple waves of translations and back translations can help ameliorate the problem. Another factor to consider is the difference in rates of mental health referrals. Cultural differences in rates of mental health referrals may influence results as samples collected from countries (e.g., Greece) where mental health services are not readily available or parents hesitate before reporting psychological problems could lead to inclusion in the sample of children who may need clinical help. The inclusion of such children in the community sample may influence overall prevalence rates (McDonald et al., 1995).
Multicutural CBCL comparisons.
To go beyond bi-cultural comparisons, Crijnen et al. (1997) conducted a multicultural comparison study, wherein they employed the concept of multicultural composites. The notion of multicultural composites is based on Ellis and Kimmel's (1992) work and is used to compare findings from each culture with a composite of findings from all the cultures combined. Ellis and Kimmel (1992) had proposed that results from 10 or more cultures would be required to provide an adequate basis for comparison. In 1997, Crijnen and colleagues compared results based on the CBCL from two groups: the first group consisted of twelve cultures, with samples based on 6 to 11-year-olds whereas the second group consisted of nine cultures with children ranging in age from 6- to 17-year-olds. Unlike previous studies, there was one U.S. sample based on a national sample that used the CBCL 4/18 (Achenbach, 1991).
Researchers reported that in both cultural comparisons (i.e., the twelve cultures group as well as the 9 culture group), there were significant differences among cultures at the Total Problems level with effect sizes of .11 and .8 respectively. Similarly, at the Externalizing Problems as well as Internalizing Problems level, there were medium effect sizes for culture in both groups. Further analysis of the nine cultures group indicated that Internalizing Problems were slightly higher than Externalizing Problems among Puerto Rican and Jamaican children whereas the converse was true for American, German and Swedish children. Although these findings indicate small but significant cross-cultural variations in the relative dominance of Externalizing versus Internalizing Problems, a step-wise and thorough investigation of environmental factors is required to understand whether the differences in the type of problem and culture support the Problem Suppression-Facilitation and/or the Adult Distress Threshold model.
Although the 1997 multicultural comparison had interesting findings, it is salient to highlight the shortcomings in the research. In both the 12 and the 9 cultures group, Puerto Rican children reportedly had the highest while Swedish children had the lowest level of Total, Externalizing and Internalizing Problems. Crijnen and colleagues (1997) attribute the differences partly to variations in sampling procedures, as well as socio-economic status differences between the two samples. The Puerto Rican sample was collected by randomly sampling households on the entire island with a 92% response rate, which was the highest of any culture. In contrast, the Swedish sample was obtained by sampling from schools in a comparatively affluent area of Sweden; the response rate was lower (84%) and special schools for children with problems were excluded. Therefore, more children with higher scores may have been omitted in the Swedish sample compared to the Puerto Rican sample (Crijnen et al.). Although there were some significant culture by age and culture by gender interactions, effect sizes were very small (<.01).
Crijnen and colleagues (1999) also investigated cross-cultural variations on the eight CBCL syndromes among children from nine cultures. Findings revealed significant effects of culture on all eight syndromes, but the effect sizes ranged from medium (.06 to .14) for Withdrawn and Social Problems to small (.01 to .06) for Somatic Complaints, Anxious/
depressed, Thought Problems, Attention Problems, Delinquent behaviour, and Aggressive Behavior syndromes. Puerto Rican children had the greatest number of problems whereas Swedish children had the lowest number of problems across all syndromes. As the 1999 study used the same data as the 1997 study, the differences between the Swedish and Puerto Rican results could be due to sampling as well as SES variations described previously (Crijnen et al.). These studies highlight the importance of consistency across researches in terms of sampling procedures as well as matching researches on demographic variables. Disparities across researches on demographic variables as well as sampling issues need to be avoided as they can be confounding factors for cross-cultural comparisons.
Most recently, Rescorla and colleagues (2007) compared multicultural findings based on the current version of the CBCL across 31 diverse societies: 12 from Western Europe, 5 from Eastern Europe, 6 from Asia, 1 from Africa, 3 from the Middle East, 2 from the Caribbean, plus Australia and the United States. Results indicated that there was considerable cross-cultural consistency at the Total Problems level. Based on the degree of Total Problems, the results were divided into three different groups. One group consisted of parent reports from nineteen societies that had Total problem scores within one standard deviation of the omni-cultural mean. The second group consisted of parent reports from six societies (i.e., Japan, China, Sweden, Norway, Germany and Iceland), which had Total problem scores more than one standard deviation below the omni-cultural mean, whereas the third group consisted of parent reports from six other societies (i.e., Puerto Rico, Portugal, Ethiopia, Greece, Lithuania and Hong Kong), which had Total Problems more than one standard deviation above the omni-cultural mean. Within each of the three groups, namely the high-, medium- and low-scoring, societies differed in terms of geographic region, ethnicity, religion, size, population and economic/political system (Rescorla et al.). Rescorla and colleagues (2007) also stated that response rates, data collection methods or exclusion of referred children could also not account for the variations in the reported problems. However, in terms of the data collection method employed, societies where data had been collected based on home interviews had the highest scores followed by school-based methods and mail surveys.
Factors such as cultural orientation and cultural notions of self-presentation (“saving face”) were also investigated as factors salient for cultural disparities at the Total Problems level. Cultural orientation was not a salient factor to distinguish between groups, as within each group there were highly individualistic societies (e.g., United States, Holland), collectivistic societies (e.g., Thailand and Korea), as well as societies that had previously been governed by collectivistic regimes (e.g., Poland and Romania). It was also investigated whether cultural disparities could be attributed to culturally based notions of self-presentation as reflected in parents' reluctance to report problems. In comparison to Euro-American societies, Asian societies may be more concerned with self-presentation (Yabuchi, 2004). The analysis led to mixed results since parents from some Asian societies such as China and Japan reported fewer problems while parents from other Asian societies such as Taiwan, Hong Kong, Thailand and Korea did not. Rescorla et al. (2007) state that high scores may reflect relatively low thresholds for reporting problems and suggest that parents in the high-scoring group have high expectations in terms of what is appropriate behaviour, and if children do not conform to those expectations, parents perceive them as displaying behaviour problems. Across the 24 societies that had children within the age range of 6 to 16 years, the effect size for society was .08. It is also pertinent to note that Rescorla et al. did not compare societies on factors such as cultural orientation or cultural notions of self presentation on Internalizing and Externalizing Problems, especially since society effect sizes for Internalizing and Externalizing Problems were .08 and .05 respectively. Rescorla et al. did state that societies differed more on Internalizing Problems than Externalizing Problems, which may suggest greater cultural disparity in the expression, perception and reporting of Internalizing Problems. However, Rescorla and colleagues did not provide details regarding which societies had the highest vs. lowest Internalizing scores. A closer inspection of the societies with higher Internalizing scores may have provided support for the models of cultural influence such as the Suppression-Facilitation model and the Adult-threshold model specific form.
At the syndrome level, the effect sizes for society were small for all syndromes (.04 to .06) except Anxious/Depressed, which had a medium effect size (.09) for society. Rescorla's (2007) multicultural study is the only study that has compared DSM-oriented scales across multiple cultures. Based on their analyses, the effect sizes for society were small (.03 to .06) for all DSM-oriented scales except DSM-Anxiety Problems, which had a medium effect size (.08). At the item level, there was cross-cultural consistency in parents' ratings of CBCL items that were most and least endorsed. These ratings were also concordant with common sense notions of children's problems as highly endorsed CBCL items included oppositional behaviour, shyness and difficulties with attention and self-control, whereas the least endorsed items were seeing things that are not there, running away from home and day time enuresis (Rescorla et al.). Apart from cultural similarities and disparities, bi-cultural and multi-cultural studies have also investigated the relationship between factors such as age, gender, SES and emotional and behavioural problems.
Identifying correlates of emotional and behavioural problems.
Among the different correlates of emotional and behavioural problems, findings related to age will be discussed first, followed by gender and family socio-economic status (SES). With regard to age, in the 1995 review of bi-cultural studies there were significant but small (.01 to .06) age effects in 3 out of 9 comparisons for Total Problems score, indicating more problems for younger children. At the broadband level, Internalizing Problems increased with age whereas Externalizing Problems decreased with age. Similarly, in the initial multicultural comparison in 1997, Crijnen and colleagues reported that there was a significant but small main effect of culture at the Total Problems level for the 9 culture group covering ages 6 to 17 years whereas there was not a significant main effect for age in the 12-culture group which consisted of children between the ages of 6 to 11 years old. However, there was a tendency for Total Problems and Externalizing Problems to decrease with age and for Internalizing Problems to increase with age. Findings based on Crijnen and colleagues' (1999) study of cross-cultural comparisons of empirically based syndromes also indicated significant age differences on all syndromes except Thought Problems and Delinquent behaviour. However, effect sizes ranged from very small (<.01) to small (.01) for Aggressive Behavior. A closer analysis revealed that on syndromes such as Withdrawn and Somatic Complaints, which constitute Internalizing Problems, problem behaviours increased from childhood to adolescence. In contrast, there was a decrease in problem behaviour from childhood to adolescence with regard to Aggressive Behavior and Delinquent Behaviour, which constitute Externalizing Problems.
With regard to gender, findings of the 1995 review revealed that cross-cultural differences at the Total Problems level were significant but had low effect sizes (i.e., .01 to .06). Results also indicated that girls had more Internalizing Problems whereas boys had more Externalizing Problems. At the syndrome level, with great cross-cultural consistency, girls had more Somatic Complaints whereas boys had more Attention Problems. In the 1997 multicultural comparison, in the 12 culture group, there was a significant but very small main effect of gender whereas there was no significant effect of gender in the 9 culture group, which covered a broader age range. Consistent with previous results, there was a tendency for boys to have more Total and Externalizing Problems and for girls to have more Internalizing Problems (Crijnen et al., 1997). Similarly, the 1999 multicultural comparison indicated significant gender differences on five syndromes, but the differences ranged from small (.01) for Attention Problems and Delinquent behaviour to very small (<.01) for Somatic Complaints, Anxious/Depressed, and Aggressive Behavior. With great cross-cultural consistency, girls had more problems linked to Somatic Complaints and Anxious/Depressed syndrome while boys had more problems on the Attention Problems, Delinquent behaviour, and Aggressive Behavior syndromes (Crijnen et al., 1999).
With reference to SES, cross-cultural studies that assessed SES in a similar fashion were compared and results indicated more problems for children from low SES backgrounds than children from more affluent backgrounds (Verhulst & Achenbach, 1995). In the 1997 study, since SES was measured differently across countries, a single SES scale could not be applied to all twelve cultures. However, within several cultures, in comparison to children from higher SES families, children from lower SES families reportedly had more problems (Crijnen et al., 1997).
In the most recent multicultural comparison, findings replicate previous results yet also give a more detailed view in terms of the trends in emotional and behavioural problems in relation to age and gender. Although all gender and age effect sizes were <.01, similar to previous studies, Rescorla et al. (2007) found that there was an increase in Internalizing Problems and a decrease in Externalizing Problems with age. Age effects were not consistent across societies with significant age effects for Withdrawn/Depressed, Social Problems, and DSM-oriented Attention Deficit/Hyperactivity problems. A closer inspection revealed that there was an increase with age for Withdrawn/Depressed and a decrease in age for Social Problems as well as DSM-oriented Attention deficit/hyperactivity problems. Rescorla at al. (2007) attribute these findings with adolescents in many societies becoming more withdrawn from parents as they become more involved with their peers.
Moreover, consistent with previous results, girls had more Internalizing Problems whereas boys had more Externalizing Problems (Rescorla et al., 2007). However, Rescorla et al. also reported an interaction of age and gender as results indicated that across most societies, girls tended to have more Internalizing Problems, especially between the ages of 12 and 16 years, whereas boys tended to have Externalizing Problems, especially between the ages of 6 and 11 years. With regard to SES, although researches varied in terms of measures of SES (e.g., parent's occupation and/or education, family income), researchers indicated that all studies reported higher Total Problems for children from lower SES families than for children from higher SES families. The results were consistent across multiple countries, including countries from Europe, the Caribbean, Asia, Australia and the U.S. These cross-cultural findings, which indicate strong inter-cultural similarities with regard to age, gender and SES, may reflect patterns that transcend cultural differences. These patterns may include biological factors or cross-cultural similarities in socialization practices.
The multicultural studies (e.g., Crijnen et al., 1997; 1999; Rescorla et al., 2007) examining cross-cultural similarities and disparities at different levels of the CBCL as well as with regard to different correlates pave the way to a more comprehensive investigation and identification of children's emotional and behavioural problems. Moreover, these findings support the application of the CBCL in more cultures as it can help unmask the culture-specific and universal aspects of psychopathology (Crijnen et al., 1999). In theory, using a measure in a different language and culture can help assess if the current knowledge about child assessment can be generalized. This can facilitate the development of a global theory of developmental psychopathology. In practical terms, the availability of cross-culturally robust measures can help clinicians identify needs of immigrant children in today's global society (Achenbach & Rescorla, 2007). Within developing countries like Pakistan, such epidemiological data is useful in numerous ways; it helps in developing awareness regarding children's problems, identifying high-risk groups as well as providing much needed information for treatment planning. However, before any further research can be conducted and conclusions drawn on the basis of the findings, it is imperative to test the psychometric properties of the CBCL in different cultures in order to validate the research findings.
Psychometric Properties of Various Translation of the CBCL
Along with using the CBCL to investigate the prevalence of emotional and behavioural problems around the globe, researchers have also examined the psychometric properties of the CBCL translations used in different cultures. With regard to reliability, researchers have focused mainly on internal consistency, with few researches based on other indices of reliability such as test-retest or inter-rater reliability. In terms of validation studies, the focus has been on investigating the construct validity of different translations of the CBCL, whereas researches investigating risk factors associated with emotional and behavioural problems help support predictive validity of the CBCL.
In terms of inter-rater reliability, at present results based on inter-parent reliability across cultures are not available. However, Achenbach & Rescorla (2007) provide mean inter-rater reliability scores based on the CBCL and TRF across 18 cultures. These findings indicate low (Total Problems = .29; Internalizing = .20; and Externalizing= .32) inter-rater reliability as the two informants (i.e., parents and teachers) view children in different environments. In terms of test-retest reliability, Leung et al. (2006) investigated the test-retest reliability of the Chinese translation of the 1991 CBCL. Results indicated that the Chinese CBCL was test-retest reliable for Internalizing, Externalizing and Total Problems (range = .76 to .83).
Internal consistency of various translations of the CBCL has been investigated in multiple cultures including Italy (Frigerio, Cattaneo, Cataldo, Schiatti, Molteni, & Battaglia, 2004), Lithuania (Zukauskiene et al., 2003), Russia (Carter, Grigorenko & Paul, 1995), and Taiwan (Yang, Soong, Chiang-Ning, & Chen, 2000) among others. Findings based on these studies indicate that Total, Internalizing and Externalizing Problems have good internal consistency (alpha values ranged between .78 and .97). More recently, Rescorla and colleagues (2007) investigated the internal consistency of data sets across 31 societies. Since 6 items were replaced in the 2001 version to ensure consistency only items that were common in all societies were analysed. In concordance with previous findings, results indicated a high level of internal consistency across Total, Internalizing as well as Externalizing Problems, as the mean alpha values across 31 societies were .93, .83, and .87, respectively. For each of the 31 societies, alphas for Total Problems were >.90, while the alphas for Internalizing and Externalizing Problems were >.72 and >.80, respectively. The mean alpha scores for the eight empirically-based syndromes ranged from .58 to .84. Specifically, Anxious/Depressed and Attention Problems had acceptable internal consistency (alpha >.70), whereas Withdrawn/Depressed, Somatic Complaints, Social Problems and Rule-Breaking Behavior had marginally acceptable internal consistency (alpha values between .60 and .70). Thought Problems had the lowest internal consistency (alpha .58). Among the DSM-oriented scales, Attention Deficit/ Hyperactivity problems, Oppositional Defiant Problems and Conduct Problems had acceptable internal consistency (alpha values between .70 and .75). On the other hand, Affective Problems, Anxiety Problems and Somatic Problems had low internal consistency (alpha values between .58 and .62). Rescorla et al. (2007) state that the greater variability of alpha values among the empirically based syndromes and DSM-oriented scales in comparison with the Total problem and broadband categories may partly be attributed to the former scales consisting of fewer items.
Rescorla at al. (2007) also indicated that in comparison to other empirical syndromes, Aggressive Behavior (alpha value .84) had the highest internal consistency whereas consistent with previous findings, Thought Problems (alpha value .58) had the lowest level of internal consistency. The low internal consistency on the Thought Problems syndrome may be due to low endorsement or prevalence of Thought Problems in non-clinical populations. In order to clarify the issue, further testing of the CBCL with a clinical sample, in which the endorsement or prevalence may be higher, may be helpful. It is important to note that in comparison with other empirical syndromes such as Aggressive Behavior (alpha .94), and Attention Problems (alpha .86), Thought Problems (alpha .78) also had relatively low internal consistency in the American sample (Achenbach & Rescorla, 2001). Based on these findings, it is hypothesized that there may be considerable variability among items that constitute the syndrome (e.g., items measuring different aspects of Thought Problems) leading to low correlation among items. Therefore, a detailed analysis may be needed to assess the variability between individual items on the Thought Problems syndrome.
In terms of validity, numerous studies have been conducted to assess the factor structure of the CBCL in different countries. These studies varied with regard to the model tested (i.e., 1983 or 1991) as well as the statistical procedures used. Statistical procedures have encompassed exploratory factor analysis (EFA) as well as confirmatory factor analysis (CFA). Although both EFA and CFA are used to replicate the observed relationship among a set of indicators with a smaller set of latent variables, the tests differ in terms of the number and type of a priori specifications and restrictions made on the factor model (Brown, 2006). In EFA, no specifications are made about the number of latent factors or the pattern of the relationship between the latent factors and indicators (i.e., factor loadings). In contrast, in CFA the number of factors and the pattern of the loadings are specified in advance (Brown, 2006). According to Brown (2006), CFA is employed to assess the degree to which a data set fits a particular model, where the model can be derived on the basis of EFA or theoretical assumptions. Additionally, CFA can test the same model across many data sets and specify which items load on which factors.
In CFA, researchers assess two aspects of construct validity: discriminant validity and convergent validity. Discriminant validity refers to the distinctiveness of the factors measured by different sets of indicators. Evidence of discriminant validity is present when different factors are not excessively correlated with each other (e.g., >0.85). On the other hand, convergent validity refers to the cohesiveness of a set of indicators in measuring their underlying factor. Evidence for convergent validity is present if a set of indicators all have relatively good correlation with the specific factor they are measuring (Sun, 2005).
Within CFA, multiple estimation methods exist that can be used to test a model. Commonly used estimation methods include the maximum likelihood (ML), generalized least square (GLS), weighted least square (WLS) and asymptomatically distribution-free (ADF) among others (Brown, 2006; Tabachnick & Fidell, 2001). Similarly, there are multiple fit indices that can be employed to indicate how well the model fits the data (Brown, 2006; Sun, 2005). Commonly used fit indices include the Chi-square statistic (χ2), Goodness of fit index (GFI), Tucker-Lewis Index (TLI), root mean square error of approximation (RMSEA), and the Comparative Fit Index (CFI). Each fit statistic has specific properties and assesses different aspects of the model (Brown, 2006; Kline, 2005), and will be described briefly.
The classic goodness of fit index is Chi-square (χ2), which tests if the correlations between the indicators can be replicated using the information in the model. The Chi-square fit index has certain drawbacks that include it being influenced by sample size (i.e., when the sample size is large, even a slight difference between the reproduced and sample covariance matrices can be magnified to statistical significance, indicating a lack of fit). Moreover, Chi-square is not the appropriate index for non-normal data as the underlying distribution of non-normal data is not Chi-square distributed, which compromises the statistical significance of the tests of the model Chi-Square. However, it is used for nested model comparisons and the calculation of other fit indices such as the TLI (Brown, 2006). The TLI can be used to compare alternative models or a proposed model against a null model. The GFI assesses the extent to which the model captures the variance shared by the indictors. Although the calculation of GFI does not involve the sample size directly, its sampling distribution is still subject to sample size. The CFI assesses the model fit relative to a baseline null or an independent model and makes use of the non-central Chi-square distribution. The RMSEA estimates how well the model parameters will reproduce the population covariances. The RMSEA is also an indicator of misfit which contrasts other approaches (Brown, 2006).
Fit indices vary in their appropriateness for specific circumstances in which goodness of fit is assessed. Thus, different fit indices may be suitable for different purposes and circumstances, and the choice of fit indices may affect the interpretation of results. Factors that help determine which fit indices to use include the aspect (convergent or divergent) of construct validity they assess, their sensitivity to sample size and availability of well-established cut-off criteria. Among the different fit indices, TLI, CFI, and RMSEA assess convergent validity or the extent to which the indicators load on specific factors (Sun, 2005). Although there is considerable debate over what constitutes the appropriate cut-off criteria, Hu and Bentler (1999) stated that the cut-off criteria should be slightly stricter than the conventional rules of thumb for model evaluation. They proposed that for a good fit, the criteria should be 0.95 for TLI and CFI, 0.90 for GFI, and below 0.06 for RMSEA. Marsh, Hau, & Wen (2004) have criticized this criterion for beign too harsh and incorrectly rejecting
models which had been specified correctly. In most CBCL studies, researchers have used Brown and Cudek's (1993) criterion of >.90 for a good fit and .80 to .90 for an acceptable fit for the CFI and TLI indices. Moreover, since recent CFA analyses have been done using the Weighted least squares with standard errors and mean- and variance-adjusted chi square estimator (WLSMV) in Mplus (Muthen & Muthen, 2004), a RMSEA value of <.06 indicating a good fit (Yu &Muthen, 2002).
Characteristics of the data including sample size, type of distribution (normal or skewed), and type of data (ordinal or interval) also help determine which estimation method and fit indices can be used (Brown, 2006; Sun, 2005, Tabachnick & Fidell, 2001). For example, since CBCL findings based on a community sample are not normally distributed, it limits the utility of estimation procedures that assume multivariate normality (e.g., M. L.) (Dumenci et al., 2004). Therefore, it is recommended that estimation methods that do not assume multivariate normality (e.g., asymptomatically distribution free (ADF) be used. Satorra and Bentler (1988) developed a statistic that incorporates a scaling correction for the chi-square statistic when distributional assumptions are violated. According to Byrne (2006), when the ML Robust option is chosen with the EQS softwarre, robust versions of fit indices such as CFI, RMSEA and the 90% confidence interval related to the latter are calculated. The ML robust fit indices are robust to violations of normality. However, since CBCL scores are in an ordinal format, ML estimation methods using product moment correlation are not appropriate since it underestimates the strength of the relationship between items that are in ordinal format. The WLSMV (Muthen & Muthen, 2004), an asymptotically distribution-free estimator, is considered most appropriate for CBCL type data. Reasons for its appropriateness include that it can be used with ordinal data without assuming multivariate normality. Since CBCL data is generally not normally distributed, the WLSMV estiamator is considered the estimator of choice.Sample size is also a salient factor in determining the appropriate fit index. Among fit indices, Chi-square is influenced by sample size (Sun, 2005).
In terms of the CBCL, tests of the 1983 or the 1991 syndrome models have been reported in data from different cultures including Thailand (Weisz, Weiss, Suwanlert, & Chaiyasit, 2003) and France (Berg et al., 1997). A comprehensive analysis of the 1991 eight-syndrome model was conducted by Hartman and colleagues (1999) which included general and/or clinical samples in Greece, Israel, the Netherlands, Norway, Portugal and Turkey. Hartman and colleagues, who used two types of estimation models: ML analysis of Pearson correlations and ULS analysis of polychoric correlations, concluded that the model was not a good fit. According to Rescorla and Achenbach (2007), Hartman et al.'s results that the model was not a good fit can be attributed to the use of inappropriate estimation models (e.g., M.L.) which assume multivariate normality. With regard to the ULS analysis, although Hartman, Hox, Aauerbach, Erol, Fonseca, Mellenbergh, & Novil (1999) stated that polychoric correlations may be distorted when cell frequencies are low, and later studies based on the same data sets revealed that cell frequencies were low, they still used the procedure to assess goodness of fit. Rescorla and Achenbach (2007) have stated that the results could also be due to Hartman et al.'s failure to exclude items that reported for less than 5% of the sample, which was the criterion for the original U.S. sample. Rescorla and Achenbach (2007) also add that Hartman et al. used the criterion of 0.01 to 0.035 for RMSEA, which is more stringent than what they cited earlier (0.03 - 0.07) in their research as indicating a good fit.
The first test of the 2001 eight syndrome model was conducted by Dumenci, Erol, Achenbach and Simsek (2004) in Turkey. The sample was divided into three groups: a general population, a clinical population, and a combined group. For analyses, Dumenci and colleagues used the weighted least squares with the mean- and variance- adjusted fit statistic (WLSMV), which they considered to be the most appropriate method to use with ordinal data that is not normally distributed (Achenbach & Rescorla, 2007). Results indicated a good model fit for all three groups. Moreover, Dumenci et al. (2004) reported that 99% of the items had significant and substantial loadings on the syndromes to which they were assigned. With regard to fit indices, the RMSEA values for the eight-factor model were .057, .054 and .041 in the clinical, combined and general population samples, respectively.
The most comprehensive test of the current CBCL's eight syndrome model was conducted by Ivanova and colleagues in 2007. In their study, confirmatory factor analysis was used to assess if CBCL problem item scores obtained from general population samples across 30 societies met the criteria for good fit to the eight syndrome model derived from the U.S. data. Multiple indices were used to assess goodness of fit, including the RMSEA, TLI and CFI. Although the TLI and the CFI are not well suited for binary numbers, the two indices were computed and considered secondary to RMSEA. Results indicated that the fit indices strongly supported the eight syndrome model in all thirty societies. According to Achenbach and Rescorla (2007), the RMSEAs were (<.06) for all samples which indicated that the eight-syndrome model was a good fit for all 30 cultures. For a majority of the cultures (i.e., 21 out of 30 societies), the RMSEAs were (<.04) indicative of a particularly good fit. Findings based on the TLI indicated acceptable (.80 to .90) to a good fit (>.90) model for all societies except Ethiopia (.79). The CFI indicated acceptable (.80 to .90) to a good (>.90) model fit for all societies except Ethiopia (.75), Germany (.73), Hong Kong (.79) and Lithuania (.73). These results lend support to the construct validity of various translations of the CBCL.
Along with construct validity, there is evidence to support the predictive validity of the CBCL in different cultures. Findings based on Mesman and Koot's (2001) longitudinal study in the Netherlands support that CBCL results in early childhood (ages 2 to 3 years) were predictive of their DSM counterparts 8 years later, independent of the influence of early parent-reported family risk factors. Numerous studies in the U.S. have indicated that there is a strong association between environmental risk factors and emotional and behavioural problems (Harland et al., 2002; Nelson et al., 2007; Turk et al., 2007). These factors include child characteristics as well as parent, family and community factors. Researchers have also found similar findings across cultures which support the predictive validity of the CBCL. Liu et al. (1999) investigated risk factors of emotional and behavioural problems among Chinese children. Logistic regression analyses revealed that a number of psychosocial and biological risk factors (e.g., parental chronic physical illness, maternal psychiatric history, poor marital relations among parents and complications during pregnancy) were associated with emotional and behavioural problems. In a community-based national sample of 4480 parents of school- age children in the Netherlands, Harland et al. found that family characteristics and recent life events, such as parental unemployment or recent separation or divorce of parents were strongly associated with children's risk of emotional and behavioural problems. In a two- phase study of male Saudi school children, Abdel-Fattah and colleagues (2004) used the CBCL in the screening phase followed by a case-control phase. The purpose of the case-control phase was to asses risk factors by comparing emotionally and behaviourally disturbed children (based on the screening results) to a randomly chosen sample of their age and school matched controls. Results indicated that medical factors (i.e., history of meningitis and bronchial asthma) were strongly associated with an increased risk of emotional and behavioural problems. In conclusion, it is evident that there is a strong pattern of correlations between environmental risk factors and emotional and behavioural problems across cultures, which provides evidence for the predictive validity of the CBCL.
Findings based on psychometric properties of the CBCL across cultures provide data that facilitate the conceptualization of children's emotional and behavioural difficulties. Such empirical evidence is essential for the use of a particular instrument across different cultures. However, when using a measure in a new culture, it is essential to interpret results in view of relevant environmental factors which may influence the findings. This perspective is rooted in the emic approach, which stresses that behaviour should be studied from the viewpoint of a particular cultural system to understand the meaning of that behaviour within that system. Thus, before investigating results from a particular culture, it is essential to have an understanding of the society, its cultural values and traditions as well as any relevant factors that may influence results. Since the current study aims to investigate emotional and behavioural problems among Pakistani children, it is essential to have an understanding of the country, its socio-cultural background and environmental factors salient to emotional and behavioural problems.
Brief cultural and socio-political background.
Pakistan is a country of more than 162 million people of diverse ethnic origins (Qamar, 2007), making it the ninth most populous country in the world (Gadit, 2007). Although 95% of the population follows the Islamic Faith, there exists a small westernized elite that wants to encourage secularism, while the majority wish to increase the role of Islam in guiding all aspects of life (Zaman, Stewart, & Zaman, 2006). Pakistan is an impoverished country with considerable internal political disputes. Additionally, due to little development and high levels of regional and political tensions, long- term prospects also remain uncertain (Qamar, 2007). Similar to other developing countries, economic underdevelopment and extreme misdistribution of resources has lead to a huge, mainly rural underclass that is deprived of good education as well as access to amenities.
Provisions for educational and health services are insufficient for the existing population. Literacy rates are very low; the overall literacy rate is 42.7 %, with great variations between gender and urban and rural populations (Zaman et al., 2006). There are 20.39 million children enrolled in schools with more boys enrolled than girls (Karim, Saeed, Rana, Mubbashar, & Jenkins, 2004). In terms of the educational setup, Pakistan has several parallel yet completely different educational systems. These include private schools, government supported public schools, public/community schools run by non-governmental organizations (NGOs), as well as religious schools or madrassas. The madrassas are not typical schools; they are run by mosques and children are instructed in religious texts only. A majority of the children in Pakistani cities attend either government public or NGO run community schools.
As far as availability of resources for health care is concerned, there is very little allocation of funds for health and a minuscule amount for mental health services (Karim et al., 2004). For the entire population, there are 250 psychiatrists, 125 psychiatric nurses, 480 psychologists, and 600 social workers (Gadit, 2007). Child mental health services are highly inadequate. There are only two child psychiatrists and the few existing facilities are only available in the main cities (Syed, Naqvi & Hussain, 2006). There is no formal referral system and patients access services directly through their families or school authorities (Karim et al.).
Salient factors for emotional and behavioural problems in Pakisatni society.
The social structure of Pakistani society, similar to most eastern societies, is collectivistic: the family and not the individual, comes first. This collectivist worldview is reflected in multiple ways, including family structure and dynamics, child-rearing practices as well as the community at large (Zaman et al., 2006).
In Pakistani society, family structure is strictly hierarchical, with power structures clearly defined according to gender and age. Each family member's role is clear and unchanging. Typically, the father is the head of the family and is responsible for the financial needs of the family whereas the mother's responsibility is to manage the home and children (Nath, 2005; Zaman et al., 2006). Extended family systems are quite common while nuclear families (i.e., parents living with children only) are less prevalent (Zaman et al.). The extended family system, where grandparents, uncles and aunts live together with the child and his or her immediate family, has a great impact on daily living in terms of financial needs as well as roles and responsibilities of each individual. As for family dynamics, the elderly, including grandparents and other elders in the extended family command respect and loyalty. The extended family can be a protective factor as other family members can provide help with the supervision of children leading to relief from burden (Syed, Hussein & Yousafzei, 2007). Moreover, it can also be a source of emotional sustenance. On the other hand, the extended family system can be a risk factor for psychological problems if relationships are enmeshed in strife and rivalry. Living in an extended family system can also be a financial burden if there is only one financial provider.
Another salient factor for emotional and behavioural problems is the marital status of parents. Researchers have reported behavioural problems among children from divorced parents versus intact families in Western societies (Achenbach & Rescorla, 2007; Harland et al., 2002; Turk et al., 2007). In Pakistani society, divorce is not readily accepted, and divorced mothers may have to deal with many more difficulties (e.g., monetary and social) compared to parents from intact families in the patriarchal Pakistani society. These issues may influence the psychological health and development of the child. Moreover, although rare and not accepted socially, polygamy does exist, which may have its own influence on children's emotional well-being. Multiple siblings may also be an important factor, especially in extended family residences with the average household comprising seven people (Mumford, Saeed, Ahmed, Latif, and Mubbashar, 1997). Large families are more likely to suffer economic hardship and parental attention is less readily available. Moreover, having multiple siblings may lead to the child feeling neglected or lost in the family (Turk et al.).
The educational level and the occupational status of parents have also been associated with emotional and behavioural problems among children (Achenbach & Rescorla, 2007). The adverse living conditions associated with low SES and low levels of parental education have been linked to the rise of psychological disorders (Turk et al., 2007). Parents who lack occupational and educational skills may be over burdened by financial concerns and may not be able to cater to the needs of their children. Due to lack of awareness, they may be less likely to identify psychological problems. Similarly, in Pakistani society, parents' educational level and occupational status may determine both their parenting practices as well as the economic burden on them. Both these factors may be linked to children's emotional and behavioural development.
In terms of child rearing and socialization, Pakistani society places a high degree of emphasis on obedience, fulfillment of duties and obligations instead of the needs or rights of individuals (Zaman et al., 2006). Therefore, parenting practices promote respect for elders as well as interpersonal harmony (Stewart et al. as cited in Zaman, 2006). Furthermore, any explicit expression of negative feelings towards parents and elders is not approved (Zaman et al.). Parental control and warmth coexist, with more control for females than males. Maintaining family izzat or honour and social approval are essential aspects of childrearing (Stewart et al., as cited in Zaman et al). Due to these cultural norms, Pakistani children are often socialized to be conforming and obedient. With regard to the female child, many girls internalize the expectations imposed upon them by the patriarchal society, which include learning household tasks and becoming child caretakers (Nath, 2005). It is pertinent to mention that although Pakistani culture, similar to other eastern cultures, is based largely on a collectivist approach, the availability of satellite dishes, the internet and western films have exposed Pakistani children to Western culture. Moreover, in this age of mass communication, cultures are open to foreign influence and are not isolated phenomena.
In Pakistan, the declining educational and public support systems, coupled with inflation and unemployment have led to high levels of frustration and despair in society. Furthermore, the changing political scenario, economic instability, a low literacy rate and a high population growth rate contribute to the gambit of issues, each of which influences the psychological makeup of the community and indirectly that of the children. For example, increased levels of violence have resulted in the proliferation of weapons and an increased level of insecurity in the population (Karim et al., 2004). Research indicates that exposure to community violence is associated with emotional and behavioural problems among children (Overstreet & Mazza, 2003). Exposure to community violence may influence the child indirectly (i.e., via its influence on the family) as well as directly, since it may have an impact on the child's perception of safety and his/her emotional regulation (Margolin & Gordis, 2000).
In developing countries, high mortality rates indicate that children are exposed to certain life events more often than their counterparts in developed countries. These life events include serious illnesses among children, illnesses and deaths among close relatives, and exposure to violence. Moreover, due to the absence of a proper welfare system, the economic hardships faced by families in developing are of a much more severe nature than those faced by families in developed countries (Hackett & Hackett, 1999).
At the cultural level there are a few salient factors that may influence reporting of symptoms. First, there is a general lack of awareness of psychological problems from a medical perspective (Sammad et al., 2005). Instead, there are different meanings attributed to psychological symptoms (Kirmayer, Groleau, Guzder, Blake, & Jarvis, 2003). Psychological problems are believed to be due to supernatural forces, such as possession by demons, black magic or punishment for sins (Gadit, 2007; Karim et al., 2004; Nath, 2005). Numerous people initially consult faith healers and spiritual leaders before consulting mental health professionals (Syed et al., 2007). Therefore, lack of awareness may lead to a non-medical approach to illness, which may influence the parent's reporting.
Culture also influences the attitude toward self-disclosure (Gary, 2005). Even in immigrant South Asian communities, such as the one in the U.K., referral rates of children to psychiatric clinics are lower than expected (James et al., 2002; Sammad et al., 2005). In Pakistani society psychological problems are considered as personal and are highly stigmatized. They are viewed as shameful to the individual as well as the family. Although no studies have documented the degree of stigmatization, it is widely acknowledged that there may be discrimination against psychological problems in multiple spheres of life such as education, employment, health care as well as social opportunity (Karim et al., 2004). According to Syed et al. (2007), the high levels of illiteracy, ignorance, indifference, and intolerance in society contribute towards stigmatization of psychological problems. Therefore, Pakistani parents may be hesitant to disclose issues and may minimize problems. Need for research.
At present there is very little research on child psychopathology or the nature of emotional and behavioural problems among Pakistani children. This lack of research may be attributed to a lack of awareness, paucity of resources, as well as dearth of appropriate assessment measures (Karim et al., 2004). Therefore, there is a strong need to conduct community based studies to investigate the prevalence of emotional and behavioural problems among children in Pakistani society. To date, very few studies have explored prevalence rates of psychological problems among school children in Pakistan. Using the Rutter scale, Javad and colleagues (1992 as cited from Sammad, 2005) found that the most common problem among children in the city of Lahore was antisocial behaviour (9.3% prevalence rate). Sammad et al (2005) tested the validity of the Urdu version of the Strengths and Difficulties questionnaire (SDQ) based on a clinical and a paediatric sample. Although Sammad and colleagues (2005) reported that the translated SDQ had adequate sensitivity for each disorder, the specificity could not be ascertained as a community sample was not included. Moreover, prevalence rates were not reported. Syed (2007) compared the validated version of the SDQ and a translated version of the CBCL in a Karachi-based population and reported that there was moderate correlation between the SDQ and the CBCL. The only recent community based prevalence study of emotional and behavioural problems among children was conducted by Syed, Hussein, and Mahmud (2007). Syed and colleagues (2007) used the SDQ to investigate prevalence rates in a community sample of 5 to 11 year olds in Karachi. They reported that based on the parents forms of the SDQ about 47% of the children were rated as normal, 19% as borderline and 34% as abnormal. In 2006, Syed and colleagues also examined prevalence rates among a clinic- based Karachi population. They found that among the 290 referrals, the most common reason for referral was Aggressive Behavior, although this possibly masked other kinds of behaviour. Moreover, they reported that ADHD was the most common disorder, diagnosed in 25 % of the children referred to the clinic. Apart fromthe studies mentioned above, there is a lackof research findings fromother parts fo the country which makes it difficult to plan and utilize the limited resources effectively.
The lack of research in Pakistan highlights the need to investigate prevalence rates of emotional and behavioural problems. A review of the few existing studies indicates that the focus has primarily been on clinical samples with the exception of the community based study using the Strengths and Difficulties questionnaire. However, unlike the current study which covers tha age range of 6 to 14 years, the Karachi based study covered the ages of 5 to 11 years. Moreover, unlike the Karachi based study, the current study will be the first to report age or gender patterns of emotional and behavioural problems among Pakistani children. Since the current study aimed to investigate emotional and behavioural problems among both boys and girls and covers a broad age range (6- to 14-year-olds) such information will be essential to the advancement of knowledge about Pakistani children. Moreover, another aim of the present study was to investigate the internal consistency and construct validity of the CBCL, thereby making it available for use by local clinicians and researchers. The current study was also the first to compare Pakistani findings with U.S. findings. Such comparisons facilitate cross-cultural communication and research, and also lead to the advancement of knowledge.
Objectives of the Current Study
The current study had multiple objectives. Each objective is listed below, and is followed by a brief description of the aforementioned literature review relevant to that particular objective.
Objective 1. :
To provide prevalence data on specific emotional and behavioural problems among Pakistani children.
Investigating the prevalence of emotional and behavioural problems is important for several reasons. First, they affect multiple aspects of children's functioning, such as academic performance and social adjustment (Montague et al., 2005; Vitaro et al., 2005). Second, such problems may persist and affect functioning during adulthood (Maughan & Kim-Cohen, 2005). And, finally, although there have been numerous studies across cultures, very little is known about the prevalence rates of emotional and behavioural problems of Pakistani children. The current study is the first to examine prevalence rates in a community sample with an age range as wide as 6 to 14 years. At the international level, results from Pakistan can contribute to a better understanding and add to the existing knowledge base about children's emotional and behavioural problems (Achenbach & Rescorla, 2007; Waddell et al., 2002). In addition, measures of child psychopathology with proven psychometric properties are strongly needed in a country such as Pakistan, where mental health resources are scarce. Within Pakistan, such epidemiological data would help raise awareness and assist in the planning of intervention programs.
Evaluate the effects of Gender and Age on emotional and behavioural problems.
The evaluation of the effects of Gender and Age on emotional and behavioural problems may be important for the following reasons. First, such data would add to the existing knowledge base about variations in emotional and behavioural problems. From a research perspective, it would be valuable to determine if emotional and behavioural problems among Pakistani children reflect patterns similar to those in other cultures. This information would help develop a unified theory of children's emotional and behavioural problems. From a clinical perspective, such findings would be useful to determine high-risk groups and develop intervention programs accordingly.
Based on gender differences present across cultures (e.g., Rescorla et al., 2007), it is hypothesized that there will be significant differences between girls and boys on the type of problem behaviour. Girls will have significantly more Internalizing Problems than boys whereas boys will have significantly more Externalizing Problems than girls.
Based on Rescorla et al.'s (2007) findings that included epidemiological data from diverse cultures, it can also be hypothesized that with regard to age, Externalizing Problems may decrease with age whereas the converse may hold true for Internalizing Problems. As the current sample consists of three age groups (i.e., 6-8 years, 9-11 years, and 12-14 years), it is hypothesized that there will be significant differences between the 6-8 year-olds and 12- to 14-year-olds, with a higher level of Internalizing Problems in the older group. It is also hypothesized that there will be significant differences between these two groups in terms of Externalizing Problems, with a higher level of Externalizing Problems among the younger age group. These two groups (i.e., the 6- to 8-year-olds and 12- to 14-year-olds) were chosen for the comparison as they would be able to capture developmental differences as opposed to comparing 6- to 8-year olds to 9- to 11-year olds. The rationale for using these two age groups is also based on results from the multi-cultural studies. In the Crijnen et al, (1999) study, there was a significant main effect for Age in the 9 culture group which ranged from 6 to 17-year-olds, whereas there were no significant age differences in the 12 culture group which ranged from 6- to 11-year-olds. Similarly, in the Rescorla et al. (2007) study, there was a significant main effect of age for societies with samples that spanned from 6 to 16 years and not so for societies based on samples ranging in age from 6- to 11-year-olds.
Determine the internal consistency as well as construct validity of the Urdu CBCL.
Prior to utilizing findings based on a measure for clinical or research purposes, it is essential to assess the reliability and validity of the measure. Numerous researchers have investigated the psychometric properties of the CBCL in different cultural settings. Recently, Rescorla et al. (2007) and Ivanova et al. (2007) have evaluated the internal consistency and construct validity of the CBCL across 30 societies and have stated that the CBCL has sound psychometric properties across cultures. Ivanova et al. (2007) also stated that these findings also provide conceptual templates for emotional and behavioural problems across different cultures. In theory, using an instrument in a new language and culture can help evaluate the extent to which the knowledge base of children's assessment can be generalized cross-culturally, which is an important step in constructing a unified theory of developmental psychopathology (Greenbaum et al., 2004). From a practical perspective, cross-culturally robust measures are essential as they are tools to identify the needs of children such as refugees and recent immigrants in the emerging global society (Crijnen et al., 1997). At the individual country level, at present the CBCL has not been validated in Pakistan. Validation of the Urdu CBCL will be a great resource for local practitioners and researchers in clinical and educational settings.
Compare Pakistani and American findings.
Cross-cultural comparisons are useful in identifying similarities and disparities in emotional and behavioural problems across countries (Crijnen et al., 1997). Such comparisons help in providing valuable insight into culture-specific aspects of universal psychological constructs (Crijnen et al., 1999). Moreover, comparison of factors such as age and gender can provide additional information regarding the correlates and course of different emotional and behavioural problems. At present, there is no study that has compared Pakistani findings to American data, the current study was the first to make a cross-cultural comparison of children's emotional and behavioural problems.
Dearth of research on emotional and behavioural problems among Pakistani children makes it difficult to make predictions about differences between Pakistani and American children. In view of the Problem Suppression-Facilitation model of cultural influence, children from traditional, collectivistic cultures, where immense importance is given to obedience and interpersonal harmony, may have more Internalizing Problems than Externalizing Problems as compared to children from individualistic societies, where children may have more Externalizing Problems than Internalizing Problems. Based on this theoretical model, it could be hypothesized that Pakistani children, belonging to a collectivist culture, may have more Internalizing Problems than Externalizing Problems. However, there is not enough research to support the model beyond the Thai-U.S. comparisons. In the recent multicultural study, Rescorla et al. did not report on differences due to cultural orientation for Internalizing and Externalizing Problems.
In the multicultural study, Rescorla et al. (2007) suggested that cultural differences on Total Problems could not be attributed to cultural orientation (i.e., individualistic or collectivistic societies). Results regarding culturally-based notions of self-presentation as reflected in parents' reluctance to report problems were also mixed as parents in some Asian countries (e.g., China and Japan) reported particularly fewer Total Problems whereas parents from other Asian societies (Hong Kong, Thailand and Korea) did not. Moreover, based on Rescorla et al.'s (2007) findings, cultural differences could not be attributed to factors such as geographic region, ethnicity, religion, sample size, population and economic/political system. However, using an emic approach and exploring factors relevant to Pakistani society, it is apparent that there is a general lack of awareness of psychological problems in Pakistani society (Sammad et al., 2005). Psychological problems are attributed to supernatural forces such as black magic, spirit possession or punishment for sins. Syed and colleagues (2007) indicate that high levels of illiteracy, ignorance, indifference and intolerance contribute stigmatization of psychological problems. Karim et al. (2004) also report that there is considerable stigma associated with psychological problems in various aspects of life. Therefore it is hypothesized that owing to the general lack of awareness, coupled with the stigmatization associated with psychological problems in the conservative Pakistani society, Pakistani mothers may report significantly fewer Total Problems than their American counterparts.