Abstract
Background: Research on depression among Caribbean children has been limited by a lack of valid and reliable measures. We addressed this problem by exploring the internal consistency reliability and the concurrent and discriminant validity of the Kutcher Adolescent Depression Rating Scale (KADS) among a wide cross-section of the student population attending elementary schools in Jamaica and Barbados.
Methods: Students enrolled in grade 6 in a cross-section of schools in Jamaica and Barbados were invited to participate in the study. Schools included a balance of government-funded public schools and privately funded preparatory schools. All schools that were invited to participate accepted, and all grade 6 classrooms in each school were sampled. The following instruments were administered to the students during the fall semester of 2015: KADS, Adolescent Depression Rating Scale (ADRS), major depression disorder subscale of the Revised Children’s Anxiety and Depression Scale (RCADS), and state and trait anxiety visual analogue scales.
Results: In total, 759 children (376 girls [49.5%] and 363 boys [47.8%]; in 20 cases [2.6%], sex was missing) with a mean age of 10.7 (standard deviation 0.66) years (median 11.0 yr) took part in the study. Overall, the KADS had an acceptable degree of reliability (α = 0.76). The instrument had reasonably good concurrent validity, as evidenced by strong correlations with scores on the ADRS (r = 0.62) and the major depression disorder subscale of the RCADS (r = −0.61). It had acceptable discriminant validity, as shown through low correlations with the state and trait anxiety visual analogue scales (r = 0.21 and 0.18, respectively). This pattern of results suggests that a large part (37%) of the variance underlying the KADS assesses depression, and a smaller degree of the variance (3%–4%) measures a conceptually similar but distinct concept.
Interpretation: The KADS is a reliable and valid measure for assessing depressive symptoms among Jamaican and Barbadian elementary school students. The sample may not be representative of all Caribbean children.
Across the world, 10%–20% of children and adolescents have substantial mental health problems.1 Elementary school children with depression are at greater risk for attempted and completed suicide than their healthy peers.2 Depressive disorders that occur during childhood and adolescence have documented high rates of substance abuse, teenage pregnancy and delinquency.3–6 Few investigations of depression have been performed among West Indian elementary school students, and even fewer among Jamaican and Barbadian students.7 Lowe and colleagues8–14 found that a relatively large proportion of adolescents in Caribbean secondary schools reported moderate to severe symptoms of depression, but little research has been done with preadolescents in this region.
An important prerequisite for conducting research on depression among Jamaican and Barbadian preadolescents is the availability of an instrument that is valid for this group. It is well established that cultural factors influence perceptions and interpretations and, therefore, the validity of screening tools for depression.15 The applicability of measures of depression developed elsewhere in the world must be established before any large-scale research can take place. Unique cultural factors of Jamaica and Barbados, including overt academic streaming, high-stakes examinations, the implications of attending a prestigious high school for future occupational and social success, and the influence of the educational system in reproducing and maintaining a highly stratified social class system, may confer a greater risk of depression among preadolescents than in other settings. These factors make the identification of an appropriate instrument for detecting and measuring depression critical. The purpose of the current study was to assess the reliability and validity of a measure of depressive symptoms developed at Dalhousie University, Halifax, the Kutcher Adolescent Depression Scale (KADS), when used with this population. We hypothesize that the KADS will achieve concurrent, discriminant validity in a sample of Jamaican and Barbadian students and that, consistent with past literature,5–7 we there will be no sex difference in preadolescents’ scores.
Methods
Setting
Jamaican society
Several social factors may place Jamaican elementary school students at elevated risk for depression. Jamaican society is strongly stratified by social class,16 which negatively affects the educational system,17 such that children of parents from lower social classes are enrolled predominantly in government-managed schools, which are less well staffed and resourced than privately funded schools.8,13,17,18 Compounding this, repeated critical competency examinations, such as the Grade Six Achievement Test, in combination with high levels of poverty and unemployment and limited opportunities for social mobility may elevate the risk of depression for students.19 Given this enhanced risk, it is important to determine the characteristics and prevalence of depression among Jamaican elementary school students.
Barbadian society
Like the system in Jamaica, the Barbadian education system requires all students to sit a secondary school entrance examination (Common Entrance Examination) at 11 years of age. Students are then streamed to schools of varying quality based on their performance. The practice of streaming has arguably served to maintain the rigid stratification of social classes within the Barbadian context.20 This is primarily because working-class children tend to have inadequate parental encouragement and resources and are more likely to speak the local dialect. Hence, children of working-class parents are less likely to obtain a place at the older, more prestigious secondary schools.
Consequences of high-stakes examinations
Given the degree to which the practice of high-stakes entrance examinations is entrenched within the fabric of Jamaican and Barbadian culture and the societal pressures placed on children to succeed in examinations, the failure to achieve a place at a prestigious school can have detrimental psychological consequences for children.21 Some parents have described their child’s experience of the Grade Six Achievement Test and the Common Entrance Examination as traumatic, and many children who have taken these examinations have been labelled as successes or failures by their performance; in the case of the latter, this has undermined their self-esteem.17,22 Perceived failure can also result in “alienation from parents, family and past school friends,”23 all of which can increase the predisposition for depression among this demographic group.
Study design
The overall project was a 4-group, 2-pretest, 1-posttest quasi-experiment. In the current study, we used data only from the first observation period, as it included measures to validate the KADS.
Participants
Students attending a cross-section of elementary schools in Barbados and in the Kingston and St. Andrews areas of Jamaica were assessed to obtain preliminary evidence for the validity of the KADS. Schools were chosen such that they included a balance of government-funded public schools and privately funded preparatory schools in both Jamaica and Barbados. We tried to ensure that the schools were roughly proportional in enrolment to those in their respective country.24,25 No schools that were invited to participate declined. All grade 6 classrooms in each school were sampled.
In both Jamaica and Barbados, the sixth grade is the final grade in elementary school. High schools in both countries include grades 7–11, with some schools offering an extended 2-year program (grades 12 and 13, similar to the Ontario Academic Credit), which is considered to be the equivalent of the first year of college.
Measures
Kutcher Adolescent Depression Scale
The KADS is an 11-item measure specifically designed to assess the cognitive, behavioural, affective and somatic symptoms of depression by self-report among students 12–17 years of age.26 It was developed to assess the core symptoms of adolescent depression using Standard English and the phraseology often used by adolescents. The scale has been standardized, and evidence of its validity has been obtained for depressive symptoms in several North American and Middle Eastern countries.27,28 Subjects record their responses to each item using a series of Guttman rank-ordered statements. The statements are assigned a score of 0–3 depending on the severity of the symptom they describe. Subjects are asked to circle the number associated with the statement that most accurately describes their feelings. Depression scores are calculated by summing the numbers associated with the circled statements. Within our sample, the KADS appeared to have a reasonable degree of internal consistency reliability (α = 0.76). Past research suggested that the KADS has adequate concurrent validity, as indicated by a correlation of 0.69 with the Children’s Depression Rating Scale.26
Adolescent Depression Rating Scale
The Adolescent Depression Rating Scale (ADRS) is a 10-item measure of depressive symptoms for use with adolescents.29 Subjects are asked to indicate whether each item is true or false for them. The ADRS has been found to have acceptable levels of internal consistency reliability (α = 0.74–0.79) as well as good concurrent validity, as shown by large correlations with the Hamilton Depression Rating Scale (0.56),30 the Beck Depression Inventory-II (0.82)31 and the Clinical Global Impression — Severity Scale (0.52).32 Within our sample, the ADRS had borderline internal consistency reliability (α = 0.58).
Revised Children’s Anxiety and Depression Scale
The Revised Children’s Anxiety and Depression Scale (RCADS) is a 47-item measure that assesses symptoms of anxiety and major depressive disorder. The 10-item major depression disorder subscale has an internal consistency reliability of 0.80 for clinical samples and 0.79 for school-based samples of children 6–18 years of age.33 Within our sample, the major depression disorder subscale was found to have an acceptable level of internal consistency reliability (α = 0.71). Past research showed that the RCADS was able to successfully distinguish students with a diagnosis of an affective disorder from those without.33
Visual analogue scale — anxiety
We measured anxiety using a 100-mm horizontal line with the 2 end points labelled “no anxiety or fear” and “worst possible anxiety or fear.”34 Participants indicated their level of anxiety by marking a spot on the horizontal line. The measure correlated 0.62–0.69 with established measures of anxiety (State–Trait Anxiety Scale — Children, State–Trait Anxiety Scale — Youth and Yale Preoperative Anxiety Scale), which suggests that it has adequate concurrent validity.34 Participants were asked to rate their current level of anxiety (state anxiety) and their level of anxiety during the previous 6 months (trait anxiety).
Rationale
We included both the ADRS and the major depression disorder subscale of the RCADS in this study as both are commonly used measures of preadolescent and adolescent depressive symptoms, they are brief measures but also have strong psychometric properties (reliability and concurrent validity), and they are accepted in their field. The major depression disorder subscale was intended to be the gold-standard/reference measure, and the ADRS was included as an additional measure of validity. In validating a measure it is always important to include at least 2 different measures of the construct being validated, because if only 1 measure is included, indeterminate findings can occur. This is because a weak relation with 1 reference measure may simply indicate that the 2 measures assess different facets of the same construct and not that they validly assess same construct.35
Procedure
During the fall semester of 2015, research assistants provided a brief introduction to the project at the start of the participants’ regular class. The project was described as an investigation into the psychosocial factors associated with mood and social relationships. Participants were informed that their participation was completely voluntary and they could withdraw from the project at any time. Questionnaires were then distributed, with 45 minutes allowed for completion. Completed questionnaires were returned to one of the investigators at the end of the class.
Definitions
For the purposes of this study, we defined psychometric reliability as the stability of responses within the same testing session, as assessed by the Cronbach coefficient α.36 Concurrent validity for the purposes of this study was operationalized by a large correlation of the KADS with the ADRS and the major depression disorder subscale of the RCADS. Discriminant validity is the degree to which a measure does not correlate with a measure of a similar but conceptually distinct construct.35 Discriminant validity was operationalized by a low correlation with the anxiety visual analogue scale. Although there are standards for interpreting and evaluating psychometric reliability, there are no standards for judging the concurrent or discriminant validity of a measure. Instead, concurrent and discriminant validity are evaluated based on the pattern of correlations between measures. In this study, for concurrent validity, there should be moderate correlation of the KADS with both the ADRS and the major depression disorder subscale of the RCADS, and low correlation of the KADS with the anxiety visual analogue scale.
Statistical analysis
We entered the data obtained from the questionnaires into SPSS version 11.5 (IBM) and analyzed them using the Pearson product-moment correlation, interitem reliability statistics, t tests and principal components analysis. We used a 2-stage approach to establish the reliability and validity of the KADS. First, we examined the internal consistency reliability of the instrument using the Cronbach coefficient α.36 We did so because the KADS was developed to assess change in depressive symptoms. As such, an estimate of internal consistency reliability is a better estimation of the reliability of scores than a test–retest reliability coefficient. We then examined the concurrent and discriminant validity of the KADS using the Pearson product-moment correlation. We used this statistic to estimate concurrent and discriminant validity because it assesses the extent to which scores on the KADS will covary with scores on the ADRS, the major depression disorder subscale of the RCADS and the anxiety visual analogue scale. We used the established guidelines set by Anastasi and Urbina37 and Nunally38 to evaluate the reliability of the study measures. Following these guidelines, a reliability coefficient of 0.60 or greater indicates an acceptable degree of reliability, a coefficient of 0.61–0.79 indicates moderate reliability, and a coefficient of 0.80 or greater indicates good reliability.37 There are no established guidelines for the judgment of validity. We evaluated validity in terms of the amount of variability shared between 2 measures of the same construct. To examine the dimensionality of the KADS, we conducted a principal components analysis of the items on the KADS using varimax rotation of components. We did so because this procedure examines the total variability rather than the common variance among the items. Before conducting all analyses, we substituted the mean score, rounded to the nearest whole number, for missing values on individual KADS items, and ADRS, major depression disorder subscale of the RCADS, and items on the state and trait anxiety visual analogue scales. We used several criteria to determine the number of components to keep (eigenvalues > 1, scree plot, total variability accounted for by the solution, simple structure and psychological meaningfulness).39
Ethics approval
This study was approved by the ethics committees of the Mona and Cave Hill campuses of The University of the West Indies. All parents and children were asked to provide their written informed consent and assent before collection of data.
Results
In Jamaica, all students who were present on the day of data collection were included in the sample. In Barbados, only 38% of the students whose parents consented took part in the study, as some children did not complete the informed assent form or were not present on the day of data collection. In total, 759 children (376 girls [49.5%] and 363 boys [47.8%]; in 20 cases [2.6%], sex was missing) aged 9–12 years (mean 10.7 yr [standard deviation (SD) 0.66 yr], median 11.0 yr) took part in the study (Figure 1). The distribution of students in our sample by grade and sex closely represented that for the government schools (Table 1). However, there were slightly more male students (48.7% v. 46.4%) and slightly fewer female students (51.3% v. 53.6%) in private schools in our sample than in the general population.
Overall, results of the validity analyses suggested that the KADS has an acceptable degree of concurrent and discriminant validity (Table 2). Scores on the KADS correlated with the major depression disorder subscale of the RCADS (r = 0.62) and with the ADRS (r = 0.61), which suggests that the KADS has a moderate degree of concurrent validity. In contrast, KADS scores correlated weakly with scores on the state and trait anxiety visual analogue scales (r = 0.21 and 0.17, respectively), which suggests that the KADS has an acceptable degree of discriminant validity. This pattern of scores suggests that a large part of the stable variance underlying the KADS assesses depression (37%) and that a smaller degree of the variability (3%–4%) measures a conceptually similar but distinct construct.
As expected, there was no significant sex difference in KADS scores. Girls had a mean score of 10.76 (SD 6.66), and boys had a mean score of 10.10 (SD 5.93).
We compared KADS scores between public and private schools. Students in public schools were found to have significantly higher KADS scores than those in private schools (11.74 [SD 6.24] v. 9.62 [SD 6.63], p < 0.001).
On principal components analysis of the KADS, 2 clear components emerged that accounted for 39.2% of the total variability in the participants’ scores. Component 1 consisted of cognitive–somatic symptoms of depression, and component 2 consisted of affective symptoms of depression (Table 3).
Interpretation
We found that the KADS had an acceptable level of concurrent and discriminant validity in Jamaican and Barbadian schoolchildren aged 9–12. It also had an acceptable level of internal consistency reliability. Thus, the instrument appears to be an appropriate measure of depression among preadolescent Caribbean children. As expected, we did not find a sex difference in KADS scores.
Our findings are similar to those of validation studies conducted in Iran,28 the Philippines40 and China.41 The KADS had comparable levels of internal consistency reliability and similar levels of concurrent validity, and consisted of 2 principal components that were similar in structure to ours.
Limitations
One limitation of the current project is the sampling of sixth-grade schoolchildren. This population is not representative of the full range of children and adolescents in Jamaica and Barbados. We plan to conduct further studies using a broader and more representative sample of the Jamaican and Barbadian population. Second, the KADS was designed to assess youth aged 12–17. However, in the current study, students ranged in age from 9 to 12 years. As is standard practice for dealing with missing values on specific variables in data sets where data are missing for less than 5% of all cases on a specific variable,39 the mean score for the specific variable was substituted for the missing value on that specific variable. Consequently, the variability of the study variables may have been slightly truncated, thereby underestimating reliability and validity values. Despite these limitations, the current study provides preliminary evidence for the concurrent and discriminant validity of the KADS. Our principal components analysis accounted for only 39% of the total variability among the KEDS items. Although this is more than acceptable and is often found in analysis of measures, it suggests that a substantial amount of the variability in the KADS may be random. Researchers who revise the KADS in the future may wish to reduce the random variability in the measure.
Conclusion
The KADS is a reliable and valid measure of depressive symptoms for children aged 9–12 years that may be used to screen Jamaican and Barbadian youth. We plan to examine the role that parenting practices, academic school performance, neighbourhood factors and socioeconomic status may play in levels of depression as measured with the KADS.
Footnotes
Competing interests: Stanley Kutcher is the creator of the Kutcher Adolescent Depression Rating Scale instruments. No other competing interests were declared.
This article has been peer reviewed.
Contributors: Garth Lipps and Gillian Lowe contributed to the conception of the project, designed the project, collected the data in Jamaica, analyzed and interpreted the data, and contributed to the drafting of the manuscript. Roger Gibson and Mia Jules contributed to the design of the project, interpretation of the data and drafting of the manuscript. Stanley Kutcher contributed to the conception of the project and critically revised the manuscript for important intellectual content. Mia Jules collected the data in Barbados. All of the authors gave final approval of the version to be published and agreed to be accountable for all aspects of the work.
Funding: This work was supported by New Initiatives grant 15112P, Principal’s Office, The University of the West Indies.
Supplemental information: For reviewer comments and the original submission of this manuscript, please see www.cmajopen.ca/content/6/3/E248/suppl/DC1.
References
- Copyright 2018, Joule Inc. or its licensors