Development of the Bullying and Health Experiences Scale
Background: Until recently, researchers have studied forms of bullying separately. For 40 years, research has looked at the traditional forms of bullying, including physical (eg, hitting), verbal (eg, threats), and social (eg, exclusion). Attention focused on cyberbullying in the early 2000s. Although accumulating research suggests that bullying has multiple negative effects for children who are targeted, these effects excluded cyberbullying from the definition of bullying.
Objective: This paper responds to the need for a multidimensional measure of the impact of various forms of bullying. We used a comprehensive definition of bullying, which includes all of its forms, to identify children who had been targeted or who had participated in bullying. We then examined various ways in which they were impacted.
Methods: We used an online method to administer 37 impact items to 377 (277 female, 100 male) children and youth, to develop and test the Bullying and Health Experience Scale.
Results: A principal components analysis of the bullying impact items with varimax rotation resulted in 8 factors with eigenvalues greater than one, explaining 68.0% of the variance. These scales include risk, relationships, anger, physical injury, drug use, anxiety, self-esteem, and eating problems, which represent many of the cognitive, psychological, and behavioral consequences of bullying. The Cronbach alpha coefficients for the 8 scales range from .73 to .90, indicating good inter-item consistency. Comparisons between the groups showed that children involved in bullying had significantly higher negative outcomes on all scales than children not involved in bullying.
Conclusions: The high Cronbach alpha values indicate that the 8 impact scales provide reliable scores. In addition, comparisons between the groups indicate that the 8 scales provide accurate scores, with more negative outcomes reported by children involved in bullying compared to those who are not involved in bullying. This evidence of reliability and validity indicates that these scales are useful for research and clinical purposes to measure the multidimensional experiences of children who bully and are bullied.
Interact J Med Res 2012;1(2):e13)
Encountering bullying behavior is one of the most distressing experiences for children and adolescents, particularly when such behaviors occur over a prolonged period of time [, ]. Bullying behavior is strongly associated with a number of detrimental cognitive, psychological, health, and behavioral outcomes that can persist into adulthood [ , ]. Although researchers have studied bullying since the 1970s [ ], few measures of children’s functioning related to bullying exist. Considering that its newest form, cyberbullying, occurs electronically, it seems reasonable to consider developing a bullying and health experiences scale that could be administered electronically. The purpose of this study was to develop, pilot, and evaluate an online, multidimensional instrument to assess the health experiences of children who are bullied and/or bully others.
Health Experiences Associated with Bullying
A substantial body of research supports specific deleterious correlates of bullying . In terms of cognitive functioning, victims of bullying report lower school attachment and lower academic achievement compared to children not involved in bullying [ , ].
Researchers have also documented psychological and health factors related to bullying. Findings from a national study that examined 123,227 children 11, 13, and 15 years of age residing in 28 countries found bullying was strongly related to psychological symptoms, including nervousness, sleep difficulty, and feelings of rejection, loneliness, and helplessness . Other studies have found that frequent exposure to bullying is significantly related to internalizing disorders, with signs and symptoms of anxiety, depression, social withdrawal, diminished self-esteem or diminished sense of self-worth, and suicide ideation [ , , ]. Fekkes [ ] extended previous research inquiries into the relationship between mental health symptoms and bullying using longitudinal data. Findings suggest that children who were regularly bullied at the beginning of the school year were more likely to develop new mental health and health-related symptoms throughout the year, including depression, anxiety, bed-wetting, abdominal pain, and tension [ ]. Similar to findings related to traditional bullying, children who are bullied online are significantly more likely to experience psychological symptoms than children not involved in bullying [ ].
In addition to these cognitive and psychological symptoms, many researchers have demonstrated a number of behavioral problems related to bullying. Similar to victims of traditional bullying, victims of cyberbullying are likely to display externalizing behaviors, such as drug and alcohol use . In addition, they are more likely to report carrying a weapon to school compared to children who have not experienced online bullying [ ]. Other studies have found that frequent exposure to bullying experiences is linked with higher reports of eating disorders, such as bulimia nervosa, among female youth [ ].
Although they may be disliked by the peers they target, aggressive children experience average levels of popularity . Additionally, they do not necessarily exhibit low levels of self-esteem [ ], are not highly anxious [ , ], nor are they highly depressed [ ]. Nevertheless, the negative effects of bullying are not restricted to victimized children. Research has also documented negative health experiences for children who perpetrate the bullying. For example, they tend to have cognitive difficulties such as poor academic achievement [ , ]. They also tend to require mental health services [ ]. They may exhibit conduct problems, such as vandalism [ ], smoking [ ], and drinking [ , ]. In addition, Holmes and Brandenburg-Ayres [ ] reported that early bullying experiences significantly predicted later gang membership and incarceration.
In addition to children who bully and those who are victimized, there exists a group of children who both bully and are victimized (bully/victims). Hanish and Guerra  described these children as having the most disturbed functioning and note that they “are more likely to have emotional, behavioral, social, academic, and family problems.” Children who both bully others and are bullied are considered the most vulnerable to negative developmental outcomes [ ].
Despite years of research, measures typically assess one or two of the cognitive, psychological, or behavioral dimensions discussed. The purpose of our research, therefore, is to create and test a multidimensional measure of the health experiences associated with bullying.
The Internet as a Method of Data Collection
The Internet is an expedient means of transferring information and is recognized as a valid method of collecting qualitative and quantitative data [, ]. It is also becoming an effective means of delivering general and personal information about health [ , ]. It is superior to traditional paper-based survey methods because it allows greater access to respondents through mass distribution and eliminates data entry errors arising from transcribing responses, which are entered into a database [ ]. In addition, online data collection yields higher rates of completion than mail-out surveys [ ]. Although it is not possible to verify the identity of the respondent, the validity of data from any method of data collection is always in question. Recall bias, misinterpretation of questions, fabrication of information, and so on are inherent to all data collection methods. It may be advantageous to use electronic methods with certain populations about certain topics of study. Given that youths ages 12-17 years use the Internet more than any other age group [ ], they are likely to have access to and skill in using it for a variety of purposes. Therefore, it seems appropriate to use the Internet to ask youths questions about their behaviors while on the Internet. Using the Internet as a means of data collection is likely to provide valid and generalizable results about youths’ Internet experiences [ ].
Power and dominance exerted through aggression are endemic in human societies. Although researchers have investigated this phenomenon for decades, multidimensional measures of children’s functioning related to bullying have not yet been developed. The purpose of our study was to comprehensively examine the health experiences of all forms of bullying using a web-based survey, and to determine whether it yields reliable and valid information when compared across groups of children who are bullied, bully others, neither, or both. We predict that children who are bullied and/or bully others are more likely to experience problems across cognitive, psychological, and behavioral domains of functioning than children who are not bullied.
We planned to recruit a convenience sample of 200 children with an open online survey. We selected the Kids Help Phone website for recruitment because of its widespread use among youth . Based on the average of 15,000 unique visits to the Kids Help Phone home page per week, and a response rate of 1%, we estimated it would take two weeks to recruit our sample. We actually recruited 377 subjects in two weeks. The completion rate was 100%. The sample included 377 children (n = 277 girls, n = 100 boys) from age 10 to 17 years (M = 13.90, SD = 1.84). Most children lived in urban (n = 307) compared to rural areas (n = 70); were born in Canada (n = 319) compared to outside of Canada (n = 58, all children were living in Canada at the time of the survey); and spoke English at home (n = 280) compared to another language (n = 97).
The Kids Help Phone website has been in existence since 1996 and provides professional counseling and referrals to people aged 5 to 20 years on any problem or concern. Through word-of-mouth and advertising, many children in Canada have become aware of this resource. Participants were recruited by placing a click-through badge with a message about the survey on the webpage. Visitors were shown a distorted password that they typed in. This password served as a security check to prevent automated programs from accessing the questionnaire. In cases where someone responded twice, we included only the first response.
The Bullying and Health Experiences Scale
Respondents first read the following definition:
There are lots of ways to hurt someone. A person who bullies wants to hurt the other person. A person who bullies does it because they can. They may be older, stronger, bigger, or have other students on their side.
Then respondents were provided with examples of forms of bullying, as shown in.
Children indicated whether and how often they experienced any of the listed forms of bullying within the last month using a Likert scale with anchors from “no” to “several times a week.” A similar question was administered about bullying others.
Children were then administered 37 items asking about their cognitive, psychological, and behavioral experiences. These items were obtained from a review of the research and various measures of children’s functioning. These were rated on a Likert scale according to their frequency using the following anchors: “never,” “only once or twice,” “two or three times a month,” “about once a week,” and “several times a week.” All items included a “no” response option. The questions constituted 8 subscales, as shown in.
For the questions measuring bullying victimization and perpetration, the Cronbach alphas were .77 and .71, respectively, indicating good internal consistency. To summarize, 31.3% of children reported being victimized in some way at least 2-3 times in the past month, and 11.1% reported bullying others at least 2-3 times in the past month. This is consistent with previous research . Pearson product moment correlation coefficients among types of bullying perpetrated (r = 0.06 to 0.64, Mean = 0.28) and experienced (r = 0.016 to 0.54, Mean = 0.33) were low, suggesting that there was little overlap across different forms of bullying.
We analyzed the factor structure of children’s reports of health experiences. The Bartlett’s test of sphericity (Chi-square = 9346.06, P < .001) and the Kaiser-Meyer-Olkin measure of sampling adequacy (0.88) indicated that the sample was sufficient to evaluate the correlations and detect factors. The items were all subjected to principal components analysis with varimax rotation. The reliability was then determined by examining the internal consistency of the items that loaded under each factor. These analyses resulted in an 8-factor solution that explained 68.0% of the variance (see). These factors have eigenvalues greater than 1 and are named anger (Cronbach alpha = .87, 6 items), relationships (Cronbach alpha = .86, 6 items), physical injury (Cronbach alpha = .90, 3 items), risk (Cronbach alpha = .84, 5 items), anxiety (Cronbach alpha = .82, 7 items), self-esteem (Cronbach alpha = .92, 3 items), drug use (Cronbach alpha = .86, 3 items), and eating problems (Cronbach alpha = .73, 4 items). A few of the items had split loadings; however, reliability analyses indicated that they were consistent with other items within the factor, as shown in the table. When an item had a loading of greater than 0.40 on two components, the higher loading was used to assign it to a component [ ].
We compared the scores on these 8 dimensions across several profiles, including bully, victim, non-bully/victim, and bully/victim. First, we classified children as bullies if they reported bullying others 2-3 times per month or more, and being bullied no more than 1-2 times per month. Similarly, we coded children as victims if they reported being bullied 2-3 times per month or more, and bullying others not more than 1-2 times per month. Children who bullied and were bullied 2-3 times per month or more were often coded as bully/victims. We considered children who bullied and were bullied no more than 1-2 times per month as non-bully/victims. This classification system is typically used in the research .
The number of children in each group is shown in. The mean scores were calculated by taking the sum of all the items that loaded under each factor and dividing by the number of items for each factor. Thus, the possible range of mean values in is from 1 to 5. Partial η2 provided effect size estimates in analyses of variance (ANOVAs) and were interpreted using Cohen’s [ ] criteria (.01 = small, .09 = medium, .25 = large). Accordingly, there was a small difference for risk, anger, physical injury, anxiety, and eating problems across the bully classifications. Also, there was a moderate to large difference in relationship experiences. Self-esteem and drug use, however, did not significantly differ across bully or victim groups.
Post hoc analyses using Tukey’s HSD showed several significant differences (P < .001). Victimized children engaged in more risk behaviors, experienced fewer positive relationships and more physical injury, and reported higher levels of anger, anxiety, and eating problems than did children not involved in bullying. Children who admitted to bullying behaviors were more likely to report anger than were non-bully/victims. Bully/victims were more likely to report anger, anxiety, eating problems, and poor peer relationships compared to non-bully/victims. Finally, non-bully/victims were the least likely to report negative experiences.
Based on an extensive review of the literature, we created a 37-item multidimensional scale. Through web-based administration of these items, and analyses of factorial structure and reliability, we obtained evidence of their usefulness when assessing children’s health experiences. The subscales measure risk behaviors, relationship experiences, anger, physical injury, anxiety, drug use, self-esteem, and eating problems. These areas address many of the cognitive, psychological, and behavioral problems of children involved in bullying, as identified in previous research. The reliability coefficients are adequate and scores were significantly elevated for children who were victimized and/or perpetrated bullying.
Significant differences in health experiences emerged among groups according to the type of involvement in bullying. Specifically, children who reported victimization also reported engaging in risk behaviors, such as weapons possession. This is consistent with findings published by Ybarra . Children may be exposed to these risk behaviors through acts of bullying, or they may participate in these risk behaviors as a coping reaction to being bullied. Participation in these risky behaviors probably increases the likelihood of physical injury, as was found in our study. Children who participated in risky behaviors also reported fewer positive relationships with peers, which is consistent with research showing that children who are bullied may experience interpersonal victimization and social skills difficulties [ ]. Several studies have substantiated that children who are bullied experience high levels of anger and anxiety [ , , , ]. Other health problems, such as eating problems involving vomiting or limiting food intake, were more often reported by children who were bullied compared to children not involved in bullying. This finding is also substantiated by previous research [ ].
Similar to children who were bullied, children who perpetrated bullying reported elevated levels of anger, as shown in previous research . They did not report high levels of anxiety, which is also consistent with other studies [ ]. In addition, they reported experiencing positive relationships, which supports research suggesting that bullies perceive themselves as liked by their peer group [ ]. Indeed, bullying attracts attention from the peer group, which bullies may interpret as positive attention. Bullies are, moreover, likely to report similar levels of self-esteem as non-bully/victims [ ], which is consistent with our findings. Contrary to our expectations, children who reported bullying others did not report high levels of risk behaviors. The absence of reported risky behaviors in this group could be related to the relatively young age of our sample. For example, a multinational study of the prevalence of substance use indicates that peak onset of alcohol and cannabis use is in mid- to late adolescence, and at age 18 for all other drugs [ ].
We also included children who were both victims and perpetrators of bullying. Several health concerns were higher for these children than those not involved in bullying. That is, the bully/victim was likely to experience poor peer relationships, anger, anxiety, and eating problems, as suggested by previous research . We found, however, that these children did not report the most negative outcomes, as expected. Rather, victims did. This could be due to the relatively strong self-esteem and positive relationships that the bully/victim reported, which may buffer negative outcomes. Moreover, children who are bullied and victimized typically receive the lowest levels of social support [ ], which may explain the negative outcomes documented in most research. In our study, however, they may have been receiving support from the Kids Help Phone website. This support may have lessened the detrimental correlates of bullying, resulting in lower reporting of these negative behaviors.
Significance and Implications
We chose not to use items about academic achievement and educational experiences because they did not appear to reliably or accurately measure the constructs of our scale. In addition, we did not include questions about depression. We recommend further research in item development on questions pertaining to academic achievement and depression. Moreover, considering that no single source could feasibly assess all aspects of functioning, we recommend that additional instruments be administered to provide a more comprehensive profile, particularly when designing or providing interventions or support. In addition, identifying children as bullies or victims based on experiencing any or all types of bullying created heterogeneous groups. However, given the large number of groups that would have been formed for each type and combinations of types of bullying, it was not possible to compare functioning across all groups. Generalizability of the results may be limited because participants were recruited from the Kids Help Phone website, which they may have been visiting because they were already troubled or seeking support. Although the prevalence of bullying we report is similar to other research, our participants may be unique on other unknown dimensions. Participants filled out a questionnaire via a computer using the Internet. Accordingly, this study was restricted to children who have access to a computer and the Internet. Because the survey was administered online, participants could not ask for clarification or help from the individual administering the questionnaire, which could have affected the accuracy of some responses. Finally, in common with all anonymous online research, it was not possible to verify the information provided by participants.
Despite these limitations, this study provides strong evidence for the usefulness of this multidimensional web-based survey on health experiences related to bullying. It further adds to the growing evidence of the detrimental impact of bullying among multiple profiles of children and youth, whether in the role of bully, victim, or bully/victim.
This research was supported by a Standard Research Grant #410-2007-0671 from the Social Sciences and Humanities Research Council of Canada.
Conflicts of Interest
- Brunstein Klomek A, Marrocco F, Kleinman M, Schonfeld, Gould MS. Bullying, depression, and suicidality in adolescents. J Am Acad Child Adolesc Psychiatry 2007 Jan;46(1):40-49. [CrossRef] [Medline]
- Frisén A, Jonsson AK, Persson C. Adolescents' perception of bullying: Who is the victim? Who is the bully? What can be done to stop bullying? Adolescence 2007;42(168):749-761. [Medline]
- Fosse GK, Holen A. Childhood maltreatment in adult female psychiatric outpatients with eating disorders. Eat Behav 2006 Nov;7(4):404-409. [CrossRef] [Medline]
- Smith JD, Cousins JB, Stewart R. Antibullying interventions in schools: Ingredients of effective programs. Canadian Journal of Education / Revue canadienne de l'éducation 2005;28:739-762. [CrossRef]
- Olweus D. Aggression in the Schools: Bullies and Whipping Boys. Washington: John Wiley & Sons Inc; 1978.
- Hawker DS, Boulton MJ. Twenty years' research on peer victimization and psychosocial maladjustment: A meta-analytic review of cross-sectional studies. J Child Psychol Psychiatry 2000 May;41(4):441-455. [Medline]
- Beran TN, Hughes G, Lupart J. A model of achievement and bullying: Analyses of the Canadian National Longitudinal Survey of Children and Youth data. Educational Research 2008;50(1):25-39. [CrossRef]
- Spriggs AL, Iannotti RJ, Nansel TR, Haynie DL. Adolescent bullying involvement and perceived family, peer and school relations: Commonalities and differences across race/ethnicity. J Adolesc Health 2007 Sep;41(3):283-293 [FREE Full text] [CrossRef] [Medline]
- Due P, Holstein BE, Lynch J, Diderichsen F, Gabhain SN, Scheidt P, et al. Bullying and symptoms among school-aged children: international comparative cross sectional study in 28 countries. Eur J Public Health 2005 Apr;15(2):128-132 [FREE Full text] [CrossRef] [Medline]
- Fekkes M, Pijpers FI, Fredriks AM, Vogels T, Verloove-Vanhorick SP. Do bullied children get ill, or do ill children get bullied? A prospective cohort study on the relationship between bullying and health-related symptoms. Pediatrics 2006 May;117(5):1568-1574 [FREE Full text] [CrossRef] [Medline]
- Mishna F, Scarcello I, Pepler D, Wiener J. Teacher's understanding of bullying. Canadian Journal of Education / Revue canadienne de l'éducation 2005;28:718-738. [CrossRef]
- Ybarra ML. Linkages between depressive symptomatology and Internet harassment among young regular Internet users. Cyberpsychol Behav 2004 Apr;7(2):247-257. [CrossRef] [Medline]
- Fosse GK, Holen A. Childhood maltreatment in adult female psychiatric outpatients with eating disorders. Eat Behav 2006 Nov;7(4):404-409. [CrossRef] [Medline]
- Woods S, Wolke D. Direct and relational bullying among primary school children and academic achievement. Journal of School Psychology 2004;42:135-155. [CrossRef]
- Leff SS, Power TJ, Manz PH, Costigan TE, Nabors LA. School-based aggression prevention programs for young children: Current status and implications for violence prevention. Sch Psych Rev 2001;30:344-362.
- Nansel TR, Overpeck M, Pilla RS, Ruan WJ, Simons-Morton B, Scheidt P. Bullying behaviors among US youth: Prevalence and association with psychosocial adjustment. JAMA 2001 Apr 25;285(16):2094-2100 [FREE Full text] [Medline]
- DeRosier ME, Kupersmidt JB, Patterson CJ. Children's academic and behavioral adjustment as a function of the chronicity and proximity of peer rejection. Child Dev 1994 Dec;65(6):1799-1813. [Medline]
- Olweus D. Bullying at School: What We Know and What We Can Do. Oxford, UK: Wiley-Blackwell; 1993.
- Holmes SR, Bradenburg-Ayres SJ. Bullying behavior in school: A predictor of later gang involvement. J Gang Res 1998;5:1-6.
- Hanish LD, Guerra NG. Aggressive victims, passive victims, and bullies: Developmental continuity or developmental change? Merrill-Palmer Quarterly 2004;50:17-38. [CrossRef]
- Parker JG, Asher SR. Peer relations and later personal adjustment: Are low-accepted children at risk? Psychol Bull 1987 Nov;102(3):357-389. [Medline]
- Hess-Biber SN, Leavy P. Handbook of Emergent Methods. New York, NY: The Guilford Press; 2008.
- Hewson C. Internet-mediated research as an emergent methodits potential role in facilitating mixed methods research. In: Hesse-Biber SJ, editor. Handbook of Emergent Methods. New York, NY: The Guilford Press; 2008:543-570.
- Brehaut JC, Santesso N, O'Connor AM, Lott A, Lindgaard G, Syrowatka A, et al. Presenting evidence to patients online: What do web users think of consumer summaries of cochrane musculoskeletal reviews? J Med Internet Res 2011;13(1):e5 [FREE Full text] [CrossRef] [Medline]
- El Emam K, Moreau K, Jonker E. How strong are passwords used to protect personal health information in clinical trials? J Med Internet Res 2011;13(1):e18 [FREE Full text] [CrossRef] [Medline]
- Reynolds RA, Woods R, Baker JD. Handbook of Research on Electronic Surveys and Measurements. Hershey, PA: IGI Global; 2007.
- Truell AD, Bartlett JE, Alexander MW. Response rate, speed, and completeness: A comparison of Internet-based and mail surveys. Behav Res Methods Instrum Comput 2002 Feb;34(1):46-49. [Medline]
- Pew Internet. 2006. 2006 URL: http://www.pewinternet.org/ [accessed 2012-06-28] [WebCite Cache]
- Pitkow JE, Kehoie CM. Emerging trends in the www user population. Communications of the ACM 1996;39:106-108. [CrossRef]
- Kids Help Phone. 2012 URL: http://org.kidshelpphone.ca/en/about-us/facts-and-stats/ [accessed 2012-06-28] [WebCite Cache]
- Craig WM, Pepler DJ. Identifying and targeting risk for involvement in bullying and victimization. Can J Psychiatry 2003 Oct;48(9):577-582. [Medline]
- Kerlinger F, Lee H. Foundations of Behavioral Research. Fort Worth, TX: Harcourt College Publishers; 2000.
- Olweus D. Bully/victim problems among schoolchildren: Basic facts and effects of a school-based intervention program. In: Rubin KH, Pepler DJ, editors. The Development and Treatment of Childhood Aggression. Hillsdale, N.J: Lawrence Erlbaum Associates, Inc; 1991.
- Cohen J. Statistical Power Analysis for the Behavioral Sciences. Hillsdale, N.J: Lawrence Erlbaum Associates, Inc; 1988.
- Sigfusdottir ID, Gudjonsson GH, Sigurdsson JF. Bullying and delinquency: The mediating role of anger. Personality and Individual Differences 2010;48:391-396. [CrossRef]
- Vega WA, Aguilar-Gaxiola S, Andrade L, Bijl R, Borges G, Caraveo-Anduaga JJ, et al. Prevalence and age of onset for drug use in seven international sites: results from the international consortium of psychiatric epidemiology. Drug Alcohol Depend 2002 Dec 1;68(3):285-297. [Medline]
- Pellegrini AD. Bullying, victimization, and sexual harassment during the transition to middle school. Educational Psychologist 2002;37:151-163. [CrossRef]
- Demaray MK, Malecki CK. Perceptions of the frequency and importance of social support by students classified as victims, bullies, and bully/victims in an urban middle school. Sch Psych Rev 2003;32:471-489.
Edited by G Eysenbach; submitted 20.04.11; peer-reviewed by J Green; comments to author 01.09.11; revised version received 17.07.12; accepted 03.10.12; published 09.11.12
©Tanya Beran, Lauren Stanton, Ross Hetherington, Faye Mishna, Shaheen Shariff. Originally published in the Interactive Journal of Medical Research (http://www.i-jmr.org/), 09.11.2012.
This is an open-access article distributed under the terms of the Creative Commons Attribution License (http://creativecommons.org/licenses/by/2.0/), which permits unrestricted use, distribution, and reproduction in any medium, provided the original work, first published in the Interactive Journal of Medical Research, is properly cited. The complete bibliographic information, a link to the original publication on http://www.i-jmr.org/, as well as this copyright and license information must be included.