What works in anti-bullying programs? Analysis of effective intervention components. Hannah Gaffney, Maria M. Ttofi, and David P. Farrington Institute of Criminology, University of Cambridge Journal of School Psychology Corresponding author: Hannah Gaffney, hg409@cam.ac.uk Institute of Criminology, University of Cambridge, Sidgwick Avenue, Cambridge, CB3 9AT Abstract Previous research has shown that many school-based anti-bullying programs are effective. A prior meta-analysis (Gaffney, Ttofi, & Farrington, 2019) found that intervention programs are effective in reducing school-bullying perpetration by approximately 19–20% and school-bullying victimization by approximately 15–16%. Using data from this prior meta-analysis, the aim of the current study was to examine the relationship between effectiveness estimates and specific elements of anti-bullying programs. Specific intervention components in line with a socio-ecological framework were coded as present or absent. Components were coded on the following levels: school, classroom, teacher, parent, peer, individual, and intervention. Meta-analytical subgroup comparisons analogous to ANOVA were computed to examine the relationship between the presence of specific components and the effectiveness in reducing bullying perpetration (n = 82) and victimization (n = 86). Results indicated that the presence of a number of intervention components (e.g., whole-school approach, anti-bullying policies, classroom rules, information for parents, informal peer involvement, and work with victims) were significantly associated with larger effect sizes for school-bullying perpetration outcomes. The presence of informal peer involvement and information for parents were associated with larger effect sizes for school-bullying victimization outcomes. Meta-regression analyses showed no significant relationship between effectiveness and the number of intervention components included in a program. The present report contributes to the understanding of 'what works' in reducing school-bullying perpetration and victimization. The impact of these findings on future anti-bullying research is discussed. Keywords: anti-bullying; bullying intervention; bullying prevention; meta-analysis; systematic review What Works in Anti-Bullying Programs? Analysis of Effective Intervention Components Introduction School-bullying is defined as repeated aggressive behaviors that occur between a bully (or bullies) and victim with the intention to harm (Centers for Disease Control and Prevention [CDC], 2014; Farrington, 1993; Olweus, 1992). Bullying differs from teasing or arguments between peers because it involves interactions with a distinct power imbalance, either physical or social, between the perpetrator(s) and the victim (CDC, 2014). Since first becoming a focal point for researchers and policy makers, school-bullying has remained prevalent in society, with continued calls for effective intervention and prevention. Bullying in schools is increasingly becoming a public health concern. There is a wealth of research exploring the outcomes associated with school bullying, both perpetration and victimization, in the short-term and throughout the lifespan. For example, studies have found that several mental health and social problems occur comorbidly with bullying victimization, such as suicidal ideation (e.g., Holt et al., 2015; Liu, Huang, & Liu, 2018), and low self-esteem (Hawker & Boulton, 2000). Moreover, previous meta-analyses of cross-sectional studies found that school-bullying perpetration and victimization areis associated with increased weapon carrying and drug use (Valdebenito et al., 2015; Valdebenito et al., , Ttofi, Eisner, & Gaffney, 2017; Valdebenito, Ttofi, & Eisner, 2015). Bullying can also negatively impact individuals’ long term mental and physical health negatively in the long-term (e.g., Copeland et al., 2013; Ttofi et al., 2011a; Wolke et al., 2013). School-bullying perpetration also in school has also been shown to be associated with long-term negative outcomes, such as offending (Ttofi et al., , Farrington, Lösel, & Loeber, 2011b), drug use (Ttofi et al., Farrington, Lösel, Crago, & Theodorakis, 2016) and violent behaviors (Ttofi, Farrington, & Lösel, 2012). Schools are increasingly viewed as important locations settings for implementation of interventions that aimed to reduce a variety of inappropriate behaviors. For example, schools are a common location for programs that aim to reduce cyberbullying (Gaffney & Farrington, 2018) and teen dating violence (e.g., Jennings et al., 2017). More andIncreasingly, more mental health interventions for young people are being implemented in schools, and recently in the United Kingdom, schools were named as one important agent for reducing knife crime and serious violence amongst British youth (e.g., Her Majesty’s M Government, 2019). 1.1 Effective Intervention and Prevention Previous research has shown that there are a variety of differences in (a) the prevalence of bullying worldwide (UNESCO, 20198), (b) bullying-related risk factors (Zych et al., , Farrington, Llorent, & Ttofi, 2017), and (c) efforts to reduce bullying (Gaffney, Farrington, & Ttofi, 2019b) vary widely across different countries. Whilst Although anti-bullying programs collectively are effective (Gaffney, Ttofi, & Farrington, Gaffney et al., 2019b), different intervention programs have been found to be differentially effective in reducing school-bullying perpetration and victimization (Gaffney, FarringtonGaffney et al., & Ttofi, 2019c). There have been numerous attempts to evaluate the effectiveness of intervention programs, but individual evaluation studies rarely provide sufficient evidence for policy and practice recommendations (Tanner-Smith et al., Tipton, & Polanin, 2016). Therefore, using meta-analytical techniques to synthesize the wider research is an important tool. Other attempts to synthesize and review the effectiveness of anti-bullying programs also have also found that programs generally are generally effective (e.g., Cantone , Piras, Vellante, Preti, Daníelsdóttir, D’Aloja et al., 2015; Chalamandaris & Piette, 2015; Evans et al., Fraser, & Cotter, 2014; Jiménez-Barbero et al., 2012, 2016; Jiménez-Barbero, Hernández, Esteban, & García, 2012). However, these previous reviews have often used restrictive inclusion criteria which may impact findings, for example, However, these previous studies have found different levels of effectiveness which may be due to their inclusion criteria. For example, Jiménez-Barbero et al. (2016) only included randomized controlled trials. We know that true randomization is often difficult to achieve in school settings, and therefore, this review may be limited in informing ‘what works’ in anti-bullying programmes. Admittedly, someOther reviews have yielded more pessimistic conclusions about the effectiveness of anti-bullying programs (e.g., Ferguson et al., 2007; Merrell et al., 2008; Yeager et al., 2015). However, these reviews are often of poorer methodological qualitynot as rigorous asthan the current meta-analysis present review, as the systematic point-by-point comparison by Ttofi and colleagues , Eisner, and Bradshaw (2014) shows. For example, Merrell et al. (2008) only searched two databases and their effect sizes for bullying perpetration was based on only eight8 studies. Yeager et al. (2015) only reviewed studies that compared the effects of programs on different age groups, which greatly limited the number of studies included in the meta-analysis, although the findings of this multi-level meta-analysis have important implications for our understanding of the relationship between age and effectiveness of anti-bullying programmes. For example, the mean effect size for bullying perpetration was only based on 16 studies. Specifically, Yeager et al. (2015) conducted several within-study comparisons and found that overall, anti-bullying programmes are generally more effective with participants aged 13 years old and younger. Finally, Ferguson et al. (2007) reported ans effect size of r = .12 for bullying perpetration based on 23 studies , with the authors using Lipsey (1998) and Cohen (1992) to describe this as a small effect sizeand the authors describe this as a small effect. Yet, using a transformation described by Farrington and Loeber (1989), we estimate that this mean effect size relates to a 24% reduction in bullying perpetration, which does not in fact constitute a small change. A more recent meta-analysis (Gaffney, Farrington, & Ttofi, Gaffney et al., 2019b), which included an even larger number of evaluations, still found positive results. Specifically, it was found that school-based anti-bullying programs were effective overall in reducing school-bullying perpetration by approximately 19 – 20% and school-bullying victimization by approximately 15 – 16%. However, there was significant heterogeneity between evaluation studies in their effectiveness in reducing school-bullying perpetration (Q = 323.99, df = 85, p < .001) and victimization (Q = 387.25, df = 87, p < .001). in this extensive meta-analysis (i.e., Gaffney et al., 2019c). Previous studies have investigated factors such as evaluation methodology, location of intervention, and packaged intervention program as potential explanations for this heterogeneity, but results have been insufficient primarily due to lack of statistical significance or very marginal differences in mean effect sizes (Gaffney, Farrington, & Ttofi, 2019Gaffney et al., 2019b; Gaffney, Ttofi, & FarringtonGaffney et al., 2019c). Thus, applying subgroup analysis techniques to meta-analytical data could help increase knowledge as to what factors could explain why this heterogeneity occurs. The present reportcurrent study focuses on which components of intervention programs are effective or which components may be more effective than others. The choice of variables for the present report was informed by previous literature on intervention components in anti-bullying programs (i.e., Farrington & Ttofi, 2009; Ttofi & Farrington, 2011) and the application of the socio-ecological framework to bullying research (e.g., Hong & Espelage, 2012). 1.2 Socio-ecological Framework Based on Bronfenbrenner’s (19797) ecological systems theory, the socio-ecological framework has widely been applied to bullying perpetration and victimization behaviors both offline and online (e.g., Baldry et al., , Farrington, & Sorrentino, 2015; Espelage, 2014; Swearer & Espelage, 2011). Moreover, whilst school -bullying occurs between a bully (or bullies) and a victim, with these behaviors representing complex social phenomenon. Salmivalli et al. (1996) highlighted the added complexity of school -bullying due to the presence of bystanders and the role of peers in supporting or encouraging bullying. Teachers and school staff are also agents in school -bullying, being as they are able to observe and intervene in bullying situations. Therefore, the current report study applies a dynamic socio-ecological theoretical framework to anti-bullying programs. Comment by Craig Albers: Not listed in References To our knowledge, this theoretical approach is used less often used to assess specific intervention components. In applying this framework to the current data analysis, we present intervention components at multiple levels of the socio-ecological model. This theory suggests that bullying occurs within a complex and diverse social environment and that there are many factors to consider when developing intervention and prevention programs. Thus, when developing the coding system for the present meta-analysis we aimed to include intervention components that target bullying behavior from all levels of the school environment and specifics concerning the intervention programs themselves. 1.3 Objectives Attempts to evaluate the effectiveness of specific components in anti-bullying programs using meta-analysis have been relatively few and far between. Ttofi and Farrington (2011) extended their meta-analysis of school-based bullying prevention programs to explore the effectiveness of specific components. This Their review found that not all components of anti-bullying programmes were equally effective at reducing bullying perpetration and victimization. For example, Ttofi and Farrington’s (2011) weighted regression analyses found that several intervention components were significantly associated with greater reductions in school-bullying perpetration (i.e., parent training/meetings,; playground supervision,; disciplinary methods,; teacher training) and victimization (i.e., disciplinary methods,; cooperative group work,; videos). However, Ttofi and Farrington's (2011) the analyses also suggested that the intervention element ‘work with peers’ was associated with an increase in bullying victimization. Yet, this report was criticized and questioned in the wider literature (Smith et al., Salmivalli, & Cowie, 2012). Concerns were raised in relation toregarding the correlational nature of the analysis and the inclusion of outdated evaluations. The need for further research and better experimental evaluations of anti-bullying programs was stressed in response to these concerns (Ttofi & Farrington, 2012). Recently, Huang et al. , Espelage, Polanin, and Hong (2019) explored the effectiveness of bullying intervention programs that incorporated a parental component. This meta-analysis found that while although the included anti-bullying programs were effective in reducing school-bullying perpetration (d = 0.179; p < 0.001) and peer victimization (d = 0.162; p = .004), the authors did not compare the effectiveness of these programs to studies where parents were not included. However, They didHuang et al. nonetheless compared different degrees of parental involvement (e.g., “informational meetings”, training/workshops for parents, or “communication sent home”; Huang et al., 2019; p. 2), but these components did not significantly predict bullying outcomes. 1.4 Current ReviewStudy Therefore, there is still little known about the relationship between specific anti-bullying program elements and overall program effectiveness. Thus, in the present reportcurrent study specific components of evaluated anti-bullying programmes were coded to further explore ‘what works’ in bullying intervention and prevention. ‘What works’ is a commonly asked question in evaluation research and is essential to sufficiently inform evidence-based policy. In the context of the current reportstudy, we explore the specific components of anti-bullying programs and the relationship between these variables components and the effectiveness of included intervention programs in reducing school-bullying perpetration and victimization outcomes. Methods As previously discussedindicated above, this report study presents additional analyseis of a recent large meta-analysis (i.e., Gaffney, Ttofi, & Farrington, 2019) of the effectiveness of school-bullying intervention programs. The current report study prioritizses the description of the methods utilized for the systematic review and coding of specific intervention component variables. The systematic review and meta-analytical methodology are briefly discussed in the following sections too; for further detail please see Gaffney, Ttofi, & Farrington, 2019Gaffney et al. (2019b). 2.1 Searches and Screening To be included in our review of the effectiveness of anti-bullying programs, studies had to: (a1) pPresent the results of an evaluation of a school-based bullying intervention and/or prevention program implemented with school-aged participants (i.e., typically between 4 and 18 years of age),; (b2) uUtilize a definition of school bullying that corresponds to existing operational definitions in the literature (i.e., Centers for Disease Control and PreventionCDC, 2014; Farrington, 1993; Olweus, 1992),; (c3) qQuantitatively measure school bullying perpetration and/or victimization behaviors and experiences using instruments such as observational data or self-/peer-report questionnaires,; and (d4) uUse an experimental design where the experimental group of participants receive, or partake in, an intervention and a comparison group of participants who did not receive, or partake in, an intervention. Searches were conducted between October and December 2016, for studies published between January 2009 to the end of 2016[footnoteRef:1]. The previous meta-analysis conducted by Farrington and Ttofi (2009) included studies published up to May 2009, and so the searches for the current review were restricted to those published from 2009 onwards to avoid overlap. Databases searched include: such as Web of Science, PsychINFO, EMBASE, DARE, Scopus, ERIC, Google Scholar were searched using keywords such as “bully”, “victim”, “bully-victim”, “school”, “intervention”, “prevention”, “program*”, “evaluation”, “anti-bullying”, and “effect*”, “effectiveness”. Studies published in English and other languages were identified, retrieved and screened using inclusion and exclusion criteria.. [1: Two included evaluations (i.e., Kaljee et al., 2017; Limber et al., 2018), had publication dates after our cut off point, but all studies were published online when searches were conducted between October 2016 and end of December 2016.] Details on exclusion criteria and examples of excluded studies can be found in earlier reports of this meta-analysis (e.g., Gaffney, Farrington, & Ttofi, et al., 2019b; Gaffney, Ttofi, & Farrington, , 2019c). In brief, of the 474 studies that were screened, the majority of studies (55%) wwere excluded during screening because they: (a1) did not present results of an evaluation of an anti-bullying program (n = 107),; (b2) reviewed several different anti-bullying programs (n = 108),; orand (c3) did not report empirical quantitative data from an evaluation of a specific anti-bullying program (n = 43). Another large portion (n = 133; 28%) of identified studies were excluded because they either: (a1) did not report school-bullying perpetration and/or victimization behavioral outcomes,; (b2) did not meet the specified methodological criteria,; or (c3) did not use an appropriate independent control group. Finally, studies were excluded if they were repeated publications of the same evaluation data (e.g., KiVa studies) or did not provide enough statistical information needed to calculate an effect size (n = 10). Furthermore, where a publication presented new evaluation data based on additional follow-up points (e.g., Jenson, Brisson, Bender, & Williford, 2013), the previous publications relating to the evaluation data were excluded (e.g., Jenson et al., 2007,; 2010). Therefore, all evaluation data was gathered from the most recent publications. Comment by Craig Albers: Should this be “or” here? 2.2 Included Studies SAdditionally, studies included in a previousTtofi and Farrington’s (2011) meta-analysis of examining the effectiveness of school-based intervention programs (i.e., Ttofi & Farrington, 2011) were also included in the present review. Therefore, following screening, a total of 100 independent evaluations of anti-bullying programs were included in the present systematic review and 103 independent effect sizes were extracted. The majority of these came from evaluations using randomized controlled trials (n = 45 effect sizes) or quasi-experimental designss using before and afterbefore and after measures of bullying behaviour measures (n = 44 effect sizes). Fourteen effect sizes were estimated from evaluations utilizing an age cohort designs. In an age cohort design, participants of age X are assessed for relevant outcomes in the first year and act as the control group. The intervention is then implemented, and different participants of the same age X, from the same school, at follow-up are treated as the experimental group, as they have received the intervention for an academic year (see Kärnä et al., 2013; Kärnä et al., 2011et al., 2011b; Kärnä et al., 2013). The experimental and control students are thus comparable in age and social background. Detailed information about each of the primary evaluations included in our meta-analysis has previously been published (see Gaffney, Ttofi, & Farrington et al., 2019c). Given that that priority is given tothe current study examines intervention components in the current paper, the reader is directed to this reportGaffney, Ttofi, and Farrington for further information regarding the participants (e.g., age range, gender, school-level) and evaluation design (e.g., location of intervention, unit of allocation, measurement instruments). 2.3 Coding of Intervention Components In some instances, primary reports of evaluations did not report sufficient information about the intervention to be included in the present analysis (e.g., Ju et al., 2009). In addition, some information about specific interventions was ascertained from sources other than the primary study included in the meta-analysis. For example, although Bull et al. (2009) reported results of the fairplayer.manual, but with little other information about the intervention activities. Thus, other reports of this intervention (i.e., Wolfer & Scheithauer, 2014) were consulted. Intervention components were coded in accordance with athe socio-ecological framework. Specifically, components were coded on the following levels: (a1) school,; (b2) classroom,; (c3) teacher,; (d4) parent,; (e5) peer,; and (f6) the individual student. Intervention components that did not fit with this categorizsation were grouped under the label of: “intervention-specific”, (i.e., the intervention componentsy wererelated to the specific to the intervention program that was implemented). For the purpose of the present analysiscurrent study, components at all levels were coded dichotomously, as either being absent (0) or present (1) in the specified intervention program. The exceptions to this were the variables relating to the type of program and the approach to anti-bullying. Further details of our codebook used for intervention component analyseis isis provided in the Table 1 (Appendix A). 2.3.1 School -Llevel. At the school -level, we coded the presence or absence of a whole-school approach (or i.e., universal approach) to anti-bullying and supervision in ‘hot spots’ for bullying. A whole-school approach actively involves all actors within the school environment in anti-bullying activities, and the supervision component involved identifying specific areas of the school environment where bullying was more likely to occur and then increasing the presence of teachers in these areas. We also coded for the implementation or use of an anti-bullying policy in intervention programs. A typical anti-bullying policy includes clear definitions and examples of what constitutes bullying behaviors and specifies that these behaviors are not accepted, along with evident strategies for dealing with bullying. 2.3.2 Classroom -Llevel. At the classroom -level we coded for the presence or absence of classroom rules throughout the implementation period of intervention programs. Similar to the anti-bullying policy intervention component, the classroom rules component refers to interventions where a clearly defined set of rules against bullying were implemented and enforced at the classroom-level. In some studies, these rules were created in conjunction with the participating students. We also coded for the inclusion of classroom management techniques in intervention activities. This component describes interventions where a particular focus was placed on teachers identifying and dealing with bullying behavior in their respective classrooms. 2.3.3 Teacher -Llevel. At the teacher -level, we found that more detail could be extracted from primary studies. Generally speaking at this levelBroadly defined, components at the teacher level refer to the participation of teachers in the anti-bullying program. However, the degree of teacher involvement varied and this is reflected in our coding of this component. Thus, the teacher level (TInfo) component describes interventions that provided information about the intervention to teachers in participating schools. Information about the intervention could have been provided in the form of intervention packs or short information sessions and / meetings with teachers. Furthermore, the teacher training (TTrain) component refers to whether teachers were trained to specifically facilitate the anti-bullying program in their respective classrooms or within their respective schools. 2.3.4 Parent/Guardian L-level. Following the socio-ecological framework of bullying prevention and intervention, parents/guardians are also frequently involved in anti-bullying activities. This may involve take-home letters (e.g., Brown et al., 2011), ‘homework’ lessons on anti-bullying materials to be completed under parental supervision and/or with parental participation, or evening meetings to inform parents about bullying-related issues. As there is some obvious discrepancy in the level of active involvement on the part of parents, we divided the ‘information for parents’ variable evaluated by Farrington and Ttofi (2009) into the two independent levels. For example, it is plausible to assume that, if anti-bullying information is provided to parents through letters or leaflets via their children, it is less likely to be translated communicated to parents, as these letters or leaflets may well stay in a child’s schoolbag. TFirstly, the parent information (PInfo) level of the parental-involvement component refers to studies that provided parents with information about bullying-related issues or the intervention being evaluated through take-home letters or leaflets. TSecondly, the parent involvement (PInvolve) component refers to active parent involvement. This dimension of the parental-involvement component refers to included studies that evaluated programsmes where parents were invited to, or attended, meetings held by school staff, or intervention facilitators. During these meetings, bullying was discussed. For example, parents may have been informed about the prevalence of bullying, the associated risk and/or protective factors, or the specific intervention that was being implemented in the respective school. Parents may also have been informed about approaches they may take to prevent, and/or reduce, bullying perpetration or victimization amongst their own children. 2.3.5 Peer L-level. In the same way as the parent-level components, the current report study added additional levels to peer-related intervention activities in order to explore the effect of peer involvement in more detail. The informal peer involvement (Peer1) component , called ‘Peer1’, refers to the general use of in-class, or group-based, discussion during intervention activities. Discussion is often led by teachers or trained intervention facilitators and occurs between peers. ASecondly, a common facet of peer-related components observed in primary studies was the emphasis on engaging bystanders and encouraging of non-involved peers to intervene when they observe bullying situations. Thus, the component ‘Peer2’ relates to the absence or presence of specific intervention strategies to encourage bystanders to prevent bullying , or to intervene in bullying situations. Finally, we coded for formal peer involvement in intervention activities. Examples of formal peer involvement could include peer-mentoring schemes, peer-led anti-bullying activities, or the training non-involved students to provide active support to participants experiencing bullying (e.g., Menesini et al., 2012; Palladino et al., 2012; Menesini et al., 2012). 2.3.6 Individual- Llevel. This The individual level withinin the socio-ecological framework refers to factors relating to the individual within the specified population. Intervention components refer to program elements that relate directly to the students experiencing bullying, either through perpetration or victimization. The bullying (‘Bull)’ component relates to intervention components that involve activities conducted with individual students identified as bullies, and the victim (‘Vic)’ component relates to intervention components that involve activities conducted with individual students identified as victims of bullying. Additionally, the co-operative group work (‘Coop)’ element describes the involvement of external professionals in intervention activities. However, this does not include interventions where external partners provided training to teachers, for example. This component only refers to studies in which these external partners worked directly with victims and/or bullies in experimental schools. 2.3.7 Intervention -Sspecific. In addition to intervention components at the school, classroom, parent, teacher, peer, and individual levels, there were a number of components coded that are related specifically to the intervention programs. Based on the previous review and the wider literature, we coded the presence or absence of curriculum materials (‘Curriculum’) and the inclusion of socio-emotional skills (‘SESkills’) or mental health issues (‘CBT/MH’) in intervention programs. The socio-emotional skills referred to intervention activities centered around specific social, emotional, and psychological concepts, such as empathy, conflict resolution, problem-solving, self-control, decision-making, and prosocial or coping skills (e.g., Holen et al., 2013; Silva et al., 2016; Trip et al., 2015; Silva et al., 2016). The component ‘CBT/MH’ refers to the absence or presence of intervention activities that incorporated cognitive-behavioral techniques or strategies and/or mental health issues, such as anxiety or depression (e.g., DeRosier & Marcus, 2005; McLaughlin, 2009; Stallard et al., 2013). In addition, we coded the use of disciplinary measures. This level involved either the presence/absence of punitive disciplinary measures (e.g., formal punitive sanctions for bullying behaviors) or the presence/absence of non-punitive disciplinary measures (e.g., restorative justice or ‘No Blame’ methods). Furthermore, we coded the different types of intervention programs evaluated by primary studies. This approach was consistent with previous evaluation research on school-based programs (Gottfredson et al., 2002) that grouped programs into ‘environmentally-focused’ interventions (e.g., discipline management interventions, school reorganization, or establishing behavioral norms or expectations) and ‘individually-focused’ interventions (e.g., cognitive-behavioral programs, self-control or social competency instruction, counselling or therapeutic interventions, individual mentoring or tutoring programs). Analysis PlanAnalysis 3.1 Pre-Aanalysis Adjustments Prior to conducting our meta-analysisanalyses, the majority of studies were corrected for the effect of clustering in evaluations. Clustering is a common problem in educational interventions (Donner & Klar, 2002) and occurs when groups of individuals (e.g., , for example, schools or classes), are used as the unit of allocation to experimental conditions. We adjusted the variance of primary studies (except for those that used individuals as the unit of assignment, e.g., Berry & Hunt, 2009) to account for clustering effects using the mean cluster size and the correlation (see Gaffney , Farrington, & Ttofi, 2019et al., 2019b). 3.2 Meta-Aanalysis The Comprehensive Meta-Analyses software (CMA) was used to estimate weighted mean effect sizes for the present report. The effectiveness of included programs was estimated based on the difference between experimental and control students on bullying behavior outcomes before and after implementation of the intervention. Primary studies either reported the percentage of bullies and non-bullies and/or the percentage of victims and non-victims or mean scores on quantitative bullying measurement instruments. As such, effect sizes were estimated as either odds ratios or Cohen’s d. Weighted mean effect sizes were converted to odds ratios for comparability. Weighted odds ratios greater than 1 indicated a desirable intervention effect (e.g., bullying decreased more in the experimental group than in the control group) and weighted odds ratio effect sizes less than 1 indicated an undesirable intervention effect (e.g., bullying decreased more in the control group than in the experimental group, or bullying increased in the experimental group more than in the control group). Additionally, an odds ratio equalling 1 suggested a null intervention effect, and where the 95% confidence intervals included 1 the effect size was not statistically significant. 3.2.1 Models of Meta-Aanalysis. There are multiple ways of assigning weights to observed effect sizes in meta-analyseis. Assigning weights to primary studies is an important aspect of meta-analysis, as not all primary studies will contribute equally to the mean effect size (Lipsey & Wilson, 2001). Typically, either a fixed effects model or a random effects model are reported. The fixed effects model assigns weight to studies as the inverse of the study variance and therefore studies with more precision (namely larger samples) have more impact on the mean effect size. However, this model fails to account for heterogeneity between primary studies and is based on the assumption that there is one common effect underlying all observed effects (Borenstein et al., 2009). The random effects model of meta-analysis is one computational alternative to the fixed effects model. This method accounts for the probable heterogeneity (estimated as the inverse of the sum of within-study variance and tau-squared) between primary studies in social and behavioral sciences. Yet the way in which heterogeneity is incorporated into the random effects model has been criticizsed. Even though meta-analysts have noted that this results in a “more balanced distribution of effect sizes”, the precision of the random effects model is known to be weak when the number of observed effect sizes is small (Borenstein et al., 2009, p. 84). More recently, meta-analysts have suggested that the multiplicative variance adjustment (MVA) is a more appropriate way to account for between-study variance. Farrington and Welsh (2013) observed that, when there is a large amount of heterogeneity in a meta-analysis, the random effects approach results in all primary studies being assigned relatively similar weights. Thus, smaller and less precise studies will inappropriately contribute approximately equally to the weighted mean effect size. Hence, the external validity of the mean effect size will be reduced. The MVA model assigns weights in the same fashion as the fixed effects model, so larger studies are appropriately represented. In the MVA model weights are estimated as: where Wi is the weight assigned to each study and VYi is the within-study variance. The summary mean effect size under the MVA model is calculated as: where Yi is each observed effect size. These formulae are identical to those used for a fixed effects model but under the MVA model the variance of the weighted summary effect size (VM) is estimated as: wWhere Q is the overall heterogeneity and df is the degrees of freedom. In other words, the variance is adjusted by multiplying the within-study variance by Q/df. Although tThe MVAhis model is being used more frequently by researchers in anti-bullying research (e.g., Gaffney, Farrington, Espelage, et al. et al., 2019a; Gaffney, Farrington, & Ttofi et al., 2019b; Zych et al., 2019), but it is not yet widely accepted in social and behavioral sciences. However, many meta-analyses withinin medicineal sciences have suggested that this adjustment for heterogeneity is the most appropriate (e.g., Ayieko et al., Abuogi, Simchowitz, Bukusi, Smith, & Reingold, 2014; Chaffee & King, 2012; Fahimi et al., Singh, & Frazee, 2015; Shore, Gardner, & Pannett et al., 1993) as the non-intuitive addition of variance in the random effects model results in a less conservative overall mean effect. Thus, the present report study presents the results under based on the MVA model. In post-hoc analysis of the assignment of weights in the MVA model and the random effects model, it was determined that a minority of studies[footnoteRef:2] were contributing the majority of weight under the MVA model. However, as anticipated, under the random effects model the mean relative weight assigned was 1.111 for school bullying perpetration outcomes (Q1 = 0.6; Q3 = 1.6; interquartile range = 1.00) and 1.075 for school bullying victimization outcomes (Q1 = 0.45; Q3 = 1.523; interquartile range = 1.073). Thus, the MVA model was deemed most appropriate , and the current report presents analysis using only this method. [2: Kärnä, Voeten, Little, Poskiparta, Kaljonen, & Salmivalli (2011); Kärnä et al. (2013), Grades 2 – 3; Kärnä, Voeten, Little, Poskiparta, Kaljoneb, & Salmivalli (2011), Grades 4 – 6; Kärnä et al. (2013), Grades 8 – 9; Limber et al. (2018); Roland et al. (2010); Waasdorp et al. (2012). Also excluded were five effect sizes from evaluations of the Olweus Bullying Prevention programme (Olweus: Bergen 1, New National, Olso 1, Olso 2)] When comparing subgroups of studies, it was deemed inappropriate for these studies to contribute overwhelmingly to the mean summary effect sizes for smaller numbers of studies. These studies would unfairly contribute too little weight under a random effects model, and too much weight under the MVA model. Thus, our planned subgroup analysis would only reflect the association between effect size and intervention components from a few studies. Furthermore, these studies evaluated only four intervention programs: KiVa, OBPP, School-wide Positive Behavioral Support, and the Zero programme (see Gaffney, Farrington, & Ttofi, 2019Gaffney et al., 2019b). Therefore, these effect sizes were omitted from planned subgroup comparative analysis (811 effect sizes were omitted for school-bullying perpetration and 7 10 effect sizes were omitted for school bullying victimization). 3.2.2. Subgroups in Meta-Aanalysis.. In traditional empirical research, when one wishes to compare two mean values to evaluate the difference between two participants, or two groups of participants, a t-test is the standard statistical test. In meta-analysis, we want to compare sub-groups of studies rather than sub-groups of individuals, so the analysis is slightly different. We followed guidelines provided by noted meta-analysts for this type of analysis (Borenstein et al., 2009; Lipsey & Wilson, 2001). Our approach involved two steps: (1) computing the mean effect and variance for each subgroup,; and (2) comparing the mean effects between subgroups (Borenstein et al., 2009, p. 152). This approach has been used previously by researchers to conduct similar analyses (e.g., Kaminski et al., 2008; Ttofi & Farrington, 2011). Comparing the mean effect sizes for subgroups involves a method that is analogous to a one-way ANOVA in primary research (Hedges, 1982a; Lipsey & Wilson, 2001; Wilson, 2002). The meta-analyst creates mutually exclusive categories of primary studies and then compares the between-studies (QB) and the within-studies (QW) variance. When comparing two groups of studies, for example group X and group Y, the between-group variance is estimated as: Comment by Craig Albers: Not in References where QW is the sum of the Q values for groups X and Y. The significance of QB is estimated using the chi-square distribution and the degrees of freedom are calculated as the number of groups minus one. The between-studies heterogeneity is the value used to evaluate whether the difference between subgroups is statistically significant (i.e., whether the difference in weighted mean effect sizes for subgroups is, at least partially, explained by the relevant intervention component). Similar to a one-way analysis of variance, this approach partitions the variance and compares the variability between-groups. The following formula is used to estimate the QB : where j represents the number of groups, Wj represents the sum of within-group weights and Yj is the weighted mean effect for each group (Lipsey & Wilson, 2001). The degrees of freedom for the between-studies heterogeneity is estimated as j – 1 and the statistical significance is determined using a chi-square distribution. As QB is estimated using the weights assigned to observed effect sizes, the value will vary between the fixed effects model and the random effects model. However, no adjustment is needed for the MVA model (i.e., the test will be the same as for the fixed effects model). 3.2.3. Meta-Rregression.. Comprehensive Meta-Analysis Vversion 3 (CMA 3) software was used to conduct meta-regression analysis to explore the relationship between program richness and perpetration and victimization outcomes. Additionally, the present report replicates meta-regression analyses conducted by Ttofi and Farrington (2011). Weighted regression analyses were used in the previous meta-analysis to explore which intervention components were independently related to school bullying perpetration and victimization. Ttofi and Farrington (2009) The previous authors reported that earlier meta-regression analyses were severely limited by the small number of effect sizes included. (Farrington & Ttofi, 2009, p. 67). It was hoped that the present reportcurrent study wouldmay be able to address this issue, as significantly more primary studies were included. Meta-regressions using multiple intervention components as predictors were conducted using CMA 3 software. Analyses was conducted independently for school-bullying perpetration and victimization outcomes. Initially, meta-regression models were computed using all intervention components as predictors to evaluate which components significantly predicted reductions in bullying outcomes. Secondly, components that were identified as significant predictors were included in meta-regression models to establish which components predicted desirable outcomes independently of one another. Meta-regression analyses were computed under a fixed effects model, and the standard errors of regression coefficients were adjusted using the MVA model. The Q and df for the mean summary effect sizes for subgroups were used to adjust the standard error to reflect between-study variance. Results Table 12 presents a detailed breakdown of which components were coded as being present in each primary evaluation for the many different intervention programs included in our meta-analysis. Reviewing the results systematically, one can see that there was a good distribution of components across primary studies. Across bullying perpetration and victimization outcomes, the most common components were the use of curriculum materials, information for teachers, informal peer involvement, and teacher training. The least common components were, the use of cognitive-behavioral and mental health techniques, disciplinary measures (both punitive and non-punitive), and formal peer involvement. Tables 23 and 34 show the results of the subgroup analysis of specific intervention components under a MVA model for school-bullying perpetration and victimization outcomes, respectively. Effect sizes are presented as weighted odds ratios for dichotomous categorical variables for each intervention component, i.e., evaluations of programs in which the component was included (present) compared to evaluations of programs in which the component was not included (absent). The 95% confidence intervals are also reported, along with the QB heterogeneity test and relevant p value. This test indicates the statistical significance of the differences observed between two weighted mean odds ratios. We took a conservative approach to understanding statistical significance, and set the threshold at p < .01. Insert Table 2 about here 4.1 School-Bbullying Pperpetration Under the MVA model of meta-analysis, the presence of the following components was significantly correlated with larger mean effect sizes for school-bullying perpetration outcomes, including: (a) whole-school approach;; (b) anti-bullying policy, (c) classroom rules;; (d) information for parents; (e) informal peer involvement; (f) work with victims; (g) co-operative group work; and (h) mental health approaches. Studies where these components were present produced a larger weighted mean effect size in comparison to studies where these components were absent. Moreover, the inclusion of of the following intervention components: classroom management (p = .039) and punitive disciplinary measures (p = .046) as intervention components gave resulted in larger mean subgroup effect sizes, but the differences between groups were only marginally significant. Interestingly, the absence of socio-emotional skills was significantly correlated with larger subgroup summary effect sizes for school-bullying perpetration outcomes. Insert Table 3 around here 4.2 School-Bbullying Vvictimization Under a MVA model of meta-analysis, the presence of only two intervention components (, i.e., informal peer involvement and information for parents), were significantly correlated with larger subgroup summary effect sizes for school-bullying victimization outcomes. Additionally, the absence of socio-emotional skills was significantly correlated with larger subgroup summary effect sizes for school-bullying victimization outcomes. At a less conservative level of statistical significance, there were also differences between groups that included or excluded the ‘encouraging bystanders’ intervention component. Namely, studies that did not include this component were correlated with larger mean effect sizes (p = .044). Insert Table 4 about here 4.3 Meta-Regression Results Multiple models of meta-regression were conducted for school bullying perpetration and victimization outcomes. The continuous variable program richness, which indicated the total number of intervention components included, did not significantly predict either school-bullying perpetration (B = 0.007; SE = 0.003) or school-bullying victimization (B = -0.003; SE = 0.003) outcomes. Moreover, when all intervention components were included in a meta-regression model, no components significantly predicted either school-bullying perpetration and/or victimization outcomes under the MVA model. Thus, the second planned meta-regression analysis, in which only significant predictors would have been included, was not conducted. Discussion Our results suggest that many components of existing anti-bullying programs are effective in reducing both school-bullying perpetration and victimization. Under the MVA model of meta-analysis, the presence, and absence, of numerous specific components was associated with larger summary effect sizes. Overall, the results presented in the current report provide good evidence for a socio-ecological based approach to anti-bullying programs, consistentin line with previous research using this theoretical framework (Swearer et al., 2012; Swearer & Hymel, 2015). It should also be noted that neither the presence nor the absence of any intervention component was significantly associated with undesirable intervention results, namely, an increase in bullying outcomes. 5.1 What Works? Our findings indicate that various components and anti-bullying activities can be implemented to reduce bullying in schools. Moreover, meta-regression analyses suggest that program richness does not significantly predict more desirable outcomes. In other words, interventions that included many, or all, of the intervention components did not result in significantly greater effectiveness. This finding will be useful to schools around the world that wish to implement measures to prevent or reduce bullying quickly and efficiently, but also, for the development of future intervention programs. Many of the intervention components can be costly, both in monetary terms and the time commitment for school staff (Beckman & Svensson, 2015; Persson et al., 2018). Our findings highlight multiple intervention components that can be implemented to have a desirable impact on bullying behavior. Thus, it is hoped that our study findings will have direct implications for future prevention programming as they open the door to tailoring evidence-based anti-bullying policies based on elements indicative of what works, what doesn’t work and what seems to have no significant impact. Tailoring practices to variations in students’ cognitive, temperament, and other needs may beis a promising approach for the future (Marizi, Dane, & Kennedy, 2010). When interpreting these results, the reader should be aware that the analysis is correlational and also could be influenced by unequal numbers of studies in subgroups (e.g., curriculum materials were: present in 66 evaluations and absent in 16 evaluations for school-bullying perpetration outcomes). Furthermore, our findings suggest that there are more components associated with effectiveness for reducing bullying perpetration in comparison to bullying victimization. This may be explained as a function of social desirability and self-report measures used widely in evaluation studies of anti-bullying programs. It may also be explained by the higher emphasis that has been given in previous research in the core notion of ‘bullying’ rather than ‘victimization’ (Finkelhor, Turner, & Hamby, 2012). In terms of the consistency between outcomes, there were some components that were significantly related to larger summary effect sizes for both perpetration and victimization outcomes. For example, the presence of both informal peer involvement (e.g., class/group discussions or role-playing activities) and information for parents (e.g., letters/leaflets about bullying or intervention sent home to parents and guardians) were significantly associated with greater effectiveness in reducing both school-bullying victimization and perpetration. Notably, the absence of socio-emotional skills was statistically correlated with larger reductions in both school-bullying perpetration and victimization. In other words, programs that did not specify that the intervention program incorporated elements relating to social-emotional skills (e.g., empathy, conflict resolution, or resilience), whether through specific intervention activities or dedicated intervention modules, were associated with greater effectiveness in our analyses. 5.2 Implications Generally, the current study’s findings of the present analysis show that components of anti-bullying programs that involve instituting and encouraging informal social control between all members of school communities are associated with greater effectiveness. This is in line with the vast evidence base on how collective efficacy and informal social control are key factors in reducing antisocial behaviours (e.g., Sampson, 1986; Silver & Miller, 2004; Williams & Guerra, 2011). For example, the presence of informal peer involvement was significantly associated with greater overall effectiveness. Studies that included informal peer involvement reduced bullying perpetration (by approximately 12.5%[footnoteRef:3]) and bullying victimization (by approximately 9%), significantly more than studies that did not incorporate informal peer involvement (1% and 4.5% respectively). [3: Percentages were estimated using the mean effect sizes for subgroups and using a method described in detail by previous publications (i.e. Farrington & Ttofi, 2009; Gaffney, Ttofi, & Farrington, 2019). ] Informal peer involvement refers to the inclusion of whole-class or small group discussions and other intervention activities whereby interaction with peers would naturally occur. In this way, individual bullies and/or victims were not directly targeted by the activities, yet bullying experiences, attitudes, and behaviors were discussed within the peer group, thus promoting an appropriate classroom and school ethos. Likewise, this intervention component does not specifically target bystanders, but would indirectly include bystanders in discussions and activities. This is contrary to previous findings by Farrington and Ttofi (2009) that indicated that work with peers was associated with increases in bullying victimization. However, this conflicting result is most likely explained by the more detailed coding system applied in the present report. Peer involvement was coded on three non-mutually exclusive levels in the present analyses, in comparison to one single component in Farrington and Ttofithe previous meta-analysis. Other forms of peer involvement coded in interventions included: teaching students assertiveness and encouraging them to intervene as bystanders when they witness bullying occurring (‘encouraging bystanders’; e.g., Menard & Grotpeter, 2014), or online forums monitored and peer-led by groups of trained students (‘formal peer involvement’; e.g., Menesini et al., 2012; Palladino et al., 2012). Interestingly, the exclusion of encouraging bystanders was significantly associated with larger effect sizes for victimization outcomes. Moreover, the inclusion of formal peer involvement was very nearly significantly associated with a greater overall reduction in bullying perpetration, (OR = 1.324, Qb = 3.544, p = .059) despite the differences in the numbers of studies that incorporated this component. When dDissecting the involvement of peers in this way, our results are consistent with the large body of bullying research which highlights the important, yet complex, role of peers in bullying amongst children and adolescents (e.g., Salmivalli et al., , 1996; 2010). Our results can provide better understanding of the mechanisms of change involved in anti-bullying programs. P Many previous studies have emphasized the importance of understanding mechanisms of change in the development of problem behaviours in general (e.g., van Lier, Vuijk, & Crijnen et al., 2005). For example, it may be that increasing awareness of bullying behaviors and involving all individuals in the classroom environment creates a social space less conducive to bullying. Additionally, having systems in place to hold bullies accountable for their behavior, such as classroom rules, may lead to larger reductions in bullying perpetration. Giving teachers the skills to manage child behavior in classrooms can also contribute to greater overall reductions in reports of bullying perpetration behaviors. This is consistent with previous studies that have emphasized the importance of utilizing mechanisms of change to further the development of bullying prevention research. Specific to ally in relation to KiVa, Saarento and colleagues (2015) found that classroom-level factors, such as the collective perceptions of teacher attitudes towards bullying, can mediate the effects of an intervention program. Most anti-bullying programs will incorporate peers and teachers in some form, especially if they the programs are school-based and implemented during school hours. Therefore, we must strive to better understand how intervention components at all levels of a socio-ecological framework contribute to the overall effectiveness of an anti-bullying program. Our findings demonstrate some of the ways in which anti-bullying programs can utilize a whole-school approach to prevent and reduce bullying. For example, the presence of classroom rules and the whole school approach were significantly associated with larger summary effect sizes, with studies that included these components collectively reducing bullying perpetration by approximately 11%. Comparatively, studies that did not include these intervention components reduced bullying perpetration by approximately 5 – 6%. In addition, information for parents was significantly associated with greater reductions in both school-bullying perpetration and victimization. Disclosure of bullying to parents remains a challenge with many parents being unaware that their children have been bullied or bully others (Harcourt, Jasperse, & Green, 2014; Mishna, Pepler, & Wiener, 2006). Yet, parents may play an important role in an ecological approach to prevent bullying and victimization. Our results indicate that communicating information about bullying and the intervention with parents and guardians via letters/leaflets may be a more appropriate method in which future anti-bullying programs can involve parents. This further supports the proposal that the most effective components of anti-bullying programs are those in which informal social control is established, particularly in relation to bullying perpetration. Furthermore, the establishment of accountability as a component of social control, whereby, others are made aware of bullying behavior (i.e., parents through information leaflets sent home and teachers and/or peers enforcing classroom rules against bullying), is an important aspect of bullying perpetration prevention. Yet the involvement of parents in anti-bullying programs more formallyofficially, for example, by conducting information evenings for parents to attend, is not significantly associated with and differences in the effectiveness of the intervention. It may be that when parents are involved in anti-bullying programs in this way, the ‘right’ parents do not engage. That is to say that possibly the parents of children involved in bullying possibly do not voluntarily participate in the anti-bullying program. Promoting a better dialogue between parents and teachers may be a key factor. Existing research shows that collaborative relations between parents and schools with respect to bullying are currently undermined by discourses of responsibility and blame (Herne, 2016). Parents who attempt to engage with schools about the victimization of their children are often dissatisfied with the school’s response (Brown, Aalsma, & Ott et al., 2013;  Hein, 2014), while whereas teachers find communications with parents about bullying to be a source of frustration and misunderstandings (Marshall, 2012). Components that targeted students were also significantly associated with greater effectiveness in reducing bullying perpetration outcomes, such as working with victims and including cognitive-behavioral and mental health techniques in the intervention. Previous research has suggested that being bullied is independently related to child and adolescent mental health, as well asbut also that experiencing internalizing and externalizing problems can increase the risk of being bullied (Arsenault, Bowes, & Shakoor et al., 2010). 5.3 Limitations and Future Research The main limitation of the present report is similar to that of most primary anti-bullying research; correlation is not causation. Conducting subgroup analysis using meta-analytical techniques is limited by the correlational nature of the comparison and by the nature of the comparison groups. TIn order to better understand any potential causal link, evaluators of anti-bullying programs could vary the implementation of components between experimental groups in future research. Some of the studies evaluations included in the present report did varyvaried aspects of the implementation of different intervention components. For example, Trip et al. (2015) incorporated two experimental intervention groups, with one group receiving the REBE intervention activities first, followed by the ViSC anti-bullying activities. The order was then reversed for the second intervention group. In comparison, Stallard et al. (2013) compared the effectiveness of a classroom-based CBT program to two forms of control group; the first control group completed the schools’ usual PSHE curriculum delivered by teachers, and the second control group also completed the PSHE curriculum, but lessons were delivered by a teacher who was assisted by two trained facilitators. However, very relatively few of the included studies compared experimental groups based on the implementation of specific intervention components. Polanin (2015) evaluated the impact of the Second Step program, that was supplemented with additional cultural-awareness lessons,; however, Polanin but did not compare the effectiveness of the Second Step program with the effectiveness of the Second Step program plus additional components. If future evaluations of anti-bullying interventions were to vary the implementation of specific intervention components, it would become clearer as to what actually works in anti-bullying programs and where there were differences in outcome according to the specific intervention component implemented. Moreover, future analyses could assess possible interaction effects, given that previous research has found that integrated models of school-based prevention are often preferable, as they address some of the challenges associated with school-based prevention without comprising intervention integrity (Domitrovich et al., 2010). The implementation fidelity and sustainability of intervention results need to be explored in greater detail. Studies have found that varying levels of implementation of intervention components may explain the variability in intervention outcomes (Bloom, Hill, & Riccio, 2003) and a narrative review found that evaluations that monitored implementation obtained mean effects that were two times larger than evaluations that did not measure implementation (Smith et al., 2004). The present analysis is estimated using data before intervention and immediately post-intervention and few of the studies included additional follow-up timepoints or quantitative measurement of implementation fidelity within the evaluation. Therefore, the long-term effectiveness of anti-bullying programs is unclear. Another limitation is that we relied on information published about included intervention programs; consequently, , so that there may well be interventions that included a particular component but did not explicitly report this in reports of evaluation studies. However, wWhere possible , however, we did consulted additional publications of included interventions. Thus, the present analyses may not adequately represent every component included in anti-bullying programs and we also do not know how well and consistently the components that we coded for were implemented. Future primary research on the effectiveness of anti-bullying interventions should aim to include and specify all relevant components of an intervention program, although, this may be difficult because space is often limited in peer-reviewed publications. This research would benefit too from deductive qualitative data that asks school staff and teachers to comment on the reality of implementing specific intervention components. It is at least equally , or more, important that these reductions in bullying are sustainable and maintained beyond the evaluation of the intervention program. Therefore, a component may be statistically associated with greater reductions in bullying behaviors, but, if such a component is not feasible for schools to implement after the official evaluation has stopped, then this needs to be addressed. Finally, any meta-analysis is impacted by the computational model chosen to assign weights to primary studies and limited too by existing meta-analytical tests. As previously outlined, both the MVA and the random effects model were deemed inadequate in the present subgroup analysis. The random effects model assigned too little weight to larger evaluations and the MVA model assigned too much weight to larger evaluations. Therefore, the decision to omit over-powered studies from our subgroup analysis means that results are presented under an appropriately weighted computational model (MVA) and better reflect the distribution of intervention components between multiple programs. Previous analysis has demonstrated that these programs are definitely effective anti-bullying initiatives (see Gaffney, Ttofi, & Farrington, 2019Gaffney et al., 2019c), but packaged interventions are often quite expensive to purchase or require high levels of training and staff commitment (e.g., KiVa). Therefore, while although packaged anti-bullying programs are a viable and reliable option to reduce bullying, our analysis provides interested stakeholders with a detailed breakdown of specific intervention activities that are shownappear to be associated with greater effectiveness. Our analyses results show that programs that are more intensive and include a larger number of intervention components were not more effective. This suggests that there are options other than extensive multi-component and packaged interventions for schools that want to tackle bullying. Moreover, our subgroup analysis is limited as intervention components were treated as being mutually exclusive. The strength of the socio-ecological theoretical approach is that it generally allows for exploration of the dynamic interactions between factors on all levels. However, we were limited in our ability to examine interaction effects in the present analyses. Recent meta-analyses have used advanced statistical tests (e.g., ‘three-level’ meta-analysis;, Yeager et al., 2015) to attempt to examine moderator effects. Future research should aim to utilize advancing statistical tests to better our understanding of ‘what works’ in anti-bullying programs, specifically in relation to the potential combinations of intervention components. 5.2 Conclusion 1 WHAT WORKS IN ANTI-BULLYING PROGRAMS? 23 WHAT WORKS IN ANTI-BULLYING PROGRAMS? While Although previous research has found that anti-bullying programs are generally effective in general, the present report shows that specific intervention components are significantly associated with greater reductions in school-bullying perpetration and victimization. Specifically, the inclusion of curriculum materials and informal peer involvement were significantly correlated withresulted in larger effect sizes for both perpetration and victimization outcomes. The absence of socio-emotional skills was associated with greater reductions in both outcomes, although the inclusion of this intervention component did not result in undesirable intervention outcomes. There were more significant associations between intervention components and larger effect sizes for bullying perpetration outcomes in comparison to victimization. The presence of a whole-school approach, classroom rules, information for parents, formal peer involvement, co-operative group work, and CBT approaches were significantly correlated withresulted in larger mean effect sizes in comparison to studies where these components were absent. The presence of non-punitive disciplinary methods was also associated with larger summary effect sizes for bullying victimization outcomes. The present reportcurrent study therefore contributes to the understanding of anti-bullying programs and ‘what works’ in reducing school bullying perpetration and victimization. References References marked with an asterisk indicate studies included in the meta-analysis.* = included in the meta-analysis *Alsaker, F. D. (2004). Bernese program against victimization to kindergarten and elementary schools. In P. K. Smith, D. Pepler, & K. Rigby (Eds.), Bullying in schools: How successful can interventions be? (pp. 289 -– 306). Cambridge: Cambridge University Press. *Alsaker, F. D., & Valkanover, S. (2001). Early diagnosis and prevention of victimization in kindergarten. In J. Juvonen & S. Graham (Eds.), Peer harassment in school, (pp. 175 -– 195). New York: Guilford. *Andreou, E., Didaskalou, E., & Vlachou, A. (2007). Evaluating the effectiveness of a curriculum-based anti-bullying intervention program in Greek primary schools. Educational Psychology, 27, 693 -– 711. Arsenault, L., Bowes, L., & Shakoor, S. (2010). Bullying victimization in youths and mental health problems: ‘Much ado about nothing’? Psychological Medicine, 40, 717 -– 729. https://doi.org/10.1017/S0033291709991383 Ayieko, J., Abuogi, L., Simchowitz, B., Bukusi, E. A., Smith, A. H., & Reingold, A. (2014). Efficacy of isoniazid prophylactic therapy in prevention of tuberculosis in children: A meta-analysis. BMC Infectious Diseases, 14(91), 1 -– 10. https://doi.org/10.1186/1471-2334-14-91 *Baldry, A. C., & Farrington, D. P. (2004). Evaluation of an intervention program for the reduction of bullying and victimization in schools. Aggressive Behavior, 30, 1 -– 15. https://doi.org/10.1002/ab.20000 Baldry, A. C., Farrington, D. P., & Sorrentino, A. (2015). “Am I at risk of cyberbullying”? A narrative review and conceptual framework for research on risk of cyberbullying and cybervictimization: The risk and needs assessment approach. Aggression and Violent Behavior, 23, 36 -– 51. https://doi.org/10.1016/j.avb.2015.05.014 *Battey, G. J. L. (2009). Can bullies become buddies? Evaluation of and theoretical support for an experiential education bully prevention curriculum with seventh grade students. (Doctoral dissertation, Available from ProQuest Dissertations and Theses. (UMI No. 3348633). *Bauer, N. S., Lozano, P., & Rivara, F. P. (2007). The effectiveness of the Olweus bullying prevention program in public middle schools: A controlled trial. Journal of Adolescent Health, 40, 266 -– 274. https://doi.org/10.1016/j.jadohealth.2006.10.005 Beckman, L., & Svensson, M. (2015). The cost-effectiveness of the Olweus Bullying Prevention Program: Results from a modelling study. Journal of Adolescence, 45, 127 -– 137. https://doi.org/10.1016/j.adolscence.2015.07.020 *Beran, T., & Shapiro, B. (2005). Evaluation of an anti-bullying program: Student reports of knowledge and confidence to manage bullying. Canadian Journal of Education, 28(4), 700 -– 717. https://doi.org/10.2307/4126451 *Beran, T., Tutty, L., & Steinrath, G. (2004). An evaluation of a bullying prevention program for elementary schools. Canadian Journal of School Psychology, 19, (1/2), 99 -– 116. https://doi.org/10.1177/082957350401900105 *Berry, K., & Hunt, C. J. (2009). Evaluation of an intervention program for anxious adolescent boys who are bullied at school. Journal of Adolescent Health, 45, 376 -– 382. https://doi.org/10.1016/j.jadohealth.2009.04.023 Bloom, H. S., Hill, C. J., & Riccio, J. A. (2003). Linking program implementation and effectiveness: Lessons from a pooled sample of welfare‐to‐work experiments. Journal of Policy Analysis and Management, 22(4), 551--575. https://doi.org/10.1002/9781444307399.ch26 *Bonnell, C., Fletcher, A., Fitzgerald-Yau, N., Hale, D., Allen, E., Elbourne, D., Jones, R., Bond, L., Wiggins, M., Miners, A., Legood, R., Scott, S., Christie, D., & Viner, R. (2015). Initiating change locally in bullying and aggression through the school environment (INCLUSIVE): A pilot randomised controlled trial. Health Technology Assessment, 19(53), 1 -– 109. https://doi.org/10.3310/hta19530 Borenstein, M., Hedges, L. V., Higgins, J. P. T., & Rothstein, H. R. (2009). Introduction to meta-analysis. Wiley. *Boulton, M. J., & Flemington, I. (1996). The effects of a short video intervention on secondary school pupils’ involvement in definitions of and attitudes towards bullying. School Psychology International, 17, 331 -– 345. https://doi.org/10.1177/0143034396174003 Brendgen, M., & Poulin, F. (2018). Continued bullying victimization from childhood to young adulthood: A longitudinal study of mediating and protective factors. Journal of Abnormal Child Psychology, 46, 27 – 39. https://doi.org/10.1007/s10802-017-0314-5Bronfenbrenner, U. (1979). Ecology of human development: Experiments by nature and design. Harvard University Press. Comment by Craig Albers: Not cited in narrative. *Brown, E. C., Low, S., Smith, B. H., & Haggerty, K. P. (2011). Outcomes from a school-randomized controlled trial of Steps to Respect: A bullying prevention program. School Psychology Review, 40(3), 423 -– 443. https://doi.org/10.1037/e734362011-045 Brown, J. R., Aalsma, M. C., & Ott, M. A. (2013). The experiences of parents who report youth bullying victimization to school officials. Journal of Interpersonal Violence, 28, 494-–518. https://doi.org/10.1177/088620512455513 *Bull, H. D., Schultze, M., & Scheithauer, H. (2009). School-based prevention of bullying and relational aggression: The fairplayer .manual. European Journal of Developmental Science, 3(3), 312 -– 317. https://doi.org/10.3233/dev-2009-3310 Cantone, E., Piras, A. P., Vellante, M., Preti, A., Daníelsdóttir, S., D’Aloja, E., Lesinskiene, S., Angermeyer, M. C., Carta, M. G., & Bhugra, D. (2015). Interventions on bullying and cyberbullying in schools: A systematic review. Clinical Practice and Epidemiology in Mental Health, 11, 58 -– 76. https://doi.org/10.2174/1745017901511010058 Centers for Disease Control and Prevention. (2014). Bullying surveillance among school- aged children: Uniform definitions and recommended data elements. Washington, DC: Centers for Disease Control and Prevention (CDC). Chaffee, B. W., & King, J. C. (2012). Effect of zinc supplementation on pregnancy and infant outcomes: A systematic review. Paediatric and Perinatal Epidemiology, 26(Suppl 1), 118 -– 137. https://doi.org/10.1111/j.1365-3016.2012.01289.x Chalamandaris, A., & Piette, D. (2015). School-based anti-bullying interventions: Systematic review of the methodology to assess their effectiveness. Aggression and Violent Behavior, 24, 131–174. https://doi.org/10.1016/j.avb.2015.04.004https://doi.org/10.1016/j.avb.2015.04.004 Chaux, E., Barrera, M., Molano, A., Velásquez, A. M., Castellanos, M., Chaparro, M. P., & Bustamante, A. (2017). Classrooms in Peace within violent contexts: Field evaluation of Aulas en Paz in Colombia. Prevention Science, 18, 828 – 838. https://doi.org/10.1007/s11121-017-0754-8 Comment by Craig Albers: Not cited in narrative *Chaux, E., Velásquez, A. M., Schultze-Krumbholz, A., & Scheithauer, H. (2016). Effects of the cyberbullying prevention program Media Heroes (Medienhelden) on traditional bullying. Aggressive Behavior, 42(2), 157 -– 165. https://doi.org/10.1002/ab.21637 *Cissner, A. B., & Ayoub, L. H. (2014). Building healthy teen relationships: An evaluation of the Fourth R curriculum with middle school students in the Bronx. Center for Court Innovation, U.S.A. *Ciucci, E., & Smorti, A. (1998). Il fenomeno delle pretonenze nella scuola: Problemi e prospettive di intervention [The phenomenon of bullying in school: Problems and prospects for intervention]. Psichiatria dell’infanzia e dell’adolescenza, 65, 147 -– 157. Cohen, J. (1992). A power primer. Psychological Bulletin, 112, 155 – 159. https://doi.org/10.1037/0033-2909.112.1.155 *Connolly, J., Josephson, W., Schnoll, J., Simkins-Strong, E., Pepler, D., MacPhearson, A., ... Jiang, D.P. (2015). Evaluation of a youth-led program for preventing bullying, sexual harassment, and dating aggression in middle schools. Journal of Early Adolescence, 35(3), 403 -– 434. https://doi.org/10.1177/0272431614535090 Copeland, W. E., Wolke, D., Angold, A., & Costello, E. J. (2013). Psychiatric outcomes of bullying and being bullied by peers in childhood and adolescence. JAMA Psychiatry, 70(4), 419 -– 426. https://doi.org/10.1001/jamapsychiatry.2013.504 *Cross, D., Hall, M., Hamilton, G., Pintabona, Y., & Erceg, E. (2004). Australia: The Friendly Schools project. In P. K. Smith, D. Pepler, & K. Rigby (Eds.),. Bullying in schools: How successful can interventions be? (pp. 187 -– 210). Cambridge, UK: Cambridge University Press. https://doi.org/10.1017/CB09780511584466.011 *Cross, D., Monks, H., Hall, M., Shaw, T., Pintabona, Y., Erceg, E., Hamilton, G., Roberts, C., Waters, S., & Lester, L. (2011). Three-year results of the Friendly School whole-of-school intervention on children’s bullying behaviour. British Educational Research Journal, 37(1), 105 -– 129. https://doi.org/10.1080/01411920903420024 *DeRosier, M. E. (2004). Building relationships and combating bullying: Effectiveness of a school-based social skills group intervention., Journal of Clinical Child & Adolescent Psychology, 33(1), 196 -– 201. https://doi.org/10.1207/S15374424JCCP3301_18 *DeRosier, M. E., & Marcus, S. R. (2005). Building friendships and combating bullying: Effectiveness of S.S. GRIN at one-year follow-up. Journal of Clinical Child & Adolescent Psychology, 34(1), 140 -– 150. https://doi.org/10.1207/s15374424jccp3401_13 *Domino, M. (2011). The impact of Take the LEAD on school bullying among middle school youth. (Doctoral Dissertation, Available from ProQuest Dissertation and Theses database (UMI No. 3434870). *Domino, M. (2013). Measuring the impact of an alternative approach to school bullying. Journal of School Health, 83(6), 430 -– 437. https://doi.org/10.1111/josh.12047 Domitrovich, C. E., Bradshaw, C. P., Greenberg, M. T., Embry, D., Poduska, J. M., & Ialongo, N. S. (2010). Integrated models of school‐based prevention: Logic and theory. Psychology in the Schools, 47(1), 71--88. https://doi.org/10.1002/pits.20452 Donner, A., & Klar, N. (2002). Issues in meta-analysis of cluster randomized trials. Statistics in Medicine, 21, 2971 – 2980. https://doi.org/10.1002/sim.1301 *Elledge, L. C., Cavell, T. A., Ogle, N. T., & Newgent, R. A. (2010). School-based mentoring as selective prevention for bullied children: A preliminary test. Journal of Primary Prevention, 31, 171 -– 187. https://doi.org/10.1007/s10935-010-0215-7 *Ertesvag, S. K., & Vaaland, G. S. (2007). Prevention and reduction of behavioural problems in school: An evaluation of the Respect program. Educational Psychology, 27, 713 -– 736. https://doi.org/10.1080/01443410701309258 Espelage, D. L. (2014). Ecological Theory: Preventing youth bullying, aggression, and victimization. Theory into Practice, 53(4), 257 -– 264. https://doi.org/10.1080/00405841.2014.947216 *Espelage, D. L., Low, S., Polanin, J. R., & Brown, E. C. (2013). The impact of a middle school program to reduce aggression, victimization, and sexual violence. Journal of Adolescent Health, 53(2), 180 -– 186. https://doi.org/10.106/j.jadohealth.2013.02.021 *Espelage, D. L., Low, S., Polanin, J. R., & Brown, E. C. (2015). Clinical trial of Second Step© middle-school program: Impact on aggression & victimization. Journal of Applied Developmental Psychology, 37, 52 -– 63. https://doi.org/10.1016/j.appdev.2014.11.007 Evans, C. B. R., Fraser, M. W., & Cotter, K. L. (2014). The effectiveness of school-bullying prevention programs: A systematic review. Aggression and Violent Behavior, 19(5), 532 -– 544. https://doi.org/10.1016/j.avb.214.07.004 *Evers, K. E., Poskiparta, J. O., van Marter, D. F., Johnson, J. L., & Prochaska, J. M. (2007). Transtheoretical-based bullying prevention effectiveness trials in middle schools and high schools. Educational Research, 49, 397 -– 414. https://doi.org/10.1080/00131880701717271 Fahimi, J., Singh, A., & Frazee, B. W. (2015). The role of adjunctive antibiotics in the treatment of skin and soft tissue abscesses: A systematic review and meta-analysis. Canadian Journal of Emergency Medicine, 17(4), 420 -– 432. https://doi.org/10.1017/cem.2014.52 Farrington, D. P. (1993). Understanding and preventing bullying. In M. Tonry (Ed.), Crime and Justice, (Vol 17; (pp. 381 -– 458). Chicago: University of Chicago Press. Farrington, D. P., & Loeber, R. (1989). Relative improvement over chance (RIOC) and phi as measures of predictive efficiency and strength of association in 2x2 tables. Journal of Quantitative Criminology, 5(3), 201 -– 213. https://doi.org/10.1007/bf01062737 Farrington, D. P., & Ttofi, M. M. (2009). School-based programs to reduce bullying and victimization, Campbell Systematic Reviews, 6, 1 -– 148. https://doi.org/10.4073/csr.2009.6 Farrington, D. P., & Welsh, B. C. (2013). Measuring effect size in meta-analysis, with special reference to area-based crime prevention programs and the effects of close-circuit television on crime. In A. A. Kuhn, C. Schawarzenegger, P. Margot, A. Donatsch, M. Aebi, & D. Jositsch (Eds.), Criminology, criminal policy and criminal law from an international perspective (pp. 75 – 89). Stampflï. *Fekkes, M., Pijpers, F. I. M., Verloove-Vanhorick, P. (2006). Effects of antibullying school program on bullying and health complaints. Archives of Pediatrics and Adolescent Medicine, 160(6), 638 -– 644. https://doi.org/10.1001/archpedi.160.6.638 *Fekkes, M., van de Sande, M. C. E., Gravesteijn, J. C., Pannebakker, F. D., Buijs, G. J., Diekstra, R. F. W., & Kocken, P. L. (2016). Effects of the Dutch Skills for Life program on the health behaviour, bullying, and suicidal ideation of secondary school students., Health Education, 116(1), 2 -– 15. https://doi.org/10.1108/HE-05-2014-0068 Ferguson, C. J., San Miguel, C., Kilburn, J. C., & Sanchez, P. (2007). The effectiveness of school-based anti-bullying programs: A meta-analytic review. Criminal Justice Review, 32, 401-414. https://doi.org/10.1177/0734016807311712 Finkelhor, S. D., Turner, H. A., & Hamby, S. L. (2012). Let’s prevent peer victimization, not just bullying. Child Abuse and Neglect, 36 (4), 271 -– 274. https://doi.org/10.1016/j.chiabu.2011.12.001 *Finn, K. O. ’K. (2009). An evaluation of the Olweus Bullying Prevention Program. Retrieved from ProQuest Dissertations Publishing (3343406). Fletcher, A., Fitzgerald-Yau, N., Wiggins, M., Viner, R. M., & Bonell, C. (2015). Involving young people in changing their school environment to make it safer: Findings from a process evaluation in English secondary schools. Health Education, 115(3-4), 322 – 338. https://doi.org/10.1108/HE-04-2014-0063 Comment by Craig Albers: Not cited in narrative. *Fonagy, P., Twemlow, S. W., Vernberg, E. M., Nelson, J. M., Dill, E. J., Little, T. D., & Sargent, J. A. (2009). A cluster randomized controlled trial of child-focused psychiatric consultation and a school systems-focused intervention to reduce aggression. The Journal of Child Psychology and Psychiatry, 50(5), 607 -– 616. https://doi.org/10.1111/j.1469-7610.2008.02025.x *Fox, C., & Boulton, M. (2003). Evaluating the effectiveness of a social skills training (SST) program for victims of bullying. Educational Research, 45, 231 -– 247. https://doi.org/10.1080/0013188032000137238 *Frey, K., Hirschstein, M. K., Snell, J. L., van Schoiack Edstrom, L., MacKenzie, E. P., & Broderick, C. J. (2005). Reducing playground bullying and supporting beliefs: An experimental trial of the Steps to Respect program. Developmental Psychology, 41, 479 -– 491. https://doi.org/10.1037/0012-1649.41.3.479 Gaffney, H., & Farrington, D. P. (2018). Cyberbullying in the United Kingdom and Ireland. In A. Baldry, C. Blaya, & D. P. Farrington (Eds.),. International perspectives on cyberbullying, (pp. 101 -- 143). Palgrave Studies in Cybercrime and Cybersecurity, Palgrave McMillian. Gaffney, H., Farrington, D. P., Espelage, D. L., & Ttofi, M. M. (2019a). Are cyberbullying intervention and prevention programs effective? A systematic and meta-analytical review. Aggression and Violent Behavior, 45, 134 -- 153. https://doi.org/10.1016/j.avb.2018.07.001 Gaffney, H., Farrington, D. P., & Ttofi, M. M. (2019b). Examining the effectiveness of school-bullying intervention programs globally: A meta-analysis. International Journal of Bullying Prevention, 1, 14 -– 31. https://doi.org/10.1007/s42380-019-0007-4 Gaffney, H., Ttofi, M. M., & Farrington, D. P. (2019c). Evaluating the effectiveness of school-bullying prevention programs: An updated meta-analytical review. Aggression and Violent Behavior, 45, 111 -– 133. https://doi.org/10.1016/j.avb.2018.07.002 *Garaigordobil, M., & Martínez-Valderrey, V. (2015). Effects of Cyberprogram 2.0 on “face-to-face” bullying, cyberbullying and empathy. Psciothema, 27(1), 45 -– 51. https://doi.org/10.7334/psciotherma2014.78 Garandeau, C. F., Poskiparta, E., & Salmivalli, C. (2014). Tackling actue cases of school bullying in the KiVa anti-bullying program: A comparison of two approaches. Journal of Abnormal Child Psychology, 42(6), 981 – 991. https://doi.org/10.1007/s10802-014-9861-1 Comment by Craig Albers: Not cited in narrative Gardner, F., Hutchings, J., Bywater, T., & Whitaker, C. (2010). Who benefits and how does it work? Moderators and mediators of outcome in effectiveness trial of a parenting intervention. Journal of Clinical Child and Adolescent Psychology, 39(4), 568 – 580. https://doi.org/10.1080/15374416.2010.486315 Comment by Craig Albers: Not cited in narrative *Gini, G., Belli, B., & Casagrande, M. (2003). Le prepotenze a scuola: Una esperienza di ricerca-intervento antibullisimo [Bullying at school: An experience of research-intervention against bullying]. Eta Evolutiva, 76, 33 -– 45. *Gollwitzer, M., Eisenbach, K., Atria, M., Strohmeier D., & Banse, R. (2006). Evaluation of aggression-reducing effects of the ‘Viennese Social Competence Training’. Swiss Journal of Psychology, 65, 125 -– 135. https://doi.org/10.1024/1421-0185.65.2.125 Gottfredson, D. C., Wilson, D. B., & Najaka, S. S. (2002). School-based crime prevention. In L. W. Sherman, D. P. Farrington, B. C. Welsh, & D. L. MacKenzie (Eds.), Evidence-based Crime Prevention, (pp. 56 – 164). Routledge. Harcourt, S., Jasperse, M., & Green, V. A. (2014). “We were sad and we were angry”: A systematic review of parents’ perspectives on bullying. Child & Youth Care Forum, 43(3), 373--391. https://doi.org/10.1007/s10566-014-9243-4 Hawker, D. S. J., & Boulton, M. J. (2000). Twenty years’ research on peer victimization and psychosocial maladjustment: A meta-analytic review of cross-sectional studies. Journal of Child Psychology and Psychiatry and Allied Disciplines, 41(4), 441-455. https://doi.org/10.1111/1469-7610.00629 Holt, M. K., Vivolo-Kantor, A.M., Polanin, J.R., Holland, K.M., Degue, S., Matjasko, J.L., Wolfe, M., & Reid, G. (2015). Bullying and suicidal ideation and behaviors: A meta-analysis. Pediatrics, 135(2), e496–e509. https://doi.org/10.1542/peds.2014-1864 Hein, N. (2014). Parental positions in school bullying: The production of powerlessness in home-school cooperation. In R. M. Schott, & D. M. Søndergaard (Eds.), School bullying: New theories in context (pp. 301-–330). Cambridge: Cambridge University Press. Her Majesty’s Government (2019). Prime Minister’s Summit on Serious Youth Violence, London: Author. Herne, K. E. (2016). ‘It’s the parents’: re-presenting parents in school bullying research. Critical Studies in Education, 57(2), 254--270. https://doi.org/10.1080/17508487.2014.988635 *Herrick, C. (2012). An investigation into the effectiveness of an anti-bullying campaign. Doctoral Dissertation, University of Nottingham, United Kingdom.  *Holen, S., Waaktaar, T., Lervåg, A., & Ystgaard, M. (2013). Implementing a universal stress management program for young school children: Are there classroom climate or academic effects? Scandinavian Journal of Educational Research, 57(4), 420 -– 444. https://doi.org/10.1080/00313831.2012.656320 Hong, J. S., & Espelage, D. L. (2012). A review of research on bullying and peer victimization in school: An ecological model. Aggression and Violent Behavior, 17, 311–322. https://doi.org/10.1016/j.avb.2012.03.003 Huang, Y., Espelage, D. L., Polanin, J. R., & Hong, J. S. (2019). A meta-analytic review of school-based anti-bullying programs with a parental component. International Journal of Bullying Prevention, 1(1), 32 -– 44. https://doi.org/10.1007/s42380-018-0002-1 *Hunt, C. (2007). The effect of an education program on attitudes and beliefs about bullying and bullying behaviour in junior secondary school students. Child and Adolescent Mental Health, 12(1), 21 -– 26. https://doi.org/10.1111/j.1475-3588.2006.00417.x Jennings, W. G., Okeem, C., Piquero, A., Sellers, C. S., Theobald, D., & Farrington, D. P. (2017). Dating and intimate partner violence among young persons ages 15-30: Evidence from a systematic review. Aggression and Violent Behavior, 33, 107-125. https://doi.org/10.1016/j.avb.2017.01.007 *Jenson, J. M., Brisson, D., Bender, K. A., & Williford, A. P. (2013). Effects of the Youth Matters prevention program on patterns of bullying and victimization in elementary and middle school. Social Work Research, 37(4), 361 -– 372. https://doi.org/10.1093/swr/svt030 *Jenson, J. M., & Dieterich, W. A. (2007). Effects of a skill-based prevention program on bullying and bully victimization among elementary school children. Prevention Science, 8, 285 -– 296. https://doi.org/10.1007/s11121-007-0076-3 *Jenson, J. M., Dieterich, W. A., Brisson, D., Bender, K. A., & Powell, A. (2010). Preventing childhood bullying: Findings and lessons from the Denver Public Schools trial. Research on Social Work Practice, 20(5), 509 -– 517. https://doi.org/10.1177/1049731509359186 Jiménez-Barbero, J. A., Ruiz-Hernández, J. A., Llor-Esteban, B., & Péres-García, M. (2012). Effectiveness of antibullying school programmes: A systematic review by evidence levels. Children and Youth Services Review, 34(9), 1646 -– 1658. https://doi.org/10.1016/j.childyouth.2012.04.025 Jiménez-Barbero, J. A., Ruiz-Hernández, J. A., Llor-Zaragoze, L., Péres-García, M., & Llor-Esteban, B. (2016). Effectiveness of anti-bullying programmes: A meta-analysis. Children and Youth Services Review, 61, 165 -– 175. https://doi.org/10.1016/j.childyouth.2015.12.015 *Joronen, K., Konu, A., Rankin, H. S., & Astedt-Kurki, P. (2011). An evaluation of a drama program to enhance social relationships and anti-bullying at elementary school: A controlled study. Health Promotion International, 27(1), 5 -– 14. https://doi.org/10.1093/heapro/dar012 *Ju, Y., Shuqiong, W., & Wenxin, Z. (2009). Intervention research on school bullying in primary schools. Frontiers of Education in China, 4, 111 -– 122. https://doi.org/10.1007/s11516-009-0007-0 *Kaljee, L., Zhang, L., Langhaug, L., Munjile, K., Tembo, S., Menon, A., … & Malungo, J. (2017). A randomized-controlled trial for the teachers’ diploma programme on psychosocial care, support and protection in Zambian government primary schools. Psychology, Health, and Medicine, 22(4), 381 -– 392. https://doi.org/10.1080/13548503.2016.1153682 Kaminski, J. W., Valle, L. A., Filene, J. H., & Boyle, C. L. (2008). A meta-analytic review of components associated with parent training effectiveness. Journal of Abnormal Child Psychology, 36, 567 -– 589. https://doi.org/10.1007/s10802-007-9201-9 *Kärnä, A., Voeten, M., Little, T.D., Alanen, E., Poskiparta, E., & Salmivalli, C. (2013). Effectiveness of the KiVa Antibullying Program: Grades 1 - 3 and 7 - 9. Journal of Educational Psychology, 105(2), 535 -- 551. https://doi.org/10.1037/a0030417 *Kärnä, A., Voeten, M., Little, T. D., Poskiparta, E., Alanen, E., & Salmivalli, C. (2011a). Going to scale: A nonrandomized nationwide trial of the KiVa antibullying Program for Grades 1 - 9. Journal of Consulting and Clinical Psychology, 79(6), 796 -- 805. https://doi.org/10.1037/a0025740 *Kärnä, A., Voeten, M., Little, T. D., Poskiparta, E., Kaljonen, A., & Salmivalli, C. (2011b). A large-scale evaluation of the KiVa antibullying program: Grades 4 - 6. Child Development, 82(1), 311 -- 330. https://doi.org/10.1111/j.1467-8624.2010.01.557.x Kaufman, T.M.L., Kretschmer, T., Huitsing, G., & Veenstra, R. (2018). Why does a universal anti-bullying program not help all children? Explaining persistent victimization during an intervention. Prevention Science, 19, 822 – 832. https://doi.org/10.1007/s11121-018-0906-5 Comment by Craig Albers: Not cited in narrative *Kimber, B., Sandell, R., & Bremberg, S. (2008). Social and emotional training in Swedish classrooms for the promotion of mental health: Results from an effectiveness study in Sweden. Health Promotion International, 23(2), 134 -- 143. https://doi.org/10.1093/heapro/dam046 *Knowler, C., & Frederickson, N. (2013).  Effects of an emotional literacy intervention for students identified with bullying behaviour. Educational Psychology, 33(7), 862 -– 883. https://doi.org/10.1080/01443410.2013.785052 *Krueger, L. M. (2010). The implementation of an anti-bullying program to reduce bullying behaviours on elementary school buses. Doctoral dissertation, D’Youville College, Buffalo, New York. (UMI 3441874)  Leadbeater, B., & Sukhawathanakul, P. (2011). Multicomponent programs for reducing peer victimization in early elementary school: A longitudinal evaluation of the WITS primary program. Journal of Community Psychology, 39(5), 606 – 620. https://doi.org/10.1002/jcop.20447 Comment by Craig Albers: Not cited in narrative Leff, S. S., Waasdorp, T. E., Paskewich, B., Gullan, R. L., Jawad, A. F., MacEvoy, J. P., …, Power, T. J. (2010). The Preventing Relational Aggression in School Everyday program: A preliminary evaluation of acceptability and impact. School Psychology Review, 39(4), 569 – 587. https://doi.org/10.1080/02796015.2010.12087742 Comment by Craig Albers: Not cited in narrative *Li, K., Washburn, I., DuBois, D. L., Vuchinich, S., Ji, P., Brechling, V., Day, J., ... Flay, B. R. (2011). Effects of the Positive Action programme on problem behaviours in elementary school students: A matched-pair randomised control trial in Chicago. Psychology and Health, 26(2), 187 -- 204. https://doi.org/10.1080/08870446.2011.531574 Lier, P. A. C., van, Vuijk, P., & Crijnen, A. A. M. (2005). Understanding mechanisms of change in the development of antisocial behavior: The impact of a universal intervention. Journal of Abnormal Child Psychology, 33, 521 -– 535. https://doi.org/10.1007/s10802-005-6735-7 *Limber, S. P., Olweus, D., Wang, W., Masiello, M. & Breivik, K. (2018). Evaluation of the Olweus Bullying Prevention Program: A large scale study of U.S. students in grades 3 -– 11. Journal of School Psychology, 69, 56 – 72. https://doi.org/10.1016/j.jsp.2018.04.004 Lipsey, M. W. (1998). Design sensitivity: Statistical power for applied experimental research. In L. Bickman & D. J. Rog (Eds.), Handbook of applied social research methods, (pp. 39 – 68). Sage. Lipsey, M. W., & Wilson, D. B. (2001). Practical meta-analysis. Thousand Oaks, CA: Sage. Littell, J. H., Corcoran, J., & Pillai, V. (2008). Systematic reviews and meta-analysis. Oxford, UK: Oxford University Press. Comment by Craig Albers: Not cited in narrative Liu, X., Huang, Y., & Liu, Y. Y. (2018). Prevalence, distribution, and associated factors of suicide attempts in young adolescents: School-based data from 40 low-income and middle-income countries. PLOS One, 13(2)., https://doi.org/10.1371/journal.pone.0207823 *Losey, R. A. (2009). An evaluation of the Olweus Bullying Prevention Program’s effectiveness in a high school setting. Doctoral dissertation, University of Cincinnati, Ohio, U.S.A. *Low, S., & Van Ryzin, M. (2014). The moderating effects of school climate on bullying prevention efforts. School Psychology Quarterly, 29(3), 306 –- 319. https://doi.org/10.1037/spq0000073 Marini, Z. A., Dane, A. V., & Kennedy, R. E. (2010). Multiple pathways to bullying: Tailoring educational practices to variations in students’ temperament and brain function. In M. Ferrari & L. M., Vuletic L. (Eeds.), The developmental relations among mind, brain and education (pp. 257--291). Dordrecht, Netherlands: Springer, Dordrecht. Marshall, M. L. (2012). Teachers’ perceived barriers to effective bullying intervention. Unpublished doctoral thesis. Atlanta, GA: Georgia State University *Martin, F. D. F., Martinez, M. del C. P., & Tirado, J. L. A. (2005). Design, implementation and evaluation of a bullying prevention pilot program. [Spanish: Diseno, aplicacion y evaluacion de un Programa Piloto para la Prevencion del Maltrato entre companeros]. Revista Mexicana de Psicologia, 22, 375 -– 384. *McLaughlin, L. P. (2009) The effect of cognitive behavioral therapy and cognitive behavioral therapy plus media on the reduction of bullying and victimization and the increase of empathy and bystander response in a bully prevention program for urban sixth-grade students., Doctoral Dissertation, University of Toledo.  *Menard, S., & Grotpeter, J. K. (2014). Evaluation of Bully-Proofing Your School as an elementary school antibullying intervention. Journal of School Violence, 13(2), 188- - 209. https://doi.org/10.1080/15388220.2013.840641 *Menard, S., Grotpeter, J., Gianola, D., & O’Neal, M. (2008). Evaluation of Bully Proofing your school: Final report. Downloaded from the NCJRS http://www.ncjrs.gov/pdffiles1/nij/grants/221078.pdf *Menesini, E., Codescasa, E., Benelli, B., & Cowie, H. (2003). Enhancing children’s responsibility to take action against bullying: Evaluation of a befriending intervention in Italian middle schools. Aggressive Behavior, 29, 1 -– 14. https://doi.org/10.1002/ab.80012 *Menesini, E., Nocentini, A., & Palladino, B. E. (2012). Empowering students against bullying and cyberbullying: Evaluation of an Italian peer-led model. International Journal of Conflict and Violence, 6(2), 314 -- 320. https://doi.org/10.1002/ab.80012 Merrell, K. W., Gueldner, B. A., Ross, S. W., & Isava, D. M. (2008). How effective are school bullying intervention programs? A meta-analysis of intervention research. School Psychology Quarterly, 23(1), 26 -– 42. https://doi.org/10.1037/1045-3830.23.1.26 *Meyer, N., & Lesch, E. (2000). An analysis of the limitations of a behavioural programme for bullying boys from a sub-economic environment. Southern African Journal of Child and Adolescent Mental Health, 12(1), 59 -– 69. https://doi.org/10.1080/16826108.2000.9632368 Mishna, F., Pepler, D., & Wiener, J. (2006). Factors associated with perceptions and responses to bullying situations by children, parents, teachers, and principals. Victims and Offenders: An International Journal of Evidence-based Research, Policy, and Practice, 1, 255-–288. https://doi.org/10.1080/15564880600626163 *Nocentini, A., & Menesini, E. (2015). KiVa antibullying program in Italy: Evidence of effectiveness in a randomized control trial. Prevention Science, 17(8), 1012 -– 1023. https://doi.org/10.1007/s11121-016-0690-z *O’Moore, A. M., & Milton, S. J. (2004). Ireland: The Donegal primary school antibullying project. In P.K. Smith, D. Pepler, & K. Rigby (Eds.), Bullying in schools: How successful can interventions be? (pp. 275 -– 288). Cambridge: Cambridge University Press. Olweus, D. (1986). The Olweus Bully/Victim Questionnaire. Mimeo. Bergen, Norway: University of Bergen. Comment by Craig Albers: I don’t believe that this currently is cited within the narrative *Olweus, D. (1992). Bullying among school children: Intervention and prevention. In R. D. Peters, R. J. McMahon, & V. L. Quinsey (Eds.), Aggression and violence throughout the lifespan (pp. 100 –- 125). London: Sage. *Olweus, D. (1994a). Bullying at school: Basic facts and effects of a school-based intervention program. Journal of Child Psychology and Psychiatry, 35, 1171 -– 1190. https://doi.org/10.1111/j.1469-7610.1994.tb01229.x  *Olweus, D. (1994b). Bullying at school: Basic facts and an effective intervention programme. Promotion and Education, 1, 27 -– 31. https://doi.org/10.1177/102538239400100414  *Olweus, D. (1994c). Bullying at school: Long-term outcome for the victims and an effective school-based intervention program. In L. R. Huesmann (Ed.), Aggressive Behavior: Current Perspectives, (pp. 97 -– 130). New York: Plenum. *Olweus, D. (1995). Peer abuse or bullying at school: Basic facts and a school-based intervention programme. Prospects, 25(1), 133 -– 139. https://doi.org/10.1007/bf02334290  *Olweus, D. (1996a). Bullying or peer abuse in school: Intervention and prevention. In G. Davies, S. Lloyd-Bostock, M. McMurran, & C. Wilson (Eds.), Psychology, law, and criminal justice: International developments in research and practice, (pp. 248 -– 267). Berlin: Walter de Gruyter. *Olweus, D. (1996b). Bullying at school: Knowledge base and effective intervention. Annals of the New York Academy of Sciences, 784, 265 -– 276. https://doi.org/10.1111/j.1749-6632.1996.tb32527.x  *Olweus, D. (1996c). Bully/victim problems at school: Facts and effective intervention. Reclaiming Children and Youth: Journal of Emotional and Behavioral Problems, 5(1), 15 -– 22. https://doi.org/10.1080/03033910.1997.10558138  *Olweus, D. (1997a). Bully/victim problems in school: Knowledge base and an effective intervention project. Irish Journal of Psychology, 18, 170 -– 190. https://doi.org/10.1007/bf02195509  *Olweus, D. (1997b). Bully/victim problems in schools: Facts and intervention. European Journal of Psychology of Education, 12, 495 -– 510. https://doi.org/10.1007/bf03172807  *Olweus, D. (1997c). Tackling peer victimization with a school-based intervention program. In D. P. Fry & K. Bjorkqvist (Eds.), Cultural variation in conflict resolution: Alternatives to violence, (pp. 215 -– 232). Mahwah, NJ: Erlbaum *Olweus, D. (2004a). The Olweus Bullying Prevention Programme: Design and implementation issues and a new national initiative in Norway. In P. K. Smith, D. Pepler, & K. Rigby (Eds.), Bullying in schools: How successful can interventions be? (pp. 13 -– 36). Cambridge: Cambridge University Press. *Olweus, D. (2004b). Bullying at school: Prevalence estimation, a useful evaluation design, and a new national initiative in Norway. Association for Child Psychology and Psychiatry Occasional Papers, 23, 5 -– 17. *Olweus, D. (2005a). A useful evaluation design, and the effects of the Olweus bullying prevention program. Psychology, Crime and Law, 11, 389- – 402. https://doi.org/10.1080/10683160500255471 *Ortega-Ruiz, R., Del Rey, R., & Casas, J. A. (2012). Knowing, building and living together on Internet and social networks: The ConRed cyberbullying prevention program. International Journal of Conflict and Violence, 6(2), 303 -– 313. https://doi.org/10.4119/UNIBI/ijcv.250 *Ostrov, J. M., Godleski, S. A., Kamper-DeMarco, K. E., Blakely-McClure, S. J., & Celenza, L. (2015). Replication and extension of the Early Childhood Friendship Project: Effects on physical and relational bullying. School Psychology Review, 44(4), 445 -– 463. https://doi.org/10.17105/spr-15.0048.1 *Pagliocca, P. M., Limber, S. P., & Hashima, P. (2007). Evaluation report for the Chula Vista Olweus Bullying Prevention Program. Final report prepared for the Chula Vista Police Department. *Palladino, B. E., Nocentini, A., & Menesini, E. (2012). Online and offline peer led models against bullying and cyberbullying. Psicothema, 24(4), 634 -- 639. Retrieved from https://www.redalyc.org/pdf/727/72723959020.pdf *Palladino, B. E., Nocentini, A., & Menesini, E. (2016). Evidence-based intervention against bullying and cyberbullying: Evaluation of the NoTrap! program in two independent trials. Aggressive Behavior, 42(2), 194 -– 206. https://doi.org/10.1002/ab.21636 *Pepler, D. J., Craig, W. M., O’Connell, P., Atlas, R., & Charach, A. (2004). Making a difference in bullying: Evaluation of a systemic school-based program in Canada. In P. K. Smith, D. Pepler, & K. Rigby (Eds.), Bullying in schools: How successful can interventions be? (pp. 125 -– 140). Cambridge: Cambridge University Press. Persson, M., Wennberg, L., Beckman, L., Salmivalli, C., & Svensson, M. (2018). The cost-effectiveness of the Kiva anti-bullying program: Results from a decision-analytic model. Prevention Science, 19, 728 -– 737. https://doi.org/10.1007/s11121-018-0893-6 *Polanin, M. K. (2015). Effects of cultural awareness training in conjunction with an established bullying prevention program. Doctoral Dissertation, Loyola University Chicago.  *Pryce, S., & Frederickson, N. (2013). Bullying behaviour, intentions and classroom ecology. Learning Environment Research, 16, 183 -– 199. https://doi.org/10.1007/s10984-013-9137-7 *Rahey, L., & Craig, W. M. (2002). Evaluation of an ecological program to reduce bullying in schools. Canadian Journal of Counselling, 36, 281 -– 295. Retrieved from https://journalhosting.ucalgary.ca/index.php/rcc/article/view/58699 *Rawana, J. S., Norwood, S. J., & Whitley, J. (2011). A mixed-method evaluation of a strength-based bullying prevention program. Canadian Journal of School Psychology, 26(4), 283 -– 300. https://doi.org/10.1177/0829573511423741 *Rican, P., Ondrova, K., & Svatos, J. (1996). The effect of a short, intensive intervention upon bullying in four classes in a Czech town. Annals of the New York Academy of Sciences, 794, 399 -– 400. https://doi.org/10.1111/j.1749-6632.1996.tb32554.x Roland, E., Bru, E., Midthassel, U. V., & Vaaland, G. S. (2010). The Zero programme against bullying: Effects of the programme in the context of the Norwegian manifesto against bullying. Social Psychology of Education, 13, 41 – 55. https://doi.org/10.1007/s11218-009-9096-0 *Rosenbluth, B., Whitaker, D. J., Sanchez, E., & Valle, L. A. (2004). The Expect Respect Project: Preventing bullying and sexual harassment in US elementary schools. In P. K. Smith, D. Pepler, & K. Rigby, (Eds.), Bullying in schools: How successful can interventions be? (pp. 211 -– 233). Cambridge: Cambridge University Press. Saarento, S., Boulton, A. J., & Salmivalli, C. (2015). Reducing bullying and victimization: Student- and classroom-level mechanisms of change. Journal of Abnormal Child Psychology, 43(1), 61 -– 76. https://doi.org/10.1007/s10802-013-9841-x Salmivalli, C. (2010). Bullying and the peer group: A review. Aggression and Violent Behavior, 15, 112 – 120. https://doi.org/10.1016/j.avb.2009.08.007 *Salmivalli, C., Kaukiainen, A., & Voeten, M. (2005). Anti-bullying intervention: Implementation and outcome. British Journal of Educational Psychology, 75, 465 -– 487. https://doi.org/10.1348/000709905x26011 Salmivalli, C., Lagerspertz, K., Björkqvist, K., Österman, K., & Kaukiainen, A. (1996). Bullying as a group process: Participant roles and their relations to social status within the group. Aggressive Behavior, 22, 1 -– 15. https://doi.org/10.4324/9781351153683-13 Sampson, R. J. (1986). Crime in cities: The effects of formal and informal social control. Crime and Justice, 8, 271 -– 311. https://doi.org/10.1086/449125 *Sapouna, M., Wolke, D., Vannini, N., Watson, S., Woods, S., Schneider, W., ... Aylett, R. (2010). Virtual learning intervention to reduce bullying victimization in primary school: A controlled trial. Journal of Child Psychology and Psychiatry, 51(1), 104 -- 112. https://doi.org/10.1111/j.1469-7610.2009.02137.x Sela-Shayovitz, R. (2011). It's not just dealing with the bullying student, rather it's also their parents: Teachers' perceptions of self-efficacy in dealing with parents of bullying students and victims. Journal of Education Research, 5(1), 1 – 13. https://web.a.ebscohost.com/abstract?direct=true&profile=ehost&scope=site&authtype=crawler&jrnl=1935052X&AN=70587787&h=6h%2fmxlujk3l74hpHl%2b6NegjehIDDu4oVr1KrcJSOR2dKX5ulQRcdyKL9yci7O32dBMFOwey437I8WOh5%2f0iSfw%3d%3d&crl=c&resultNs=AdminWebAuth&resultLocal=ErrCrlNotAuth&crlhashurl=login.aspx%3fdirect%3dtrue%26profile%3dehost%26scope%3dsite%26authtype%3dcrawler%26jrnl%3d1935052X%26AN%3d70587787 Comment by Craig Albers: Not cited in narrative Shore, R. E., Gardner, M. J., & Pannett, B. (1993). Ethylene oxide: An assessment of the epidemiological evidence on carcinogenicity. British Journal of Industrial Medicine, 50, 971 -– 997. https://doi.org/10.1136/oem.50.11.971 *Silva, J. L. da, Oliveira, W. A. de, Braga, I. F., Farias, M. S., Silva Lizzi, E. A. da, …, & Iossi Silva, M. A. (2016). The effects of a skill-based intervention for victims of bullying in Brazil. International Journal of Environmental Research and Public Health, 13, 1042. https://doi.org/10.3390/ijerph13111042 Silver, E., & Miller, L. L. (2004). Sources of informal social control in Chicago neighborhoods. Criminology, 42(3), 551 -– 584. https://doi.org/10.1111/j.1745-9125.2004.tb00529.x Smith, P. K., Salmivalli, C., & Cowie, H. (2012). Effectiveness of school-based programs to reduce bullying: A commentary. Journal of Experimental Criminology, 8, 433 –- 441. https://doi.org/10.1007/s11292-012-9142-3 Smith, J. D., Schneider, B. H., Smith, P. K., & Ananiadou, K. (2004). The effectiveness of whole-school antibullying programs: A synthesis of evaluation research. School Psychology Review, 33, 547-–560. https://doi.org/10.1080/02796015.2004.12086267 *Solomontos-Kountouri, O., Gradinger, P., Yanagida, T., & Strohmeier, D. (2016). The implementation and evaluation of the ViSC program in Cyprus: Challenges of cross-national dissemination and evaluation results. European Journal of Developmental Psychology, 13(6), 737 -– 755. https://doi.org/10.1080/17405629.2015.1136618 *Spröber, N., Schlottke, P. F., & Hautzinger, M. (2006). ProACT + E: Ein Programm zur Pravention von “bullying” an Schulen und zur Forderung der positiven Entwicklung von Schulern: Evalation eines schulbasierten, universalen, primar-praventiven Programms fur weiterfuhrende Schulen unter Einbeziehung von Lehrern, Schulern und Eltern. [German: ProACT + E: A programme to prevent bullying in schools and to increase the positive development of students. Evaluation of a school-based, universal, primary preventive programme for secondary schools that includes teachers, students, and parents]. Zeitschrift fur Klinische Psychologie und Psychotherapie: Forschung und Praxis, 35, 140 -– 150. *Stallard, P., Phillips, R., Montgomery, A. A., Spears, M., Anderson, R., Taylor, J., Araya, R., Lewis, G., Ukoumunne, O. C., Millings, A. Georgiou, L., Cook, E., & Sayal, K. (2013). A cluster randomised controlled trial to determine the clinical effectiveness and cost-effectiveness of classroom-based cognitive–behavioural therapy (CBT) in reducing symptoms of depression in high-risk adolescents. Health Technology Assessment, 17(47)., https://doi.org/10.3310/hta17470 *Strohmeier, D., Hoffmann, C., Schiller, E., Stefanek, E., & Spiel, C. (2012). ViSC Social Competence Program. New Directions for Youth Development, 133, 71 –- 84. https://doi.org/10.1002/yd.20008 *Sutherland, A. E. (2010). The roles of school climate and peers in bullying. Unpublished doctoral dissertation, Queen’s University, Canada. Swearer, S. M., & Espelage, D. L. (2011). Expanding the social-ecological framework of bullying among youth. In D. L. Espelage & S. M. Swearer (Eds.), Bullying in North American Schools, (pp. 3 – 10). Routledge. Swearer, S. M., Espelage, D. L., Koenig, B., Berry, B., Collins, A., & Lembeck, P. (2012). A socio-ecological model for bullying prevention and intervention in early adolescence. In S. Jimerson, A. Nickerson, M. J. Mayer, & M. J. Furlong (Eds.), Handbook of school violence and school safety: International research and practice (pp. 338 -– 360). Routledge. Swearer, S. M., & Hymel, S. (2015). Understanding the psychology of bullying: Moving toward a social-ecological diathesis-stress model. American Pscychologist, 70(4), 344 -– 353. https://doi.org/10.1037/a0038929 Tanner-Smith, E. E., Tipton, E., & Polanin, J. R. (2016). Handling complex meta-analytic data structures using robust variance estimates: A tutorial in R. Journal of Developmental and Life Course Criminology, 2, 85 -– 112. https://doi.org/10.1007/s40865-016-0026-5 *Toner, B. K. (2010). The implementation of the bully prevention program: Bully Proofing Your School and its effect on bullying and school climate on sixth grade suburban Students. (Doctoral dissertation, Available from the ProQuest Dissertation and Theses database (UMI No. 3414552). *Topper, L. R. (2011). Bullying victimisation and alcohol-misuse in adolescence: Investigating the functional relationship and new prevention strategies. Unpublished doctoral dissertation, King’s College London, United Kingdom. *Trip, S., Bora, C., Sipos-Gug, S., Tocai, I., Gradinger, P., Yanagida, T., & Strohmeier, D. (2015). Bullying prevention in schools by targeting cognitions, emotions, and behavior: Evaluating the effectiveness of the REBE-ViSC program. Journal of Counselling Psychology, 62(4), 732 -– 740. https://doi.org/10.1037/cou0000084 *Tsiantis, A. C. J., Beratis, I. N., Syngelaki, E. M., Stefanakou, A., Asimopoulos, C., Sideridis, G. D., & Tsiantis, J. (2013). The effects of a clinical prevention program on bullying, victimization, and attitudes toward school of elementary school students. Behavioral Disorders, 38(4), 243 -– 257. https://10.1177/019874291303800406 Ttofi, M. M., Eisner, M., & Bradshaw, C. P. (2014). Bullying prevention: Assessing existing meta-evaluations. In G. Bruinsma & D. Weisburd (Eds), Encyclopedia of Criminology and Criminal Justice, (pp. 231 – 242). . New York: Springer. https://doi.org/10.1007/978-1-4614-5690-2 Ttofi, M. M. & Farrington, D. P. (2011). Effectiveness of school-based programs to reduce bullying: A systematic and meta-analytic review. Journal of Experimental Criminology, 7, 27 -– 56. https://doi.org/10.1007/s11292-010-9109-1 Ttofi, M. M., Farrington, D. P., & Lösel, F. (2012). School bullying as a predictor of violence later in life: A systematic review and meta-analysis of prospective longitudinal studies. Aggression and Violent Behavior, 17, 405–418. https://doi.org/10.1016/j.avb.2012.05.002 Ttofi, M. M., Farrington, D. P., Lösel, F., Crago, R. V., & Theodorakis, N. (2016). School bullying and drug use later in life: A meta-analytic investigation. School Psychology Quarterly, 31(1), 8-27. https://doi.org/10.1037/spq0000120 Ttofi, M. M., Farrington, D. P., Lösel, F., & Loeber, R. (2011a). Do the victims of school bullies tend to become depressed later in life? A systematic review and meta-analysis of longitudinal studies. Journal of Aggression, Conflict, and Peace Research, 3(2), 63 – 73. https://doi.org/10.1108.17596591111132873 Ttofi, M. M., Farrington, D. P., Lösel, F., & Loeber, R. (2011b). The predictive efficiency of school bullying versus later offending: A systematic/meta-analytic review of longitudinal studies. Criminal Behaviour and Mental Health, 21, 80 – 89. https://doi.org/10.1002/cbm.808 Ttofi, M. M., & Farrington, D. P. (2012). Bullying prevention programs: The importance of peer intervention, disciplinary methods and age variations. Journal of Experimental Criminology, 8(4), 443 -– 462. https://doi.org/10.1007/s11292-012-9161-0 United National Educational, Scientific, and Cultural Organisation (2019). School violence and bullying: Global status and trends, drivers and consequences. UNESCO. Valdebenito, S., Ttofi, M., & Eisner, M. (2015). Prevalence rates of drug use among school bullies and victims: A systematic review and meta-analysis of cross-sectional studies. Aggression and Violent Behavior, 23, 137–146. https://doi.org/10.1016/j.avb.2015.05.004 Valdebenito, S., Ttofi, M. M., Eisner, M., & Gaffney, H. (2017). Weapon carrying in and out of school among pure bullies, pure victims and bully-victims: A systematic review and meta-analysis of cross-sectional and longitudinal studies. Aggression and Violent Behavior, 33, 62-77. https://doi.org/10.1016/j.avb.2017.01.004 *Waasdorp, T. E., Bradshaw, C. P., & Leaf, P. J. (2012). The impact of schoolwide positive behavioural interventions and supports on bullying and peer rejection. Archives of Pediatric and Adolescent Medicine, 166(2), 149 -– 156. https://doi.org/10.1001/archpediatrics.2011.755 *Wang, C., & Goldberg, T. S. (2017). Using children’s literature to decrease moral disengagement and victimization among elementary school students. Psychology in the Schools, 54, 918 -– 931. https://doi.org/10.1002/pits.22042 *Whitney, I., Rivers, I., Smith, P. K., & Sharp, S. (1994). The Sheffield Project: Methodology and findings. In P. K. Smith, & S. Sharp, (Eds.), School bullying: Insights and perspectives, (pp. 20 -– 56). London: Routledge Williams, K. R., & Guerra, N. G. (2011). Perceptions of collective efficacy and bullying perpetration in schools. Social Problems, 58(1), 126 -– 143. https://doi.org/10.1525/sp.2011.58.1.126 *Williams, J., Miller, S., Cutbush, S., Gibbs, D., Clinton-Sherrod, M., & Jones, S. (2015). A latent transition model of the effects of a teen dating violence prevention initiative. Journal of Adolescent Health, 56, S27 -– S32. https://doi.org/10.1013.j,adohealth.2014.08.019 *Wölfer, R., & Scheithauer, H. (2014). Social influence and bullying behavior: Intervention-based network dynamics of the fairplayer.manual bullying prevention program. Aggressive Behavior, 40, 309 -– 319. https://doi.org/10.1002/ab.21524 Wolke, D., Copeland, W. E., Angold, A., & Costello, E. J. (2013). Impact of bullying in childhood on adult health, wealth, crime, and social outcomes. Psychological Science, 24(10), 1958-1970. https://doi.org/10.1177/0956797613481608 *Wong, D. S. W., Cheng, C. H. K., Ngan, R. M. H., & Ma, S. K. (2011). Program effectiveness of a restorative whole-school approach for tackling school bullying in Hong Kong. International Journal of Offender Therapy and Comparative Criminology, 55(6), 846 -– 862. https://doi.org/10.1177/0306624X10374638 Wolke, D., Copeland, W. E., Angold, A., & Costello, E. J. (2013). Impact of bullying in childhood on adult health, wealth, crime, and social outcomes. Psychological Science, 24(10), 1958 – 1970. https://doi.org/10.1177/0956797613481608 *Yaakub, N. F., Haron, F., & Leong, G. C. (2010). Examining the efficacy of the Olweus prevention programme in reducing bullying: The Malaysian experience. Procedia – Social and Behavioral Sciences, 5, 595 -– 598. https://doi.org/10.1016/j.sbspro.2010.07.14 Yeager, D. S., Fong, C. J., Lee, H. Y., & Espelage, D. L. (2015). Declines in efficacy of anti-bullying programs among older adolescents: Theory and a three-level meta-analysis. Journal of Applied Developmental Psychology, 37, 36–51. https://doi.org/10.1016/j.appdev.2014.11.005 Zych, I., Farrington, D. P., Llorent, V. J., & Ttofi, M. M. (2017). Protecting children against bullying and its consequences. Springer Briefs in Behavioral Criminology, https://doi.org/10.1007/978-3-319-53028-4 Zych, I., Viejo, C., Vila, E., & Farrington, D. P. (2019). School bullying and dating violence in adolescents: A systematic review and meta-analysis. Trauma, Violence, & Abuse. https://doi.org/10.1177/1524838019854460 Table 1 Intervention components coded for each included primary evaluation Study Intervention Rich_Score School Classroom Teacher Parent Peer Individual Intervention WSA SUP ABP CRule CManage TInfo TTrain PInfo PInvolve Peer1 Peer2 Peer3 BULL VIC CoOp Curriculum SESkills MH Punitive Non-punitive Randomized Controlled Trials Baldry2004 Bulli & Pupe 5 ✔︎ ✔︎ ✔︎ ✔︎ ✔︎ Beran2005 Project Ploughshares Puppets for Peace 1 ✔︎ Berry2009b Confident Kids Bonnell2015b INCLUSIVE Boulton1996a Short Video ABP 1 ✔︎ Brown2011 Steps to Respect ✔︎ ✔︎ ✔︎ ✔︎ ✔︎ ✔︎ ✔︎ ✔︎ ✔︎ ✔︎ ✔︎ Chaux2016 MediaHeroes 8 ✔︎ ✔︎ ✔︎ ✔︎ ✔︎ ✔︎ ✔︎ ✔︎ Cissner2014 Fourth R 3 ✔︎ ✔︎ ✔︎ Connolly2015b Youth-led Cross2011 Friendly Schools 9 ✔︎ ✔︎ ✔︎ ✔︎ ✔︎ ✔︎ ✔︎ ✔︎ ✔︎ DeRosier2005 S.S. GRIN 7 ✔︎ ✔︎ ✔︎ ✔︎ ✔︎ ✔︎ ✔︎ Domino2013 Take the LEAD 7 ✔︎ ✔︎ ✔︎ ✔︎ ✔︎ ✔︎ ✔︎ Espelage2015 Second Step 7 ✔︎ ✔︎ ✔︎ ✔︎ ✔︎ ✔︎ ✔︎ Fekkes2006 Skills for Life 9 ✔︎ ✔︎ ✔︎ ✔︎ ✔︎ ✔︎ ✔︎ ✔︎ ✔︎ Study Intervention Rich_Score WSA SUP ABP CRule CManage TInfo TTrain PInfo PInvolve Peer1 Peer2 Peer3 BULL VIC CoOp Curriculum SESkills MH Punitive Non-punitive Fekkes2016 Skills for Life 8 ✔︎ ✔︎ ✔︎ ✔︎ ✔︎ ✔︎ ✔︎ ✔︎ Fonagy2009 SPC + CAPSLE 9 ✔︎ ✔︎ ✔︎ ✔︎ ✔︎ ✔︎ ✔︎ ✔︎ ✔︎ Frey2005 Steps to Respect 11 ✔︎ ✔︎ ✔︎ ✔︎ ✔︎ ✔︎ ✔︎ ✔︎ ✔︎ ✔︎ ✔︎ Garaigordobil2015 Cyberprogram 2.0 3 ✔︎ ✔︎ ✔︎ Holen2013a Zippy’s Friends 8 ✔︎ ✔︎ ✔︎ ✔︎ ✔︎ ✔︎ ✔︎ ✔︎ Hunt2007 Australian ABP 5 ✔︎ ✔︎ ✔︎ ✔︎ ✔︎ Jenson2013 Youth Matters 6 ✔︎ ✔︎ ✔︎ ✔︎ ✔︎ ✔︎ Ju2009b Chinese ABP 6 ✔ ✔ ✔ ✔ ✔ ✔ Kaljee2017 Teacher Diploma 3 ✔︎ ✔︎ ✔︎ Kärnä2011bc KiVa 15 ✔︎ ✔︎ ✔︎ ✔︎ ✔︎ ✔︎ ✔︎ ✔︎ ✔︎ ✔︎ ✔︎ ✔︎ ✔︎ ✔︎ ✔︎ Krueger2010a School-bus ABP 4 ✔︎ ✔︎ ✔︎ ✔︎ Li2011a Positive Action 5 ✔︎ ✔︎ ✔︎ ✔︎ ✔︎ McLaughlin2009 CBT + Media 6 ✔︎ ✔︎ ✔︎ ✔︎ ✔︎ ✔︎ Meyer2000a “Bullying Boys” 4 ✔︎ ✔︎ ✔︎ ✔︎ Nocentini2016 KiVa 14 ✔︎ ✔︎ ✔︎ ✔︎ ✔︎ ✔︎ ✔︎ ✔︎ ✔︎ ✔︎ ✔︎ ✔︎ ✔︎ ✔︎ Ostrov2015a Early Childhood Friendship 4 ✔︎ ✔︎ ✔︎ ✔︎ Polanin2015 Second Step 7 ✔︎ ✔︎ ✔︎ ✔︎ ✔︎ ✔︎ ✔︎ Rosenbluth2004 Expect Respect 11 ✔︎ ✔︎ ✔︎ ✔︎ ✔︎ ✔︎ ✔︎ ✔︎ ✔︎ ✔︎ Study Intervention Rich_Score WSA SUP ABP CRule CManage TInfo TTrain PInfo PInvolve Peer1 Peer2 Peer3 BULL VIC CoOp Curriculum SESkills MH Punitive Non-punitive Sprober2006 Pro-ACT+E 10 ✔︎ ✔︎ ✔︎ ✔︎ ✔︎ ✔︎ ✔︎ ✔︎ ✔︎ ✔︎ Topper2011b Adventure 4 ✔ ✔ ✔ ✔ Stallard2013a Resourceful Adolescent 5 ✔︎ ✔︎ ✔︎ ✔︎ ✔︎ Trip2015 ViSC + REBE 7 ✔︎ ✔︎ ✔︎ ✔︎ ✔︎ ✔︎ ✔︎ Tsiantis2013 Greek ABP (1) 8 ✔︎ ✔︎ ✔︎ ✔︎ ✔︎ ✔︎ ✔︎ ✔︎ Waasdorp2012a, c SWPBIS 11 ✔︎ ✔︎ ✔︎ ✔︎ ✔︎ ✔︎ ✔︎ ✔︎ ✔︎ ✔︎ Wölfer2014a fairplayer.manual 9 ✔︎ ✔︎ ✔︎ ✔︎ ✔︎ ✔︎ ✔︎ ✔︎ ✔︎ Yanagida2016 ViSC 7 ✔︎ ✔︎ ✔︎ ✔︎ ✔︎ ✔︎ ✔︎ Before-After/Quasi-experimental designs Alsaker2001 Be-Prox 10 ✔︎ ✔︎ ✔︎ ✔︎ ✔︎ ✔︎ ✔︎ ✔︎ ✔︎ ✔︎ Andreou2007 Greek ABP (2) 5 ✔︎ ✔︎ ✔︎ ✔︎ ✔︎ Battey2009b BPCCC 3 ✔ ✔ ✔ Bauer2007b OBPP 14 ✔ ✔ ✔ ✔ ✔ ✔ ✔ ✔ ✔ ✔ ✔ ✔ ✔ ✔ Beran2004b Bully Proofing Your School 11 ✔ ✔ ✔ ✔ ✔ ✔ ✔ ✔ ✔ ✔ ✔ Bergen 2/Olweus OBPP 14 ✔︎ ✔︎ ✔︎ ✔︎ ✔︎ ✔︎ ✔︎ ✔︎ ✔︎ ✔︎ ✔︎ ✔︎ ✔︎ ✔︎ Bull2009 fairplayer.manual 9 ✔︎ ✔︎ ✔︎ ✔︎ ✔︎ ✔︎ ✔︎ ✔︎ Ciucci1998 Progetto Pontassieve 8 ✔︎ ✔︎ ✔︎ ✔︎ ✔︎ ✔︎ ✔︎ ✔︎ Elledge2010b Lunch Buddy 2 ✔ ✔ Study Intervention Rich_Score WSA SUP ABP CRule CManage TInfo TTrain PInfo PInvolve Peer1 Peer2 Peer3 BULL VIC CoOp Curriculum SESkills MH Punitive Non-punitive Evers2007 Transtheroetcial ABP 3 ✔︎ ✔︎ ✔︎ Finn2009 OBPP 14 ✔︎ ✔︎ ✔︎ ✔︎ ✔︎ ✔︎ ✔︎ ✔︎ ✔︎ ✔︎ ✔︎ ✔︎ ✔︎ Fox2003b Social Skills training 4 ✔ ✔ ✔ ✔ Gini2003 Stare bene a scuola 5 ✔︎ ✔︎ ✔︎ ✔︎ ✔︎ Gollwitzer2006 ViSC 6 ✔︎ ✔︎ ✔︎ ✔︎ ✔︎ ✔︎ ✔︎ ✔︎ Joronen2011 Drama program 5 ✔︎ ✔︎ ✔︎ ✔︎ ✔︎ Kimber2008b Socio-emotional training 6 ✔ ✔ ✔ ✔ ✔ ✔ Losey2009 OBPP 13 ✔︎ ✔︎ ✔︎ ✔︎ ✔︎ ✔︎ ✔︎ ✔︎ ✔︎ ✔︎ ✔︎ ✔︎ ✔︎ Martin2005 Granada ABP 10 ✔︎ ✔︎ ✔︎ ✔︎ ✔︎ ✔︎ ✔︎ ✔︎ ✔︎ ✔︎ Melton1998 OBPP 14 ✔︎ ✔︎ ✔︎ ✔︎ ✔︎ ✔︎ ✔︎ ✔︎ ✔︎ ✔︎ ✔︎ ✔︎ ✔︎ ✔︎ Menard2014 Bully-Proofing Your School 11 ✔︎ ✔︎ ✔︎ ✔︎ ✔︎ ✔︎ ✔︎ ✔︎ ✔︎ ✔︎ ✔︎ Menesini2003 Befriending intervention 7 ✔︎ ✔ ✔︎ ✔︎ ✔︎ ✔︎ ✔︎ ✔︎ Menesini2012 NoTrap! 2 ✔︎ ✔︎ Ortega-Ruiz2012 ConRed 7 ✔︎ ✔︎ ✔︎ ✔︎ ✔︎ ✔︎ ✔︎ Palladino2012 NoTrap! 5 ✔︎ ✔︎ ✔︎ ✔︎ ✔︎ Palladino2016 NoTrap! 6 ✔︎ ✔︎ ✔︎ ✔︎ ✔︎ ✔︎ Pepler2004 Toronto ABP 12 ✔︎ ✔︎ ✔︎ ✔︎ ✔︎ ✔︎ ✔︎ ✔︎ ✔︎ ✔︎ ✔︎ ✔︎ Pryce2013 Anti-bullying Pledge Scheme 4 ✔︎ ✔︎ ✔︎ ✔︎ Study Intervention Rich_Score WSA SUP ABP CRule CManage TInfo TTrain PInfo PInvolve Peer1 Peer2 Peer3 BULL VIC CoOp Curriculum SESkills MH Punitive Non-punitive Rahey2002 Ecological ABP 9 ✔︎ ✔︎ ✔︎ ✔︎ ✔︎ ✔︎ ✔︎ ✔︎ ✔︎ Rawana2011 Strengths in Motion 6 ✔︎ ✔︎ ✔︎ ✔︎ ✔︎ ✔︎ Rican1996 Short intensive ABP 12 ✔︎ ✔︎ ✔︎ ✔︎ ✔︎ ✔︎ ✔︎ ✔︎ ✔︎ ✔︎ ✔︎ ✔︎ Sapouna2010 FearNot 2 ✔︎ ✔︎ Silva2016 Social-skills training 7 ✔︎ ✔︎ ✔︎ ✔︎ ✔︎ ✔︎ ✔︎ Sismani2014 Daphne III 1 ✔︎ Solomontos-Kountouri2016 ViSC 10 ✔︎ ✔︎ ✔︎ ✔︎ ✔︎ ✔︎ ✔︎ ✔︎ ✔︎ ✔︎ Sutherland2010 Beyond the Hurt 3 ✔︎ ✔︎ ✔︎ Toner2010 Bully-Proofing Your School 7 ✔ ✔︎ ✔︎ ✔︎ ✔︎ ✔︎ ✔︎ Williams2015b Start Strong 1 ✔ Wong2011 Restorative Whole-school approach 5 ✔︎ ✔︎ ✔︎ ✔︎ ✔︎ Yaakub2010 OBPP 14 ✔︎ ✔︎ ✔︎ ✔︎ ✔︎ ✔︎ ✔︎ ✔︎ ✔︎ ✔︎ ✔︎ ✔︎ ✔︎ ✔︎ Age Cohort designs Busch2013 Healthy Schools 7 ✔︎ ✔︎ ✔︎ ✔︎ ✔︎ ✔︎ ✔︎ Ertesvåg2004 Respect 6 ✔︎ ✔︎ ✔︎ ✔︎ ✔︎ ✔︎ Kärnä2011ac KiVa 15 ✔︎ ✔︎ ✔︎ ✔︎ ✔︎ ✔︎ ✔︎ ✔︎ ✔︎ ✔︎ ✔︎ ✔︎ ✔︎ ✔︎ ✔︎ Limber2018c OBPP 14 ✔︎ ✔︎ ✔︎ ✔︎ ✔︎ ✔︎ ✔︎ ✔︎ ✔︎ ✔︎ ✔︎ ✔︎ ✔︎ ✔︎ Olweus/Bergen 1 OBPP 14 ✔︎ ✔︎ ✔︎ ✔︎ ✔︎ ✔︎ ✔︎ ✔︎ ✔︎ ✔︎ ✔︎ ✔︎ ✔︎ ✔︎ Study Intervention Rich_Score WSA SUP ABP CRule CManage TInfo TTrain PInfo PInvolve Peer1 Peer2 Peer3 BULL VIC CoOp Curriculum SESkills MH Punitive Non-punitive Olweus/New National c OBPP 14 ✔︎ ✔︎ ✔︎ ✔︎ ✔︎ ✔︎ ✔︎ ✔︎ ✔︎ ✔︎ ✔︎ ✔︎ ✔︎ ✔︎ Olweus/Olso 1 OBPP 14 ✔︎ ✔︎ ✔︎ ✔︎ ✔︎ ✔︎ ✔︎ ✔︎ ✔︎ ✔︎ ✔︎ ✔︎ ✔︎ ✔︎ Olweus/Olso 2 OBPP 14 ✔︎ ✔︎ ✔︎ ✔︎ ✔︎ ✔︎ ✔︎ ✔︎ ✔︎ ✔︎ ✔︎ ✔︎ ✔︎ ✔︎ O’Moore2004 Donegal ABP 11 ✔︎ ✔︎ ✔︎ ✔︎ ✔︎ ✔︎ ✔︎ ✔︎ ✔︎ ✔︎ ✔︎ Pagliocca2007 OBPP 14 ✔︎ ✔︎ ✔︎ ✔︎ ✔︎ ✔︎ ✔︎ ✔︎ ✔︎ ✔︎ ✔︎ ✔︎ ✔︎ Purugulla2011 OBPP 14 ✔︎ ✔︎ ✔︎ ✔︎ ✔︎ ✔︎ ✔︎ ✔︎ ✔︎ ✔︎ ✔︎ ✔︎ ✔︎ ✔︎ Roland2010c Zero Program 10 ✔︎ ✔︎ ✔︎ ✔︎ ✔︎ ✔︎ ✔︎ ✔︎ ✔︎ ✔︎ Salmivalli2005 Finnish ABP 10 ✔︎ ✔︎ ✔︎ ✔︎ ✔︎ ✔︎ ✔︎ ✔︎ ✔︎ ✔︎ Whitney1994 Sheffield ABP 13 ✔︎ ✔︎ ✔ ✔︎ ✔︎ ✔︎ ✔︎ ✔︎ ✔︎ ✔︎ ✔︎ ✔︎ ✔︎ Note. ✔︎ = component present; ABP = Anti-bullying program; OBPP = Olweus Bullying Prevention Program; BPCCC = Bully Prevention Challenge Course Curriculum; Rich_Score = sum total number of components included in intervention; WSA = whole school approach; SUP = increased supervision; ABP = anti-bullying policy; CRule = classroom rules; CManage = classroom management; TInfo = Information for teachers; TTrain = Teacher training; PInfo = Information for parents; PInvolve = Parental involvement; Peer1 = informal peer involvement; Peer2 = Encouraging bystanders; Peer3 = Formal peer involvement; BULL = Work with individual bullies; VIC = Work with individual victims; CoOp = Co-operative group work; Curriculum = Set intervention curriculum materials; SESkills = Socio-emotional skills; MH = Mental health; Punitive = Punitive disciplinary methods; Non-punitive = Non-punitive disciplinary methods. a. Studies only reported effectiveness in reducing bullying perpetration outcomes b. Studies only reported effectiveness in reducing bullying victimization outcomes c. Studies were deemed ‘over-powered’ and thus removed from the model for the purpose of intervention component analyses. Table 2: Subgroup analysis for school-bullying perpetration outcomes under the MVA model of meta-analysis, omitting over-powered studies (N = 82) Intervention Component Component Present Component Absent Qb p N OR 95% CI N OR 95% CI School-level Whole-school approach 43 1.263 1.159 – 1.377 39 1.095 0.955 – 1.256 10.291 .001* Increased supervision 21 1.238 1.117 – 1.371 61 1.194 1.073 – 1.329 .812 .368 Anti-bullying policy 25 1.288 1.167 – 1.422 57 1.150 1.013 – 1.282 7.992 .005* Classroom-level Classroom rules 31 1.289 1.205 – 1.379 51 1.137 1.290 – 1.002 9.787 .002* Classroom management 22 1.265 1.166 – 1.372 60 1.165 1.038 – 1.307 4.222 .039** Teacher-level Information for teachers 66 1.219 1.124 – 1.321 16 1.155 0.894 – 1.492 .533 .465 Teacher training 51 1.194 1.089 – 1.309 31 1.292 1.118 – 1.492 2.501 .114 Parent-level Information for parents 35 1.280 1.177 – 1.392 47 1.141 1.078 – 1.209 8.149 .004* Involvement of parents 21 1.149 0.964 – 1.370 61 1.226 1.125 – 1.335 1.368 .242 Peer-level Informal peer involvement 57 1.294 1.199 – 1.396 25 1.022 0.948 – 1.102 27.440 .001* Encouraging bystanders 25 1.170 1.066 – 1.285 57 1.237 1.178 – 1.298 1.729 .188 Formal peer involvement 13 1.324 1.129 – 1.553 69 1.194 1.096 – 1.301 3.544 .059 Individual-level Work with Bullies 27 1.147 1.116 – 1.179 55 1.166 1.045 – 1.301 0.163 .686 Works with Victims 31 1.285 1.177 – 1.404 51 1.151 1.025 – 1.292 7.593 .006* Co-operative group work 37 1.329 1.207 – 1.464 45 1.148 1.029 – 1.279 12.619 .001* Intervention-specific Curriculum materials 69 1.263 1.172 – 1.361 13 0.980 0.762 – 1.260 21.343 .001* Socio-emotional skills 27 1.027 0.866 – 1.218 55 1.307 1.217 – 1.403 30.733 .001* Mental Health 8 1.523 1.157 – 2.004 77 1.163 1.091 – 1.239 11.201 .001* Punitive disciplinary methods 16 1.279 1.162 – 1.409 66 1.178 1.066 – 1.302 3.966 .046** Non-punitive disciplinary methods 11 1.284 1.125 – 1.466 71 1.196 1.096 – 1.306 1.994 .158 Note. 1. * = p < 0.001; ** = p < 0.05, i.e. the difference between mean effect sizes for subgroups is statistically significant at the respective p level. 2. Odds ratios presented in bold were the significantly larger subgroup mean summary effect size. 3. Values for Qb were estimated using the fixed effects model. Table 3: Subgroup analysis for school-bullying victimization outcomes under the MVA model of meta-analysis, omitting over-powered studies (N = 86) Intervention Component Component Present Component Absent Qb p N OR 95% CI N OR 95% CI School-level Whole-school approach 42 1.186 1.096 – 1.307 44 1.226 1.065 – 1.412 0.575 .448 Increased supervision 21 1.215 1.077 – 1.371 65 1.179 1.071 – 1.297 0.607 .436 Anti-bullying policy 26 1.219 1.101 – 1.351 60 1.169 1.051 – 1.300 1.158 .282 Classroom-level Classroom rules 30 1.236 1.125 – 1.358 56 1.152 1.033 – 1.285 3.209 .073 Classroom management 22 1.196 1.114 – 1.285 64 1.159 1.038 – 1.294 0.646 .420 Teacher-level Information for teachers 70 1.249 1.199 – 1.301 16 1.151 0.904 – 1.465 1.205 .272 Teacher training 55 1.192 1.091 – 1.303 31 1.211 1.065 – 1.377 0.115 .735 Parent-level Information for parents 36 1.246 1.132 – 1.371 50 1.125 1.007 – 1.257 6.492 .011* Involvement of parents 24 1.197 0.979 – 1.463 62 1.196 1.111 – 1.289 0.001 .992 Peer-level Informal peer involvement 55 1.246 1.138 – 1.363 31 1.096 0.975 – 1.232 9.36 .002* Encouraging bystanders 25 1.199 1.049 – 1.369 62 1.293 1.225 – 1.366 4.042 .044** Formal peer involvement 15 1.263 1.087 – 1.466 71 1.178 1.085 – 1.279 2.151 .143 Individual-level Work with Bullies 28 1.203 1.073 – 1.349 58 1.191 1.082 – 1.311 0.071 .791 Works with Victims 36 1.214 1.129 – 1.305 50 1.178 1.072 – 1.295 0.581 .446 Co-operative group work 43 1.213 1.089 – 1.349 43 1.184 1.072 -1.307 0.385 .535 Intervention-specific Curriculum materials 71 1.192 1.049 – 1.354 15 1.118 0.976 – 1.281 1.481 .224 Socio-emotional skills 30 1.039 0.884 – 1.221 56 1.252 1.161 – 1.349 16.859 .001* Mental Health 8 1.103 0.811 – 1.501 78 1.201 1.114 – 1.294 0.775 .378 Punitive disciplinary methods 14 1.257 1.092 – 1.447 72 1.169 1.073 – 1.273 3.044 .081 Non-punitive disciplinary methods 11 1.126 – 1.370 75 1.182 1.084 – 1.289 1.211 .271 Note. 1. * = p < 0.001; ** = p < 0.05, i.e. the difference between mean effect sizes for subgroups is statistically significant at the respective p level. 2. Odds ratios presented in bold were the significantly larger subgroup mean summary effect size. 3. Values for Qb were estimated using the fixed effects model.