Article Critique #4
Note: This is a required assignment for a course (CEP 900) at Michigan State University. The article reviewed below was assigned by the instructor.
Li, Q. (2006). Cyberbullying in Schools:A Research of Gender Differences. School Psychology International, 27(2), 157-170. doi:10.1177/0143034306064547
1,725 words (Questions are 478 words.)
A. Theoretical Perspective
1. Critique the author’s conceptual framework. (NOTE: This means the conceptual logic the authors use to introduce their study. Usually starting broad and narrowing down to something specific).
Li’s conceptual logic takes two disparate paths and attempts to bring them together in establishing the context for her research. The first path is to start with the problem of school violence and establish that secondary school is a time of increased aggression amongst peers and bullying plays a role in aggressive behaviors. Li then diverts to the adoption of new technologies in schools and establishes that (in 2006) not many school administrators were aware of the cyberbullying problem. Li then concludes that cyberbullying is new territory for research. Li is correct in assuming that new technologies may pose not only a new medium for student misbehavior, but for reexamining the assumed relationships between gender and bullying in the new context of changing technologies. The introduction of readily available communications technology in schools may be a new exogenous factor in the existing relationship between bullying and gender, requiring the retesting of old assumptions in the new environment.
2. Comment on the need for this study and its importance. (NOTE: What do the authors say about why this study is needed? Is it compelling? Non-existent?)
Li’s justification for her research is twofold. At the time of publication little was known about the extent of cyber bullying and what role gender might play in the experience. Li spends most of her efforts arguing for justification based upon specific examples of harm and survey data to address the scope of the issue. More time is given in her justification to novel empirical examples than research backed data on the prevalence of the issue – however, the examples bring a sense of urgency to the threat. This logically leads the reader to conclude that significant numbers of students are most likely being harmed by a poorly understood phenomena. Li describes educators as being unaware (in 2006) of cyberbullying and needing insight to better intervene and prevent student harm. No further rationale is necessary for this research.
3. How effectively does the author tie the study to relevant theory and prior research? (Do they cover the right literature? Anything missing? Anything not needed? How comprehensive? Tied to their conceptual framework?)
Li links her study of gender roles through research indicating that perpetrators and victims of bulling and cyberbullying overlapped. This existing research has indicated that bullies tend to be cyber bullies and their victims are often bullied both online and off. Therefore, it stands to reason that cyberbullies and offline bullies share a great deal in common. She then, reasonably, assumes that research on cyberbullying can be guided by examining existing correlations in the physical world. Prior research had also demonstrated that junior high school was a period of peak behavior and situating the study in this time-period should yield the most observable results.
4. Evaluate the clarity and appropriateness of the research questions or hypotheses. (NOTE: Either their explicit or implied questions. Appropriateness can take many forms, but especially its’ connection to 1-3 above)
Li’s research questions examine the effect of gender upon the experiences with cyberbullying, belief in adult prevention of cyberbullying, and reporting behavior when cyberbullying occurs. Previous research in offline behavior had demonstrated that gender played a meaningful role in the likelihood of perpetrating bullying, the nature of bullying, and victimization in bullying. Li’s objective is to examine if that role is different due to exogenous changes in student interactions from the introduction of technology. As such, her choice of research question fits well within the existing research and may provide valuable insight to educators seeking to adapt existing strategies to a new environment.
B. Research Design and Analysis
- Critique the appropriateness and adequacy of the study’s design in relation to the research questions or hypotheses. (NOTE: Try to name a design — pre-post; longitudinal, quasi-experimental; experimental; ethnographic; qualitative (which kind?); etc. Then try to talk about if that design is or is not well-suited to answer the type of research questions that were stated.
Li employs a casual-comparative experimental design in her study. Because the introduction of technology has already occurred this is an appropriate method of attempting to examine the impact of technology upon bullying behavior. More importantly Li’s independent variable is gender, which cannot be manipulated and therefore lends itself to this experimental design. Li’s research is attempting to capture the past experiences of a broad sample, and therefore she appropriately applies a survey method to her research questions. There are not many other methods for describing the experiences of entire populations beyond large scale observation. Since bullying, and particularly cyberbullying, occur in secret – observational methods may be prone to underreporting since the presence of the observer would dramatically alter behavior. To a large extent Li’s choice of survey method was dictated by the lack of other options.
2. Critique the adequacy of the study’s sampling methods (e.g., choice of participants) and their implications for generalizability.
A strength of Li’s study is the size of the study sample (n=264). Li also perfectly targets the population of interest through her examination of 7th-9th graders. Li’s sample is gender balanced, but predominantly white and reported well above average academic achievement. Li’s population is from a large urban area in Canada which may limit its generalizability to rural areas and less polite cultures.
3. Critique the adequacy of the study’s procedures and materials (e.g., interventions, interview protocols, data collection procedures). (NOTE: Sometimes it’s helpful to think of “procedures” as all the steps taken to get to the point of data analysis. Example: obtained consent; randomly sampled, filled out questionnaire, etc.)
Li’s vague description of her methods leave critical readers with as many questions as answers. Li’s method was to distribute an anonymous paper survey to middle school students. Li gave no mention of informed consent nor the conditions under which the survey was conducted such as groups, classrooms, or individually. No mention was made of the instructions given to respondents, such as the defining of terms like cyberbullying (Li does define them for the readers of the study, but not for the middle schoolers completing the survey). The survey itself leaves much to be desired. There are no questions to check for consistency of answers. All surveys are subject to issues of bias in self-reporting, particularly behaviors that are socially undesirable or secretive in nature. To some extent the social desirability bias can be clearly seen in the large numbers of students who reported their grades as “above average”.
4. Critique the appropriateness and quality (e.g., reliability, validity) of the measures used. (NOTES: Google reliability and validity. To what extent do the authors discuss these issues and what implications does this have for their study?)
While motivations may have distorted self-reporting in the survey, other factors may have acted upon student’s perceptions of the prevalence of cyberbullying. Dramatically higher numbers of students reported knowing another student who had been cyberbullied, when compared to previous surveys. This could be due to the availability heuristic where dramatic or novel events tend to stand out in an individuals awareness, or exciting information gets shared with greater frequency. It may also stem from the fact that Li asked an awareness question on her survey, “I know someone who has been cyberbullied.” (Li, 2006) Li could have asked an experience-based question such as “I have personally seen someone cyberbullied online”. A personal-experience question may have given greater meaning to this measure of the breadth of cyberbullying. Li categorically ignores all these possibilities during the analysis and interpretation of results.
Li’s measurements are, however, consistent with the aims of the research. Li compares male and female rates of bullying and cyber bullying. Discovering that incidents of online and offline bulling are similar when correlated to gender.
5. Critique the adequacy of the study’s data analyses. For example: Have important statistical assumptions been met? Are the analyses appropriate for the study’s design? Are the analyses appropriate for the data collected? (NOTE: Consider also if the analyses answer the questions being asked.)
Li’s data analysis shows a significant difference in likelihood of perpetrating bullying (p=0.028) or cyberbullying (p=0.021) based upon gender. Gender did not play a statistically significant role in frequency of bullying, cyber bullying, or being a victim of either. Li is careful to make the distinction between what is significant and what is insignificant in the results. Some of this caution is warranted by the causal-comparative design study which does not clearly establish a cause and effect case like a traditional experiment where the independent variable can be manipulated. Unfortunately, Li intertwines these points with other findings for which no statistical analysis was done – to an extent disguising them as findings of interest. For example, Li reports that “55 percent of cyberbullies harassed others between one to three times” and “Only 64 percent of students believed that adults in schools tried to stop cyberbullying” (Li, 2006). These results can easily be read as alarming and meaningful, unless closely scrutinized. Her word choice matters here – using the exact same data points, one could also conclude that the majority of cyberbullies reported limited offensive behavior and an overwhelming majority of students believed adults would intervene if informed. Li’s choice of language and interspersing of findings unrelated to the variable in question demonstrates a bias towards the sensational. Alternatively, a clearly stated null hypothesis and the implications of the support for the null alternative may have led to a clearer and less misleading discussion.
C. Interpretation and Implications of Results
1. Critique the author’s discussion of the methodological and/or conceptual limitations of the results. (NOTE: Just like we did in class)
Li’s consideration of the limits of this study, given in the final paragraph, is perfunctory at best. Li notes that the urban setting should be reason to give caution when generalizing to other contexts. Additionally, Li makes note of that it may be valuable to examine the issue of gender in cyberbullying at other ages such as elementary or high school. Li leaves it to the reader to speculate about the limitations in this research stemming from survey use, self-reporting bias, availability heuristic, and external validity (frequency of bullying).
2. How consistent and comprehensive are the author’s conclusions with the reported results? (NOTE: Do they over-state or understate their findings? Does their conclusions “go beyond the data.” Have they not talked about something they should have?)
Li reaches several sweeping conclusions in the discussion of her research. Li discusses the prevalence of bullying but notes that in her sample the frequency of bullying was higher than in previous research. Li’s second point is that the frequency of students who know of another being bullied deserves serious consideration. These are, of course, important issues to discuss, but have little to do with the central research questions of this study. The only gender specific finding of any significance that Li has available to discuss is that males are more likely to cyberbully than females, a finding that is indistinguishable from offline bullying research. Li’s other points of discussion such as the underreporting of (offline) bullying and her call for communities, families, schools, and societies to address cyberbullying can only be loosely affiliated with the data from her research. Instead there is a stronger case connecting these ideas and recommendations to the existing background research.
3. How well did the author relate the results to the study’s theoretical base? (NOTE: The theoretical base (conceptual framework + literature review) were established in the introduction. Do they return to this framework and talk about how their data fits — or doesn’t — within this framework?)
By and large, Li did an adequate job relating her data back to the theoretical base. The vast majority of Li’s results are found to be consistent with offline behavior, and gender does not play a different role as a mediator of online versus offline bullying. Most of Li’s discussion is so consistent with the background research that it leads a reader to wonder if her findings needed to be part of the research. This is not to state or imply that only significant findings should be published, such a belief would lead to an amplification of what has been called the “file drawer bias” in research publications. Instead, deriving more of her discussion from her results, instead of additional background research may have proven useful.
4. In your view, what is the significance of the study, and what are its primary implications for theory, future research, and practice? (Remember: Be specific)
In not finding that gender plays a different role in cyber bullying than in offline bullying, Li’s research points to an important possibility. If much of what we know about regular bullying applies to cyber bullying then expansion of existing efforts might be logical, and training can involve the application of existing strategies to the new domain. Further research is necessary to examine the degree of applicability between these two domains of student behavior.
Reference:
Li, Q. (2006). Cyberbullying in Schools:A Research of Gender Differences. School Psychology International, 27(2), 157-170. doi:10.1177/0143034306064547