Research responding to this call for a better understanding of IGD is still at a formative stage, and there are active debates surrounding it. There is a growing literature that suggests there is a basis to expect that excessive or problematic gaming may be related to lower health, though findings in this area are mixed. Some argue for a theoretical framing akin to a substance abuse disorder (i.e. where gaming is considered to be inherently addictive), while others frame Internet-based gaming as a self-regulatory challenge for individuals.

In their article “A prospective study of the motivational and health dynamics of Internet Gaming Disorder“, Netta Weinstein, the OII’s Andrew Przybylski, and Kou Murayama address this gap in the literature by linking self-regulation and Internet Gaming Disorder research. Drawing on a representative sample of 5,777 American adults they examine how problematic gaming emerges from a state of individual “dysregulation” and how it predicts health — finding no evidence directly linking IGD to health over time.

This negative finding indicates that IGD may not, in itself, be robustly associated with important clinical outcomes. As such, it may be premature to invest in management of IGD using the same kinds of approaches taken in response to substance-based addiction disorders. Further, the findings suggests that more high-quality evidence regarding clinical and behavioural effects is needed before concluding that IGD is a legitimate candidate for inclusion in future revisions of the Diagnostic and Statistical Manual of Mental Disorders.

We caught up with Andy to explore the implications of the study:

Ed: To ask a blunt question upfront: do you feel that Internet Gaming Disorder is a valid psychiatric condition (and that “games can cause problems”)? Or is it still too early to say?

Andy: No, it is not. It’s difficult to overstate how sceptical the public should be of researchers who claim, and communicate their research, as if Internet addiction, gaming addiction, or Internet gaming disorder (IGD) are recognized psychiatric disorders. The fact of the matter is that American psychiatrists working on the most recent revision of the Diagnostic and Statistical Manual of Mental Disorders (DSM5) highlighted that problematic online play was a topic they were interested in learning more about. These concerns are highlighted in Section III of the DSM5 (entitled “Emerging Measures and Models”). For those interested in this debate see this position paper.

Ed: Internet gaming seems like quite a specific activity to worry about: how does it differ from things like offline video games, online gambling and casino games; or indeed the various “problematic” uses of the Internet that lead to people admitting themselves to digital detox camps?

Andy: In some ways computer games, and Internet ones, are distinct from other activities. They are frequently updated to meet players expectations and some business models of games, such as pay-to-play are explicitly designed to target high engagement players to spend real money for in-game advantages. Detox camps are very worrying to me as a scientist because they have no scientific basis, many of those who run them have financial conflicts of interest when they comment in the press, and there have been a number of deaths at these facilities.

Ed: You say there are two schools of thought: that if IGD is indeed a valid condition, that it should be framed as an addiction, i.e. that there’s something inherently addictive about certain games. Alternatively, that it should be framed as a self-regulatory challenge, relating to an individual’s self-control. I guess intuitively it might involve a bit of both: online environments can be very persuasive, and some people are easily persuaded?

Andy: Indeed it could be. As researchers mainly interested in self-regulation we’re most interested in gaming as one of many activities that can be successfully (or unsuccessfully) integrated into everyday life. Unfortunately we don’t know much for sure about whether there is something inherently addictive about games because the research literature is based largely on inferences based on correlational data, drawn from convenience samples, with post-hoc analyses. Because the evidence base is of such low quality most of the published findings (i.e. correlations/factor analyses) regarding gaming addiction supporting it as valid condition likely suffer from the Texas Sharpshooter Fallacy.

Ed: Did you examine the question of whether online games may trigger things like anxiety, depression, violence, isolation etc. — or whether these conditions (if pre-existing) might influence the development of IGD?

Andy: Well, our modelling focused on the links between Internet Gaming Disorder, health (mental, physical, and social), and motivational factors (feeling competent, choiceful, and a sense of belonging) examined at two time points six months apart. We found that those who had their motivational needs met at the start of the study were more likely to have higher levels of health six months later and were less likely to say they experienced some of the symptoms of Internet Gaming Disorder.

Though there was no direct link between Internet Gaming Disorder and health six months later, we perform an exploratory analysis (one we did not pre-register) and found an indirect link between Internet Gaming Disorder and health by way of motivational factors. In other words, Internet Gaming Disorder was linked to lower levels of feeling competent, choiceful, and connected, which was in turn linked to lower levels of health.

Ed: All games are different. How would a clinician identify if someone was genuinely “addicted” to a particular game — there would presumably have to be game-by-game ratings of their addictive potential (like there are with drugs). How would anyone find the time to do that? Or would diagnosis focus more on the individual’s behaviour, rather than what games they play? I suppose this goes back to the question of whether “some games are addictive” or whether just “some people have poor self-control”?

Andy: No one knows. In fact, the APA doesn’t define what “Internet Games” are. In our research we define ask participants to define it for themselves by “Think[ing] about the Internet games you may play on Facebook (e.g. Farmville), Tablet/Smartphones (e.g. Candy Crush), or Computer/Consoles (e.g. Minecraft).” It’s very difficult to overstate how suboptimal this state of affairs is from a scientific perspective.

Ed: Is it odd that it was the APA’s Substance-Related Disorders Work Group that has called for research into IGD? Are “Internet Games” unique in being classed as a substance, or are there other information based-behaviours that fall under the group’s remit?

Andy: Yes it’s very odd. Our research group is not privy to these discussions but my understanding is that a range of behaviours and other technology-related activities, such as general Internet use have been discussed.

Ed: A huge amount of money must be spent on developing and refining these games, i.e. to get people to spend as much time (and money) as possible playing them. Are academics (and clinicians) always going to be playing catch-up to industry?

Andy: I’m not sure that there is one answer to this. One useful way to think of online games is using the example of a gym. Gyms are most profitable when many people are paying for (and don’t cancel) their memberships but owners can still maintain a small footprint. The world’s most successful gym might be a one square meter facility, with seven billion members but no one ever goes. Many online games are like this, some costs scale nicely, but others have high costs, like servers, community management, upkeep, and power. There are many studying the addictive potential of games but because they constantly reinvent the wheel by creating duplicate survey instruments (there are literally dozens that are only used once or a couple of times) very little of real-world relevance is ever learned or transmitted to the public.

Ed: It can’t be trivial to admit another condition into the Diagnostic and Statistical Manual of Mental Disorders (DSM-5)? Presumably there must be firm (reproducible) evidence that it is a (persistent) problem for certain people, with a specific (identifiable) cause — given it could presumably be admitted in courts as a mitigating condition, and possibly also have implications for health insurance and health policy? What are the wider implications if it does end up being admitted to the DSM-5?

Andy: It is very serious stuff. Opening the door to pathologizing one of the world’s most popular recreational activities risks stigmatizing hundreds of millions of people and shifting resources in an already overstretched mental health systems over the breaking point.

Ed: You note that your study followed a “pre-registered analysis plan” — what does that mean?

Andy: We’ve discussed the wider problems in social, psychological, and medical science before. But basically, preregistration, and Registered Reports provide scientists a way to record their hypotheses in advance of data collection. This improves the quality of the inferences researchers draw from experiments and large-scale social data science. In this study, and also in our other work, we recorded our sampling plan, our analysis plan, and our materials before we collected our data.

Ed: And finally: what follow up studies are you planning?

Andy: We are now conducting a series of studies investigating problematic play in younger participants with a focus on child-caregiver dynamics.

Read the full article: Weinstein N, Przybylski AK, Murayama K. (2017) A prospective study of the motivational and health dynamics of Internet Gaming Disorder. PeerJ 5:e3838 https://doi.org/10.7717/peerj.3838

Additional peer-reviewed articles in this area by Andy include:

Przybylski, A.K. & Weinstein N. (2017). A Large-Scale Test of the Goldilocks Hypothesis: Quantifying the Relations Between Digital Screens and the Mental Well-Being of Adolescents. Psychological Science. DOI: 10.1177/0956797616678438.

Przybylski, A. K., Weinstein, N., & Murayama, K. (2016). Internet Gaming Disorder: Investigating the Clinical Relevance of a New Phenomenon. American Journal of Psychiatry. DOI: 10.1176/appi.ajp.2016.16020224.

Przybylski, A. K. (2016). Mischievous responding in Internet Gaming Disorder research. PeerJ, 4, e2401. https://doi.org/10.7717/peerj.2401

For more on the ongoing “crisis in psychology” and how pre-registration of studies might offer a solution, see this discussion with Andy and Malte Elson: Psychology is in crisis, and here’s how to fix it.

Andy Przybylski was talking to blog editor David Sutcliffe.

While nearly a third of the adolescent respondents reported experiencing significant bullying in the past few months, cyberbullying was much less common, with around five percent of respondents reporting recent significant experiences. Both traditional and cyberbullying were independently associated with lower mental well-being, but only the relation between traditional bullying and well-being was robust. This supports the view that cyberbullying is unlikely to provide a source for new victims, but rather presents an avenue for further victimisation of those already suffering from traditional forms of bullying.

This stands in stark contrast to media reports and the popular perception that young people are now more likely to be victims of cyberbullying than traditional forms. The results also suggest that interventions to address cyberbullying will only be effective if they also consider the dynamics of traditional forms of bullying, supporting the urgent need for evidence-based interventions that target *both* forms of bullying in adolescence. That said, as social media and Internet connectivity become an increasingly intrinsic part of modern childhood, initiatives fostering resilience in online and every day contexts will be required.

We caught up with Andy and Lucy to discuss their findings:

Ed.: You say that given “the rise in the use of mobile and online technologies among young people, an up to date estimation of the current prevalence of cyberbullying in the UK is needed.” Having undertaken that—what are your initial thoughts on the results?

Andy: I think a really compelling thing we learned in this project is that researchers and policymakers have to think very carefully about what constitutes a meaningful degree of bullying or cyberbullying. Many of the studies and reports we reviewed were really loose on details here while a smaller core of work was precise and informative. When we started our study it was difficult to sort through the noise but we settled on a solid standard—at least two or three experiences of bullying in the past month—to base our prevalence numbers and statistical models on.

Lucy: One of the issues here is that studies often use different measures, so it is hard to compare like for like, but in general our study supports other recent studies indicating that relatively few adolescents report being cyberbullied only—one study by Dieter Wolke and colleagues that collected between 2014-2015 found that whilst 29% of school students reported being bullied, only 1% of 11-16 year olds reported only cyberbullying. Whilst that study was only in a handful of schools in one part of England, the findings are strikingly similar to our own. In general then it seems that rates of cyberbullying are not increasing dramatically; though it is concerning that prevalence rates of both forms of bullying—particularly traditional bullying—have remained unacceptably high.

Ed.: Is there a policy distinction drawn between “bullying” (i.e. young people) and “harassment” (i.e. the rest of us, including in the workplace)—and also between “bullying” and “cyber-bullying”? These are all basically the same thing, aren’t they—why distinguish?

Lucy: I think this is a good point; people do refer to ‘bullying’ in the workplace as well. Bullying, at its core, is defined as intentional, repeated aggression targeted against a person who is less able to defend him or herself—for example, a younger or more vulnerable person. Cyberbullying has the additional definition of occurring only in an online format—but I agree that this is the same action or behaviour, just taking place in a different context. Whilst in practice bullying and harassment have very similar meanings and may be used interchangeably, harassment is unlawful under the Equality Act 2010, whilst bullying actually isn’t a legal term at all. However certain acts of bullying could be considered harassment and therefore be prosecuted. I think this really just reflects the fact that we often ‘carve up’ human behaviour and experience according to our different policies, practices and research fields—when in reality they are not so distinct.

Ed.: I suppose online bullying of young people might be more difficult to deal with, given it can occur under the radar, and in social spaces that might not easily admit adults (though conversely, leave actual evidence, if reported..). Why do you think there’s a moral panic about cyberbullying — is it just newspapers selling copy, or does it say something interesting about the Internet as a medium — a space that’s both very open and very closed? And does any of this hysteria affect actual policy?

Andy: I think our concern arises from the uncertainty and unfamiliarity people have about the possibilities the Internet provides. Because it is full of potential—for good and ill—and is always changing, wild claims about it capture our imagination and fears. That said, the panic absolutely does affect policy and parenting discussions in the UK. Statistics and figures coming from pressure groups and well-meaning charities do put the prevalence of cyberbullying at terrifying, and unrealistically high, levels. This certainly has affected the way parents see things. Policy makers tend to seize on the worse case scenario and interpret things through this lens. Unfortunately this can be a distraction when there are known health and behavioural challenges facing young people.

Lucy: For me, I think we do tend to panic and highlight the negative impacts of the online world—often at the expense of the many positive impacts. That said, there was—and remains—a worry that cyberbullying could have the potential to be more widespread, and to be more difficult to resolve. The perpetrator’s identity may be unknown, may follow the child home from school, and may be persistent—in that it may be difficult to remove hurtful comments or photos from the Internet. It is reassuring that our findings, as well as others’, suggest that cyberbullying may not be associated with as great an impact on well-being as people have suggested.

Ed.: Obviously something as deeply complex and social as bullying requires a complex, multivalent response: but (that said), do you think there are any low-hanging interventions that might help address online bullying, like age verification, reporting tools, more information in online spaces about available help, more discussion of it as a problem (etc.)?

Andy: No easy ones. Understanding that cyber- and traditional bullying aren’t dissimilar, parental engagement and keeping lines of communication open are key. This means parents should learn about the technology their young people are using, and that kids should know they’re safe disclosing when something scary or distressing eventually happens.

Lucy: Bullying is certainly complex; school-based interventions that have been successful in reducing more traditional forms of bullying have tended to involve those students who are not directly involved but who act as ‘bystanders’—encouraging them to take a more active stance against bullying rather than remaining silent and implicitly suggesting that it is acceptable. There are online equivalents being developed, and greater education that discourages people (both children and adults) from sharing negative images or words, or encourages them to actively ‘dislike’ such negative posts show promise. I also think it’s important that targeted advice and support for those directly affected is provided.

Ed.: Who’s seen as the primary body responsible for dealing with bullying online: is it schools? NGOs? Or the platform owners who actually (if not-intentionally) host this abuse? And does this topic bump up against wider current concerns about (e.g.) the moral responsibilities of social media companies?

Andy: There is no single body that takes responsibility for this for young people. Some charities and government agencies, like the Child Exploitation and Online Protection command (CEOP) are doing great work. They provide a forum for information for parents and professionals for kids that is stratified by age, and easy-to-complete forms that young people or carers can use to get help. Most industry-based solutions require users to report and flag offensive content and they’re pretty far behind the ball on this because we don’t know what works and what doesn’t. At present cyberbullying consultants occupy the space and the services they provide are of dubious empirical value. If industry and the government want to improve things on this front they need to make direct investments in supporting robust, open, basic scientific research into cyberbulling and trials of promising intervention approaches.

Lucy: There was an interesting discussion by the NSPCC about this recently, and it seems that people are very mixed in their opinions—some would also say parents play an important role, as well as Government. I think this reflects the fact that cyberbullying is a complex social issue. It is important that social media companies are aware, and work with government, NGOs and young people to safeguard against harm (as many are doing), but equally schools and parents play an important role in educating children about cyberbullying—how to stay safe, how to play an active role in reducing cyberbullying, and who to turn to if children are experiencing cyberbullying.

Ed.: You mention various limitations to the study; what further evidence do you think we need, in order to more completely understand this issue, and support good interventions?

Lucy: I think we need to know more about how to support children directly affected by bullying, and more work is needed in developing effective interventions for cyberbullying. There are some very good school-based interventions with a strong evidence base to suggest that they reduce the prevalence of at least traditional forms of bullying, but they are not being widely implemented in the UK, and this is a missed opportunity.

Andy: I agree—a focus on flashy cyberbullying headlines presents the real risk of distracting us from developing and implementing evidence-based interventions. The Internet cannot be turned off and there are no simple solutions.

Ed.: You say the UK is ranked 20th of 27 EU countries on the mental well-being index, and also note the link between well-being and productivity. Do you think there’s enough discussion and effort being put into well-being, generally? And is there even a general public understanding of what “well-being” encompasses?

Lucy: I think the public understanding of well-being is probably pretty close to the research definition—people have a good sense that this involves more than not having psychological difficulty for example, and that it refers to friendships, relationships, and doing well; one’s overall quality of life. Both research and policy is placing more of an emphasis on well-being—in part because large international studies have suggested that the UK may score particularly poorly on measures of well-being. This is very important if we are going to raise standards and improve people’s quality of life.

Read the full article: Andrew Przybylski and Lucy Bowes (2017) Cyberbullying and adolescent well-being in England: a population-based cross sectional study. The Lancet Child & Adolescent Health.

Andrew Przybylski is an experimental psychologist based at the Oxford Internet Institute. His research focuses on applying motivational theory to understand the universal aspects of video games and social media that draw people in, the role of game structure and content on human aggression, and the factors that lead to successful versus unsuccessful self-regulation of gaming contexts and social media use. @ShuhBillSkee

Lucy Bowes is a Leverhulme Early Career Research Fellow at Oxford’s Department of Experimental Psychology. Her research focuses on the impact of early life stress on psychological and behavioural development, integrating social epidemiology, developmental psychology and behavioural genetics to understand the complex genetic and environmental influences that promote resilience to victimization and early life stress. @DrLucyBowes

Andy Przybylski and Lucy Bowes were talking to the Oxford Internet Institute’s Managing Editor, David Sutcliffe.

Letting your child use social media is like giving them cocaine, alcohol and cigarettes – all at once, or so we’re told. If you have been following recent press reports about the effects of social media on young people, you may well believe this. But there is no scientific evidence to support such extreme claims.

The real story is far more complex. It is very difficult to predict how social media will affect any specific individual – the effect depends on things like their personality, type of social media use and social surroundings. In reality, social media can have both positive and negative outcomes.

Media reports that compare social media to drug use are ignoring evidence of positive effects, while exaggerating and generalising the evidence of negative effects. This is scaremongering – and it does not promote healthy social media use. We would not liken giving children sweets to giving children drugs, even though having sweets for every meal could have serious health consequences. We should therefore not liken social media to drugs either.

For a claim to be proved scientifically it needs to be thoroughly tested. To fully confirm The Independent’s headline that: “Giving your child a smartphone is like giving them a gram of cocaine, says top addiction expert”, you would need to give children both a gram of cocaine and a smartphone and then compare the effects. Similarly, you would need to provide millennials with social media, drugs and alcohol to test The Conversation’s headline that: “Social media is as harmful as alcohol and drugs for millennials”. But ethical guidelines at universities were put in place so that such studies will never be done.

The diversity of social media

But maybe news headlines should be discounted – as exaggerations are often used to grab the readers’ attention. But even when ignoring these grand claims, the media coverage of social media is still misleading. For example, reports that talk about the effects of social media are often oversimplifying reality. Social media is incredibly diverse – different sites providing a host of different features. This makes it extremely difficult to generalise about social media’s effects.

A recent review of past research concluded that the effect of Facebook depends on which of the platform’s features you use. A dialog with friends over Facebook messenger can improve your mood, while comparing your life to other people’s photos on the Newsfeed can do the opposite. By treating all social media sites and features as one concept, the media is oversimplifying something that is very complex.

Focusing on the negative

Past media coverage has not only oversimplified social media, but has often only focused on social media’s negative aspects. But scientific research demonstrates that there are both positive and negative outcomes of social media use. Research has shown that Facebook increases self-esteem and promotes feeling connected to others. People’s physiological reactions also indicate they react positively to Facebook use.

By contrast, it has also been found that social media can decrease well-being and increases social anxiety. An analysis of 57 scientific studies found that social media is associated with slightly higher levels of narcissism. This array of conflicting evidence suggests that social media has both negative and positive effects. Not just one or the other.

The amount matters

The effect of social media also depends on the amount of time you spend using it. In a recent study we conducted of more than 120,000 UK teenagers, we found that moderate social media use is not harmful to mental health.

We compared the relationship between screen time and well-being. We found that those who used screens a moderate amount – between one and three hours each day – reported higher well-being compared with those who didn’t use social media at all and those who used it more than three hours a day. So, unlike drugs, those who practise abstinence do not appear to fare better.

Recent media reports may have made parents unnecessarily anxious about their child’s use of social media. A flashy quote or headline can often distract from the real challenges of parenting. It’s time the media covered not only the bad, but also the beneficial and complex sides of social media. The effects of social media cannot be summarised by comparing social media to drugs. It is just not that simple.

Andy Przybylski, Associate Professor and Senior Research Fellow, University of Oxford and Amy C Orben, College Lecturer and DPhil Candidate, University of Oxford

This article was originally published on The Conversation. Read the original article.

However, a new article by Andrew Przybylski and Victoria Nash, “Internet Filtering Technology and Aversive Online Experiences in Adolescents”, published in the Journal of Pediatrics, finds equivocal to strong evidence that household-level Internet filtering does not reduce the chance of adolescents having recent aversive online experiences. The authors analysed data from 1030 in-home interviews conducted with early adolescents as part of Ofcom’s Children and Parents Media Use and Attitudes Report.

The Internet is now a central fixture of modern life, and the positives and negatives of online Internet use need to be balanced by caregivers. Internet filters have been adopted as a tool for limiting the negatives; however, evidence of their effectiveness is dubious. They are expensive to develop and maintain, and also carry significant informational costs: even sophisticated filters over-block, which is onerous for those seeking information about sexual health, relationships, or identity, and might have a disproportionate effect on vulnerable groups. Striking the right balance between protecting adolescents and respecting their rights to freedom of expression and information presents a formidable challenge.

In conducting their study to address this uncertainty, the authors found convincing evidence that Internet filters were not effective at shielding early adolescents from aversive experiences online. Given this finding, they propose that evidence derived from a randomized controlled trial and registered research designs are needed to determine how far Internet-filtering technology supports or thwarts young people online. Only then will parents and policymakers be able to make an informed decision as to whether their widespread use justifies their costs.

We caught up with Andy and Vicki to discuss the implications of their study:

Ed.: Just this morning when working from home I tried to look up an article’s author and was blocked, Virgin Media presumably having decided he might be harmful to me. Where does this recent enthusiasm for default-filtering come from? Is it just that it’s a quick, uncomplicated (technological) fix, which I guess is what politicians / policy-people like?

Vicki: In many ways this is just a typical response to the sorts of moral panic which have long arisen around the possible risks of new technologies. We saw the same concerns arise with television in the 1960s, for example, and in that case the UK’s policy response was to introduce a ‘watershed’, a daily time after which content aimed at adults could be shown. I suppose I see filtering as fulfilling the same sort of policy gap, namely recognising that certain types of content can be legally available but should not be served up ‘in front of the children’.

Andy: My reading of the psychological and developmental literature suggests that filters provide a way of creating a safe walled space in schools, libraries, and homes for young people to use the internet. This of course does not mean that reading our article will be harmful!

Ed.: I suppose that children desperate to explore won’t be stopped by a filter; those who aren’t curious probably wouldn’t encounter much anyway — what is the profile of the “child” and the “harm-filtering” scenario envisaged by policy-makers? And is Internet filtering basically just aiming at the (easy) middle of the bell-curve?

Vicki: This is a really important point. Sociologists recognised many years ago that the whole concept of childhood is socially constructed, but we often forget about this when it comes to making policy. There’s a tendency for politicians, for example, either to describe children as inherently innocent and vulnerable, or to frame them as expert ‘digital natives’, yet there’s plenty of academic research which demonstrates the extent to which children’s experiences of the Internet vary by age, education, income and skill level.

This matters because it suggests a ‘one-size-fits-all’ approach may fail. In the context of this paper, we specifically wanted to check whether children with the technical know-how to get around filters experienced more negative experiences online than those who were less tech-savvy. This is often assumed to be true, but interestingly, our analysis suggests this factor makes very little difference.

Ed.: In all these discussions and policy decisions: is there a tacit assumption that these children are all growing up in a healthy, supportive (“normal”) environment — or is there a recognition that many children will be growing up in attention-poor (perhaps abusive) environments and that maybe one blanket technical “solution” won’t fit everyone? Is there also an irony that the best protected children will already be protected, and the least protected, probably won’t be?

Andy: Yes this is ironic and somewhat tragic dynamic. Unfortunately because the evidence for filtering effectiveness is at such an early age it’s not possible to know which young people (if any) are any more or less helped by filters. We need to know how effective filters are in general before moving on to see the young people for whom they are more or less helpful. We would also need to be able to explicitly define what would constitute an ‘attention-poor’ environment.

Vicki: From my perspective, this does always serve as a useful reminder that there’s a good reason why policy-makers turn to universalistic interventions, namely that this is likely to be the only way of making a difference for the hardest-to-reach children whose carers might never act voluntarily. But admirable motives are no replacement for efficacy, so as Andy notes, it would make more sense to find evidence that household internet filtering is effective, and second that it can be effective for this vulnerable group, before imposing default-on filters on all.

Ed.: With all this talk of potential “harm” to children posed by the Internet .. is there any sense of how much (specific) harm we’re talking about? And conversely .. any sense of the potential harms of over-blocking?

Vicki: No, you are right to see that the harms of Internet use are quite hard to pin down. These typically take the form of bullying, or self-harm horror stories related to Internet use. The problem is that it’s often easier to gauge how many children have been exposed to certain risky experiences (e.g. viewing pornography) than to ascertain whether or how they were harmed by this. Policy in this area often abides by what’s known as ‘the precautionary principle’.

This means that if you lack clear evidence of harm but have good reason to suspect public harm is likely, then the burden of proof is on those who would prove it is not likely. This means that policies aimed at protecting children in many contexts are often conservative, and rightly so. But it also means that it’s important to reconsider policies in the light of new evidence as it comes along. In this case we found that there is not as much evidence that Internet filters are effective at preventing exposure to negative experiences online as might be hoped.

Ed.: Stupid question: do these filters just filter “websites”, or do they filter social media posts as well? I would have thought young teens would be more likely to find or share stuff on social media (i.e. mobile) than “on a website”?

Andy: My understanding is that there are continually updated ‘lists’ of website that contain certain kinds of content such as pornography, piracy, gambling, or drug use (see this list on Wikipedia, for example) as these categories vary by UK ISP.

Vicki: But it’s not quite true to say that household filtering packages don’t block social media. Some of the filtering options offered by the UK’s ‘Big 4’ ISPs enable parents and carers to block social media sites for ‘homework time’ for example. A bigger issue though, is that much of children’s Internet use now takes place outside the home. So, household-level filters can only go so far. And whilst schools and libraries usually filter content, public wifi or wifi in friends’ houses may not, and content can be easily exchanged directly between kids’ devices via Bluetooth or messaging apps.

Ed: Do these blocked sites (like the webpage of that journal author I was trying to access) get notified that they have been blocked and have a chance to appeal? Would a simple solution to over blocking simply be to allow (e.g. sexual health, gender-issue, minority, etc.) sites to request that they be whitelisted, or apply for some “approved” certification?

Vicki: I don’t believe so. There are whitelisted sites, indeed that was a key outcome of an early inquiry into ‘over-blocking’ by the UK Children’s Council on Internet Safety. But in order for this to be a sufficient response, it would be necessary for all sites and apps that are subject to filtering to be notified, to allow for possible appeal. The Open Rights Group provide a tool that allows site owners to check the availability of their sites, but there is no official process for seeking whitelisting or appeal.

Ed.: And what about age verification as an alternative? (however that is achieved / validated), i.e. restricting content before it is indexed, rather than after?

Andy: To evaluate this we would need to conduct a randomised controlled trial where we tested how the application of age verification for different households, selected at random, would relate (or not) to young people encountering potentially aversive content online.

Vicki: But even if such a study could prove that age verification tools were effective in restricting access to underage Internet users, it’s not clear this would be a desirable scenario. It makes most sense for content that is illegal to access below a certain age, such as online gambling or pornography. But if content is age-gated without legal requirement, then it could prove a very restrictive tool, removing the possibility of parental discretion and failing to make any allowances for the sorts of differences in ability or maturity between children that I pointed out at the beginning.

Ed: Similarly to the the arguments over Google making content-blocking decisions (e.g. over the “right to forget”): are these filtering decisions left to the discretion of ISPs / the market / the software providers, or to some government dept / NGO? Who’s ultimately in charge of who sees what?

Vicki: Obviously when it comes to content that is illegal for children or adults to access then broad decisions about the delineation of what is illegal falls to governments and is then interpreted and applied by private companies. But when it comes to material that is not illegal, but just deemed harmful or undesirable, then ISPs and social media platforms are left to decide for themselves how to draw the boundaries and then how to apply their own policies. This increasing self-regulatory role for what Jonathan Zittrain has called ‘private sherriffs’ is often seen a flexible and appropriate response, but it does bring reduced accountability and transparency.

Ed.: I guess it’s ironic with all this attention paid to children, that we now find ourselves in an information environment where maybe we should be filtering out (fake) content for adults as well (joke..). But seriously: with all these issues around content, is your instinct that we should be using technical fixes (filtering, removing from indexes, etc.) or trying to build reflexivity, literacy, resilience in users (i.e. coping strategies). Or both? Both are difficult.

Andy: It is as ironic as it is tragic. When I talk to parents (both Vicki and I are parents) I hear that they have been let down by the existing advice which often amounts to little more than ‘turn it off’. Their struggles have nuance (e.g. how do I know who is in my child’s WhatsApp groups? Is snapchat OK if they’re just using it amongst best friends?) and whilst general broad advice is heard, this more detailed information and support is hard for parents to find.

Vicki: I agree. But I think it’s inevitable that we’ll always need a combination of tools to deal with the incredible array of content that develops online. No technical tool will ever be 100% reliable in blocking content we don’t want to see, and we need to know how to deal with whatever gets through. That certainly means having a greater social and political focus on education but also a willingness to consider that building resilience may mean exposure to risk, which is hard for some groups to accept.

Every element of our strategy should be underpinned by whatever evidence is available. Ultimately, we also need to stop thinking about these problems as technology problems: fake news is just as much a feature of increasing political extremism and alienation just as online pornography is a feature of a heavily sexualised mainstream culture. And we can be certain: neither of these broader social trends will be resolved by simple efforts to block out what we don’t wish to see.

Read the full article: Przybylski, A. and Nash, V. (2017) Internet Filtering Technology and Aversive Online Experiences in Adolescents. Journal of Pediatrics. DOI: http://dx.doi.org/10.1016/j.jpeds.2017.01.063

Andy Przybylski and Vicki Nash were talking to blog editor David Sutcliffe.

However, a number of open science initiatives have been successful recently in raising awareness of the benefits of open science and encouraging public sharing of datasets. These are discussed by Malte Elson (Ruhr University Bochum) and the OII’s Andrew Przybylski in their special issue editorial: “The Science of Technology and Human Behavior: Standards, Old and New”, published in the Journal of Media Psychology. What makes this issue special is not the topic, but the scientific approach to hypothesis testing: the articles are explicitly confirmatory, that is, intended to test existing theory.

All five studies are registered reports, meaning they were reviewed in two stages: first, the theoretical background, hypotheses, methods, and analysis plans of a study were peer-reviewed before the data were collected. The studies received an “in-principle” acceptance before the researchers proceeded to conduct them. The soundness of the analyses and discussion section were reviewed in a second step, and the publication decision was not contingent on the outcome of the study: i.e. there was no bias against reporting null results. The authors made all materials, data, and analysis scripts available on the Open Science Framework (OSF), and the papers were checked using the freely available R package statcheck (see also: www.statcheck.io).

All additional (non-preregistered) analyses are explicitly labelled as exploratory. This makes it easier to see and understand what the researchers were expecting based on knowledge of the relevant literature, and what they eventually found in their studies. It also allows readers to build a clearer idea of the research process and the elements of the studies that came as inspiration after the reviews and the data collection were complete. The issue provide a clear example of how exploratory and confirmatory studies can coexist — and how science can thrive as a result. The articles published in this issue will hopefully serve as an inspiration and model for other media researchers, and encourage scientists studying media to preregister designs and share their data and materials openly.

Media research — whether concerning the Internet, video games, or film — speaks directly to everyday life in the modern world. It affects how the public forms their perceptions of media effects, and how professional groups and governmental bodies make policies and recommendations. Empirical findings disseminated to caregivers, practitioners, and educators should therefore be built on an empirical foundation with sufficient rigor. And indeed, the promise of building an empirically-based understanding of how we use, shape, and are shaped by technology is an alluring one. If adopted by media psychology researchers, this approach could support rigorous testing and development of promising theories, and retirement of theories that do not reliably account for observed data.

The authors close by noting their firm belief that incremental steps taken towards scientific transparency and empirical rigor — with changes to publishing practices to promote open, reproducible, high-quality research – will help us realize this potential.

We caught up with the editors to find out more about preregistration of studies:

Ed.: So this “crisis“ in psychology (including, for example, a lack of reproducibility of certain reported results) — is it unique to psychology, or does it extend more generally to other (social) sciences? And how much will is there in the community to do something about it?

Andy: Absolutely not. There is strong evidence in most social and medical sciences that computational reproducibility (i.e. re-running the code / data) and replicability (i.e. re-running the study) are much lower than we might expect. In psychology and medical research there is a lot of passion and expertise focused on improving the evidence base. We’re cautiously optimistic that researchers in allied fields such as computational social science will follow suit.

Malte: It’s important to understand that a failure to successfully replicate a previous finding is not a problem for scientific progress. Quite contrary, it tells us that previously held assumptions or predictions must be revisited. The number of replications in psychology’s flagship journals is still not overwhelming, but the research community has begun to value and incentivize this type of research.

Ed.: It’s really impressive not just what you’ve done with this special issue (and the intentions behind it), but also that the editor gave you free reign to do so — and also to investigate and report on the state of that journal’s previously published articles, including re-running stats, in what you describe as an act of “sincere self-reflection” on the part of the journal. How much acceptance is there in the field (psychology, and beyond) that things need to be shaken up?

Malte: I think it is uncontroversial to say that, as psychologists adapt their research practices (by preregistering their hypotheses, conducting high-powered replications, and sharing their data and materials) the reliability and quality of the scientific evidence they produce increases. However, we need to be careful not to depreciate the research generated before these changes. But that is exactly what science can help with: Meta-scientific analyses of already published research, be it in our editorial or elsewhere, provide guidance how it may (or may not) inform future studies on technology and human behavior.

Andy: We owe a lot to the editor-in-chief Nicole Krämer and the experts that reviewed submissions to the special issue. This hard work has helped us and the authors deliver a strong set of studies with respect to technology effects on behaviour. We are proud to say that registered reports is now a permanent submission track at the Journal of Media Psychology and 35 other journals. We hope this can help set an example for other areas of quantitative social science which may not yet realise they face the same serious challenges.

Ed: It’s incredibly annoying to encounter papers in review where the problem is clearly that the study should have been designed differently from the start. The authors won’t start over, of course, so you’re just left with a weak paper that the authors will be desperate to offload somewhere, but that really shouldn’t be published: i.e. a massive waste of everyone’s time. What structural changes are needed to mainstream pre-registration as a process, i.e. for design to be reviewed first, before any data is collected or analysed? And what will a tipping point towards preregistration look like, assuming it comes?

Andy: We agree that this is experience is aggravating as researchers invested in both the basic and applied aspects of science. We think this might come down to a carrot and stick approach. For quantitative science, pre-registration and replication could be a requirement for articles to be considered in the Research Exercise Framework (REF) and as part of UK and EU research council funding. For example, the Wellcome Trust now provides an open access open science portal for researchers supported by their funding (carrots). In terms of sticks, it may be the case that policy makers and the general public will become more sophisticated over time and simply will not value work that is not transparently conducted and shared.

Ed.: How aware / concerned are publishers and funding bodies of this crisis in confidence in psychology as a scientific endeavour? Will they just follow the lead of others (e.g. groups like the Center for Open Science), or are they taking a leadership role themselves in finding a way forward?

Malte: Funding bodies are arguably another source of particularly tasty carrots. It is in their vital interest that funded research is relevant and conducted rigorously, but also that it is sustainable. They depend on reliable groundwork to base new research projects on. Without it funding becomes, essentially, a gambling operation. Some organizations are quicker than others to take a lead, such as the The Netherlands Organisation for Scientific Research (NWO), who have launched a Replication Studies pilot programme. I’m optimistic we will see similar efforts elsewhere.

Andy: We are deeply concerned that the general public will see that science and scientists are missing a golden opportunity to correct itself and ourselves. Like scientists, funding bodies are adaptive and we (and others) speak directly to them about these challenges to the medical and social sciences.The public and research councils invest substantial resources in science and it is our responsibility to do our best and to deliver the best science we can. Initiatives like the Center for Open Science are key to this because they help scientists build tools to pool our resources and develop innovative methods for strengthening our work.

Ed.: I assume the end goal of this movement to embed it in the structure of science as-it-is-done? i.e. for good journals and major funding bodies to make pre-registration of studies a requirement, and for a clear distinction to be drawn between exploratory and confirmatory studies? Out of curiosity, what does (to pick a random journal) Nature make of all this? And the scientific press? Is there much awareness of preregistration as a method?

Malte: Conceptually, preregistration is just another word for how the scientific method is taught already: Hypotheses are derived from theory, and data are collected to test them. Predict, verify, replicate. Matching this concept by a formal procedure on some organizational level (such as funding bodies or journals) seems only consequential. Thanks to scientists like Chris Chambers, who is promoting the Registered Reports format, confidence that the number of journals offering this track will ever increase is warranted.

Andy: We’re excited to say that parts of these mega-journals and some science journalists are on board. Nature: Human Behavior now provides registered reports as a submission track and a number of science journalists including Ed Yong (@edyong209), Tom Chivers (@TomChivers), Neuroskeptic (@Neuro_Skeptic), and Jessie Singal (@jessesingal) are leading the way with critical and on point work that highlights the risks associated with the replication crisis and opportunities to improve reproducibility.

Ed.: Finally: what would you suggest to someone wanting to make sure they do a good study, but who is not sure where to begin with all this: what are the main things they should read and consider?

Andy: That’s a good question; the web is a great place to start. To learn more about registered reports and why they are important see this, and to learn about their place in robust science see this. To see how you can challenge yourself to do a pre-registered study and earn $1,000 see this, and to do a deep dive into open scientific practice see this.

Malte: Yeah, what Andy said. Also, I would thoroughly recommend joining social networks (Twitter, or the two sister groups Psychological Methods and PsychMAP on Facebook) where these issues are lively discussed.

Ed.: Anyway .. congratulations to you both, the issue authors, and the journal’s editor-in-chief, on having done a wonderful thing!

Malte: Thank you! We hope the research reports in this issue will serve as an inspiration and model for other psychologists.

Andy: Many thanks, we are doing our best to make the social sciences better and more reliable.

Read the full editorial: Elson, M. and Przybylski, A. (2017) The Science of Technology and Human Behavior: Standards, Old and New. Journal of Media Psychology. DOI: 10.1027/1864-1105/a000212

Malte Elson and Andrew Przybylski were talking to blog editor David Sutcliffe.