The Design of Project AWeSome

Project AWeSome is a multimethod project that started off in 2018 to investigate the psychosocial consequences of adolescents’ social media use. The project consists of several subprojects, including interviews and focus groups among 30 adolescents, a national representative survey among 1,000 adolescents, and a pilot experience sampling method (ESM) study among 63 students.

The flagship subproject of Project AWeSome is a longitudinal cohort study that ran from November 2019 to June 2020. During this period, a cohort of 388 middle adolescents were intensively investigated with different research methods, including online surveys, ESM surveys, digital trace data collection, and Instagram archive downloads. 

  • How was Project AWeSome set up?

    Project AWeSome consists of the following subprojects.

    • Step 1: Interviews

      In a first step, we interviewed 30 adolescents. These interviews provided initial insights into the terms and expressions used by adolescents to describe their social media activities and emotional experiences.

      The qualitative interviews also gave us a feel for how adolescents use social media. We learned, for example, that (a) Facebook was “for older generations”, that (b) each platform was used differently by different adolescents, and that (c) many adolescents considered and used gaming as a social medium.

    • Step 2: A national survey

      In a second step, we conducted a national representative survey among 1,000 Dutch adolescents, which was fielded by a national survey institute [1]. This national survey functioned as a benchmark for the experience sampling method (ESM) studies and surveys.

      We also used the national survey to identify the most popular social media among Dutch adolescents and to investigate the frequency with which they used these social media.

      Finally, we obtained insight in the frequency per activity on each platform. Such insights are important to understand what can be measured in an ESM study. For instance, it is not feasible to measure social media activities that occur too infrequently in an ESM study.

      [1] van Driel, I.I., et al., 'Posten, scrollen, appen en snappen': Jongeren (14-15 jaar) en social media in 2019. 2019, Universiteit van Amsterdam: Center for Research on Children, Adolescents, and the Media (CcaM).

    • Step 3: A pilot study

      In a final step, we developed the ESM measures, based on the insights from the interviews, the national survey, and theoretical and empirical insights from earlier studies.

      In a study among 63 Dutch adolescents, we piloted our ESM measures, the study procedure, the sampling scheme, and monitoring procedure. This pilot study yielded reliable data and was published in Nature Scientific Reports [1]. The data of this study are available on Figshare.  

      [1] Beyens, I., et al., The effect of social media on well-being differs from adolescent to adolescent. Scientific Reports, 2020. 10: p. 10763.

    • Step 4: A longitudinal cohort study

      The longitudinal cohort study ran from November 2019 to June 2020. During this period, a group of 388 middle adolescents (13- to 15-year-olds) were intensively investigated with different research methods. See the question below for a more detailed explanation on the  different research parts. 

  • What are the different research parts of the longitudinal cohort study?

    The longitudinal cohort study consists of a measurement burst design. Such a design, which was first described by Nesselroade (1991), combines two types of data: (1) longitudinal data with widely spaced intervals (e.g., months, years), and (2) longitudinal data with closely spaced intervals (e.g., hours, days), which are, for example, obtained via ESM or social media data downloads. These second types of data are called "measurement bursts.”  The longitudinal cohort study comprises the following five research parts:

    • 1. A first three-week experience sampling (ESM) study

      In this ESM study, which was fielded in November and December 2019, each adolescent was asked to fill out six daily ESM surveys, totaling to 126 per adolescent. Adolescents completed the ESM surveys using the Ethica Data App on their own smartphone. We measured adolescents’ social media activities, their self-esteem, loneliness, well-being, distractibility, and their closeness to their friends.

    • 2. Sixteen biweekly surveys

      From November 2019 to June 2020, we administered 16 biweekly surveys. These surveys included multiple state and trait measures, for example of well-being, friendship support, and parenting.

      Some of these survey measures are planned to be included as moderators in our models, for instance, to investigate which adolescents are more susceptible to the effects of social media activities than others. Other measures will be used to investigate the longer-term effects of social media use.

    • 3. A second three-week ESM study

      In the second three-week ESM study, which was fielded in June 2020, we again asked adolescents to fill out six daily ESM surveys, totaling to 126 measurements per person.

      Some of the measures of the second ESM wave, such as well-being and self-esteem, were identical to the first ESM wave. Others were different. In comparison to the first ESM wave, the second one was focused more on self-regulation and parent-child communication.

    • 4. Digital trace data

      During the second ESM wave, we collected digital trace data of adolescents’ app usage and screen time among a subsample of Android users using the Ethica App Usage Stream and Ethica Data App.

    • 5. Instagram data donation

      A subsample of adolescents shared their Instagram archives, which include their posts, stories, and comments.

  • Can we see a flowchart of the cohort study’s design?

    The figure below shows how the data collection process of the cohort study was organized.

    measure burst design

    Nesselroade, J.R. (1991), The warp and the woof of the developmental fabric, in Visions of aesthetics, the environment & development: The legacy of Joachim F. Wohlwill. Erlbaum Hillsdale, NJ. p. 213-240.

  • Is the sample of the cohort study large enough?

    The power of an ESM study does not only depend on the number of participants, but also on the number of within-person assessments. Compared to most earlier ESM studies among both adolescents and adults, our sample size and sampling scheme is large and highly intensive.

    Even though a sample of 50-120 participants with 50 assessments is typical in ESM studies among adolescents [1] and adults [2], we sampled 388 adolescents with 126 assessments in each ESM wave (21 days x 6 assessments per wave; 252 in total).

    This large number of assessments enables us to conduct N=1 time series analyses to estimate the effects of social media use among each individual adolescent. More details about the sample size estimation can be found in our power analyses.

    [1] van Roekel, E., L. Keijsers, and J.M. Chung, A review of current ambulatory assessment studies in adolescent samples and practical recommendations. Journal of Research on Adolescence, 2019. 29(3): p. 560-577.

    [2] Uy, M.A., M.D. Foo, and H. Aguinis, Using experience sampling methodology to advance entrepreneurship: Theory and research. Organizational Research Methods, 2009. 13(1): p. 31-54

  • How representative is the sample of the cohort study?

    Our sample was recruited from all the 2nd and 3rd grades of a large Dutch secondary school. This school was carefully selected in terms of heterogeneity in educational level (which is highly correlated with SES in the Netherlands).

    Our sample is representative of Dutch youth of the south of the Netherlands in terms of educational level, sex, and ethnicity [1]. The social media habits of our sample are also comparable to those found in our national survey study [2]. Details on the recruitment procedures can be found on OSF.

    [1] Statistics Netherlands. Kerncijfers wijken en buurten 2020 [StatLine]. 2020; Available from: https://www.cbs.nl/nl-nl/maatwerk/2020/29/kerncijfers-wijken-en-buurten-2020.

    [2] van Driel, I.I., et al., 'Posten, scrollen, appen en snappen': Jongeren (14-15 jaar) en social media in 2019. 2019, Universiteit van Amsterdam: Center for Research on Children, Adolescents, and the Media (CcaM).

  • What are the different research lines in the project?

    Project AWeSome employs an integrative approach to investigate the consequences of social media use for different aspects of adolescents’ psychosocial functioning.

    Our ongoing research lines focus on the effects of social media use on adolescents’ well-being (e.g., affective well-being, self-esteem, loneliness), social relationships (e.g., friendship closeness, parent-child communication), and self-regulation (e.g., attentional control, behavioral control).

    An overview of our preprints and publications can be found here.

  • What is Project AWeSome’s take on open science?

    Project AWeSome is strongly committed to open science and adheres to the recommendations of the International Communication Association’s Code of Ethics and the FAIR Data Principles.

    Recognizing that transparency is key, our commitment to open science is reflected in our open science practices. These practices include (1) publication of materials, (2) preregistration of studies, (3) publication of preprints, (4) open access publication, and (5) publication of datasets.

    • Open materials

      All steps in the design, sampling, and data collection procedures of the project are published and made available online. All materials and code are published at OSF.

    • Preregistrations

      All studies, including their hypotheses, research questions, and analysis plans are preregistered at OSF before testing the hypotheses and conducting data analyses related to the research plan.

      Any existing insights into the data are fully disclosed in the preregistrations (e.g., assessment of psychometric properties or insights based on analyses for other manuscripts). And any deviations from the preregistrations (e.g., non-feasibility of the preregistered analysis plan, unforeseen circumstances) are fully disclosed in our publications.

    • Preprints on PsyArXiv

      As an additional open science practice, preprints of our papers are published online at PsyArXiv and made available on our website and/or ResearchGate.

    • Open Access Publication

      An overview of all publications and preprints that resulted from our project can be found at our website. All our journal articles are published open access and are made freely available online.

    • Publication of datasets

      Finally, together with each (published) paper, we upload an anonymized dataset on Figshare. This enables fellow researchers to reproduce and verify our analyses and replicate our studies.

      We publish our data under a restrictive license. This means that the data can be accessed but may not be reused without our explicit permission.

This project is funded
by a NWO Spinoza Prize awarded to Patti Valkenburg

NWO-logo_-_witruimte.pnguva logo